Reconsider data types

[nichtich]

The Table Schema tag lists several issues related to data types. These might better be discussed together. Subgroups may be:

date and times

• extend and/or more precisely define date types Clarity around dates in Table Schema #366, Time formats should be more consistent and portable across platforms #487, year-quarter time format #562, Extend specification with non default format for types yearmonth and year #659, Shall datetime with format=default support optional timezone offsets? #830

numeric types

• constraints or other details of numeric types: Exclusive minimum and maximum values are not supported as numeric constraints #856, Ability to differentiate decimal and floating point numbers #709, Add finite constraint for numbers or number.nanValues, number.infValues #644, Add decimal place constraint to number fields #641, bareNumber to interpret -$1000 as -1000 #609, Support groupChar for integer type #498

units

• add unit types Support units in Table Schema #537
• add currency type Type to represent a currency #352

compound types

• add json type Add a json field type and jsonSchema constraint support #640
• add table type Create new type table #410
• add list type Future possibility for delimiter-separated list for arrays (instead of JSON array)? #736
• add/extend geography types Add wkt to field types #772, New property crs #812 (may be related to units)

[peterdesmet]

I agree it would be easier if these were discussed as subgroups. What would be the best forum for this? Call, discussions, grouping issues, etc.?

[khusmann]

Can we also include categorical / ordinal types as its own group? Right now they're implemented as constraints / extensions on other types, but I think deserve first-class discussion. (relevant issue: #844)

[roll]

In my opinion, the most pressing issue for v2 is number type ambiguity. With a simple example like this:

schema.json

fields:
  -name: value
   type: number

data.csv

value
1.1
2.2
3.3

We get into an undefined state when we need to export data into e.g. SQL because float and numeric (decimal) types are completely different both physically and logically. Table Schema doesn't provide any information regarding this similar to JSON (and JSON really suffers from it when it comes to e.g. monetary data)

[nichtich]

See also this comparison of SQL data types.