Storage container structure¶
To support multitenancy, each tenant (customer) has its own storage container. Each storage container contains individual datasets. Each dataset is defined by unique and static id
Dataset structure¶
- Dataset root folder contains files with dataset definition and optional spatial indexes
- Subfolders contains data in different formats
- Raw contains the data in raw format , reorganized by spatial index
- Other subfolders can contain for example generated tiles, ect.
- Each subfolder is further organized by item and layer
- The data blobs contain data organized by the selected strategy
Dataset metadata¶
Each dataset folder contains dataset.json blob with full metadata description of the dataset including
- dataset identification and version
- spatial domain with optional vertical domain definition (see Spatial Domain storage)
- temporal domain definition
- items definitions
- item storage options (see Item data storage)
Dataset.json sample:
{
"Id": "efae3d23-7d0f-4073-ae4b-8e6678ba9d08",
"Name": "2018010100F000",
"SchemaVersion": "2019-09-30",
"SpatialDomain": {
"Type": "EquidistantGridD2",
"ElementCount": 8817120,
"Dy": 0.083335876,
"Ny": 2041,
"Y0": -80.0416667,
"Dx": 0.083328247,
"Nx": 4320,
"X0": -180.0416667,
"SRID": 4326,
"VerticalDomain": {
"Name": "VerticalDomain",
"UnitId": "eumUmeter",
"Datum": { "Origin": 0.49402499 },
"Orientation": "Down",
"MaxZ": [],
"Transformation": {
"Depths": [ 0.49402499, 1.541375, 2.645669, 3.819495, 5.0782242, 6.4406142, 7.9295602, 9.5729971, 11.405, 13.46714, 15.81007, 18.49556, 21.59882, 25.211411, 29.444731, 34.434151, 40.344051, 47.373692, 55.76429, 65.807266, 77.853851, 92.326073, 109.7293, 130.666, 155.85069, 186.1256, 222.4752, 266.04031, 318.12741, 380.21301, 453.93771, 541.08893, 643.56677, 763.33313, 902.33929, 1062.4399, 1245.291, 1452.251, 1684.2841, 1941.8929, 2225.0779, 2533.3359, 2865.7029, 3220.8201, 3597.032, 3992.4839, 4405.2241, 4833.291, 5274.7842, 5727.917 ],
"Name": "NonEquidistantTransformation"
},
"LayerCount": 50,
"HasTimeVariableValues": false
}
},
"TemporalDomain": {
"Stamps": [ "2018-01-01T12:00:00" ],
"StartTime": "2018-01-01T12:00:00",
"EndTime": "2018-01-01T12:00:00",
"Count": 1
},
"Items": [
{
"StoredType": "Int16",
"StoredNoDataValue": -32767,
"ValueConverter": {
"ScaleFactor": 0.00061037018895149231,
"ScaleOffset": 0.0
},
"NoDataValue": -19.999999981373549,
"Name": "Eastward velocity",
"Quantity": {
"ItemId": "eumIBedThickness",
"UnitId": "eumUmeter"
},
"Type": "Double",
"ItemType": "ConvertibleItem`2",
"ConverterType": "Scaled"
}
],
"ItemStorage": {
"UseSpatialIndex": true,
"TimeGroupSize": 0,
"StorageProvider": "SingleTimestep"
}
}