Create timeseries oriented dataset
Dataset optimized for timeseries queries for discrete points must be imported with specific parameters.
Parameter | Value | Notes |
---|---|---|
StorageType |
TimeSeries (string) | Mandatory |
TimeGroupSize |
Number of steps per block (int) | Mandatory , for variable blocks it is the max size, |
SpatialGroupSize |
Number of spatial elements per block (int) | Mandatory, more details below |
TimeGroupFixedStep |
Number of seconds between steps (int) | Optional, if set fixed blocks will be created, otherwise blocks have variable size |
Read Data storage options description for more information.
Use case - Historical timeseries with variable block size¶
Example : long historical data, e.g 40 years of data history in metocean datasets.
Prepare writer parameters¶
var writerParams = new ParameterInput[]
{
new ParameterInput {Name="StorageType",Value="TimeSeries"},
new ParameterInput {Name="TimeGroupSize",Value="integer value"},
new ParameterInput {Name="SpatialGroupSize",Value="integer value"}
};
TimeGroupSize should be bigger than the number of timesteps in 1 imported file e.g in 1 year to avoid data splitting.
Example :
Timestep Interval : 1 hour
Each imported file represents 1 year of data
Set TimeGroupSize > 365*24.
Use case - Forecast flow with fixed block size¶
For a forecast flow with regular schedule and moving window size, set the TimeGroupSize to the window size or bigger (e.g. the whole forecast period).
When new forecast is ready, the update operation is invoked and appropriate time blocks will be replaced.
The TimeGroupFixedStep
parameter must be provided to use fixed blocks.
var writerParams = new ParameterInput[]
{
new ParameterInput {Name="StorageType",Value="TimeSeries"},
new ParameterInput {Name="TimeGroupFixedStep",Value="integer value"}
new ParameterInput {Name="TimeGroupSize",Value="integer value"},
new ParameterInput {Name="SpatialGroupSize",Value="integer value"}
};
SpatialGroupSize parameter¶
- if the Timegroup size is small (~< 1000) then set SpatialGroupSize >= number of elements in spatial domain
- Otherwise ensure that
SpatialGroupSize * TimeGroupSize * (4 for float or 8 for double) < 2GB
There are extra considerations when importing 3D DFSU file into Timesteries oriented storage. This example is for 3D DFSU file with hourly time step and 10000 spatial elements:
- TimeGrouSize = 24 (for one day in each group = 24*TimeGroupFixedStep)
- SpatialGroupSize = 1000 (set it one order lower than for 2D data, because the 3D data includes information about all layers)
- TimeGroupFixedStep = 3600 (one hour between time steps within each group)
Import new dataset with writer parameters¶
var transfer = transferClient.CreateFileImport(_projectId, "sampleTS.dfsu")
.WithReader(new DfsuReader())
.WithWriter(new GenericWriter("MDWriter", writerParams))
Note: Dfsu is just an example reader.