Skip to content

Create timeseries optimized dataset

Dataset optimized for timeseries queries for discrete points must be imported with specific parameters.

Parameter Value Notes
StorageType TimeSeries (string) Mandatory
TimeGroupSize Number of steps per block (int) Mandatory , for variable blocks it is the max size,
SpatialGroupSize Number of spatial elements per block (int) Mandatory, more details below
TimeGroupFixedStep Number of seconds between steps (int) Optional, if set fixed blocks will be created, otherwise blocks have variable size

Read Data storage options description for more information.

Use case - Historical timeseries with variable block size

Example : long historical data, e.g 40 years of data history in metocean datasets.

Prepare writer parameters

var writerParams = new ParameterInput[]
{
    new ParameterInput {Name="StorageType",Value="TimeSeries"},
    new ParameterInput {Name="TimeGroupSize",Value="integer value"}, 
    new ParameterInput {Name="SpatialGroupSize",Value="integer value"} 
};

TimeGroupSize should be bigger than the number of timesteps in 1 imported file e.g in 1 year to avoid data spliting.

Example : 

Timestep Interval : 1 hour 
Each imported file represents 1 year of data

Set TimeGroupSize > 365*24.

Use case - Forecast flow with fixed block size

For a forecast flow with regular schedule and moving window size, set the TimeGroupSize to the window size or bigger (e.g. the whole forecast period).

image.png

When new forecast is ready , the update operation is invoked and appropriate time blocks will be replaced.

The TimeGroupFixedStep parameter must be provided to use fixed blocks.

var writerParams = new ParameterInput[]
{
    new ParameterInput {Name="StorageType",Value="TimeSeries"},
    new ParameterInput {Name="TimeGroupFixedStep",Value="integer value"} 
    new ParameterInput {Name="TimeGroupSize",Value="integer value"}, 
    new ParameterInput {Name="SpatialGroupSize",Value="integer value"} 
};

SpatialGroupSize parameter

  • if the Timegroup size is small (~< 1000) then set SpatialGroupSize >= number of elements in spatial domain
  • Otherwise ensure that SpatialGroupSize * TimeGroupSize * (4 for float or 8 for double) < 2GB

Import new dataset with writer parameters

var transfer = transferClient.CreateFileImport(_projectId, "sampleTS.dfsu")
        .WithReader(new DfsuReader())
        .WithWriter(new GenericWriter("MDWriter", writerParams))

Note: Dfsu is just an example reader.