Overview
Time series service in the platform provides storage for time series data organized in TimeSeriesDataset
. TimeSeriesDataset
contains collection of ITimeSeries
which represent individual time series. There are multiple implementations of ITimeSeries
interface to deal with differences between time series definition, time series with data, and time series with schema definition. All time series in a dataset have the same TimeSerieProperty
that define name and type of properties used across all time series in the dataset. Each time series in a dataset also includes DataFields
, which define the data type for the time series data - item and flags, represented by TimeSeriesDataFrame
. TimeSeriesDataFrame
is a table-like structure that includes time series data - date time values, the item (the variable of interest), and a number of flags.
Item ITimeSeriesItem
in the time series service inherits from Platform general model IItem
and includes:
- TimeSeriesType (of type DHI.Services.TimeSeries.TimeSeriesDataType)
- Quantity
- Type (as in data type)
- Name
Compatibility with Domain Services¶
Internal structures used in the platform Time series service are different from Domain Services, but interoperability at C# level can be achieved using the .AsTimeSeriesData()
extension method of TimeSeriesDataFrame
class. At web api level the json serialization also matches the format used by Domain Services with minor deviations:
- platform time series can contain more than one flag;
- the time series is wrapped in response object
{"data": [...]}
.
Loading of big time series data¶
In addition to a web app the platform provides times series readers and writers. These can be used to load large sets of time series from files into the time series storage using the platform transfer pipeline. This is useful for example when loading data from files exported from existing 3rd party system into the platform. Current implementation supports loading for these data formats:
Format | Description |
---|---|
DFS0 | DFS0 files that are widely used with MIKE Powered by DHI software |
CSV - DIMS | CSV files exported from DIMS as described in Data format for time series pushed into MIKE Data Admin |
CSV - GHM | CSV files with many time series in one file and custom time series ids in the file header. Data are then stored in a format optimized for high number of time series with similar time steps. |
Limitations¶
- One flag can have at most 15 distinct values.
- Maximum difference between time steps is approximately 65 years, this is due to the the type of compression used in the time series storage.
- The service does not consider time zones and expects datatime in the format
yyyy-mm-ddThh:mm:ss
- TimeSeries datasets cannot store text (string) as a value or column value. The intention is to use flags instead - for instance, if you need to store text like 'raw', 'measured', 'simulated', define a numerical flag and record in timeseries properties what numerical flag value refers to the relevant string.