File storage Concepts

A dataset in the File storage service is an ordinary file in its raw form. No transformation or processing is applied when users import files from their computers to the File storage. The files in raw storage are actually stored in a relatively flat structure, the file location, i.e. path, is represented by the corresponding metadata dataset record in the metadata subproject hirarchy. The Platform does not know anything about the internal stucture of the files, it often does not know even the type of file by extension. Therefore, unlike services such as GIS, TimeSeries, or Multidimensoal, the File storage does not offer any query or update capabilities. Users can use the files to their best knowledge of the content.

Files can be accessed by requesting a SAS URL (a url with extra secutirty token in query parameters) and downloading content of the SAS URL to local hard drive. This url can be obtained by calling GET/api/raw/dataset/{id}.

Direct access over file protocol or the SMB protocol to the files in the File storage service is not possible.

The File storage is very popular for storing input files for an engine excecution and then for storing the execution result files again.

Several endpoints of the File storage service are dedicated to interaction with desktop clients which want to synchronize files between the client's computer and the File storage. These endpoints allow checking content of individual files by checksum and also operations on multiple files.

Endpoints to prepare upload and prepare download, as well as endpoints for staging files during transfer or conversion process are intended primarily for internal use by the Platform or in the Platform SDK.