Skip to content

Simple file upload and download

The recommended upload and download can be executed using the fluent API of the Platform SDK as described in SDK / How-to guides / Upload files and SDK / How-to guides / Download files.

There is yet another option to upload and downloa plain files using the Platform SDK, as follows: 

ITransferClient transferClient; // see SDK on how to get the transfer client

var filePath = "c:/Data/EqD2.dfs2";
var projectId = new Guid("5df363fa-10e1-4fc0-8eec-9e145cbd4746");

var dataset = await _transferClient.UploadAsync(projectId, filePath);

var outputFilePath = "c:/Data/downloadedEqD2.dfs2";
await _transferClient.DownloadAsync(projectId, dataset.Id, outputFilePath);

if(File.Exists(outputFilePath))
    Console.WriteLine($"Downloaded {outputFilePath}");

When it comes to interaction with the REST API, there are several options as described in the Conversion pipeline and Transfers section. However, for uploading simple files (meaning files that do not have to be imported into any special storage, do not have to be processed in any way) the options descibed in the mentioned section are an overkill and they can be cumbersome to use. For these cases a simpler, more straightforward API is available. It is only usable for uploading files without processing, without metadata or properties.

Endpoints

  • POST/api/conversion/project/{id}/file/upload - allows directly posting a file (file content is part of the request). This endpoint can be used for smaller files (up to approx. 30 MB). Larger files need to be staged first.

  • POST/api/conversion/project/{id}/file/import-staged - allows importing files which have been previously staged in the Raw service. The process is that first the caller gets a Url that can be used to stage a file (via GET/api/conversion/transfer/upload-url), uploads the file there, and then uses the same url (or multiple of urls) in a POST request to this endpoint. The files are then imported into the platform.

Both these endpoints accept a DestinationPath specification. DestinationPath is a relative path from the provided project (specified by projectId in the path) where the file(s) should end up. If the parameter CreateDestinationPathIfNotExists is set to true, then the folders on the path are created for the user if they do not exist (with default settings, such as Shared access level and the calling user as the owner).

Authentication/authorization

Multiple options exist on how to authenticate when calling these endpoints

  • AD registration: the caller is registered in the AD, has the right to call the Metadata service, and provides the bearer token and custom DHI headers

  • OpenAPI token: the caller provides the OpenAPI token in a header with the request. No other information is necessary, as the token identifies the caller and authenticates him. The authorization happens based on the user's current privileges recorded in the system.

  • Platform SAS token: the caller provides a Platform SAS token (issued by the Metadata service, using one of the endpoints described here). The privileges are taken from the token only. So if the DestinationPath is to be used, it might not work for a token that is generated for the given project only. For these cases the token should be generated as "recursive".

Sample

curl -X GET "https://api.mike-cloud-test.com/api/services/recursivesastoken?projectId=<projectId>" -H "Authorization: Bearer <bearer token>"

Will return a sas token (recursive) for the given project. With that, the following request will upload a file:

curl -X POST "https://api.mike-cloud-test.com/​/api​/conversion​/project/​<projectId>​/file​/upload?<append sas token here into query string> "

-F "File=@temperature.dfs2;type=text/plain" <---------- which file to send
-F "FileName=temperature2.dfs2"             <---------- how the file is named
-H "Content-Type: multipart/form-data"      <---------- header to indicate how the data is sent
-H "api-version: 2"                         <---------- header to indicate web api version
-F "DestinationPath=myfolder/mysubfolder"   <---------- destination path
-F "CreateDestinationPathIfNotExists=True"  <---------- to create the path if it does not exist

To download a plain file, issue the following request with a valid Platform SAS token 

curl -X POST "https://api.mike-cloud-test.com/api/raw/dataset/<datasetId>?<append sas token here into query string>"
-H "Content-Type: application/json"         <---------- header to indicate how the data is sent
-H "dhi-service-id: raw"                    <---------- header to indicate underlying service
-H "api-version: 2"                         <---------- header to indicate web api version

This will return a json object like: 

{
    "data" : "<SAS URL for download of the file>"
}

Then to finally download the file as output.file.name to client's hard drive, run: 

curl -X GET "<SAS URL for download of the file>" -o output.file.name