Last updated: 2024-11-14
File API
Caution
Sample code provided in here is for demonstration and educational purposes only which provides customers with programming information regarding the products and is not intended for use in a production environment. All production code should always follow development best practises. All sample code here is supplied "AS IS" without any warranties or support.
This document explains how to use the "File API" to upload/download files to/from the "unmanaged files" in ShareAspace as well as how to delete files.
The provided C# code samples below uses HttpClient to make the REST calls.
Warning
A common mistake when using HttpClient is to use multiple instances of the
HttpClient. You should reuse the same instance within an application.
Unmanaged file areas
There are several unmanaged file areas that you may freely add files and directories to. They are listed below and the URLs to access these areas via REST API.
| Area | URL |
|---|---|
| Collection | /api/collection/fs/file |
| Space | /api/space/[spaceId]/fs/file |
| User | /api/space/[spaceId]/user/fs/fileNote: The actual user is not in the URL but is who you are authenticated as |
| Participant/[Id] | /api/space/[spaceId]/participant/[participantId]/fs/fileWhere 'participantId' is a valid participant in ShareAspace |
The URLs above can in some cases have additional fields appended depending on the type of transfer you are doing and your file-path and file-name.
Authentication
To do any file API operations you must always first get an authentication token to be used in subsequent file API calls.
Below is a re-useable function for authentication that is used in all the following code sections.
CSharp
This code is used in all the other examples of code in this page.
First there is an implementation of a BasicAuthenticationHeaderValue
that is then used in the actual Authentication.
BasicAuthenticationHeaderValue
using System;
using System.Net.Http.Headers;
using System.Text;
namespace Demo.FileApi
{
public class BasicAuthenticationHeaderValue : AuthenticationHeaderValue
{
public BasicAuthenticationHeaderValue(string clientId, string clientSecret)
: base("Basic", EncodeCredential(clientId, clientSecret))
{
}
private static string EncodeCredential(string clientId, string clientSecret)
{
return Convert.ToBase64String(
Encoding.UTF8.GetBytes(string.Format("{0}:{1}", new object[2]
{
clientId,
clientSecret
})));
}
}
}
Authentication
using Newtonsoft.Json.Linq;
using System.Collections.Generic;
using System.Net.Http;
namespace Demo.FileApi
{
public static partial class Program
{
public static JObject Authenticate(string baseAddress, string userName, string password, string clientId, string clientSecret)
{
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new BasicAuthenticationHeaderValue(clientId, clientSecret);
var form = new Dictionary<string, string>
{
{"grant_type", "password"},
{"username", userName},
{"password", password},
{"scope", "All" }
};
HttpContent content = new FormUrlEncodedContent(form);
string requestUri = $"{baseAddress}/AuthorizationServer/sasweb/oauth/token";
HttpResponseMessage response = client.PostAsync(requestUri, content).Result;
response.EnsureSuccessStatusCode();
string responseContent = response.Content.ReadAsStringAsync().Result;
JObject token = JObject.Parse(responseContent);
client.Dispose();
return token;
}
}
}
Usage would look something like the following:
public static void TestAuthenticate()
{
string baseAddress = @"https://my.machine1.net";
string userName = "MyEmail";
string password = "MyPassword";
string clientName = "ResourceOwnerClientId";
string clientSecret = "CLIENTSECRET";
JObject token = Authenticate(baseAddress, userName, password, clientName, clientSecret);
}
PowerShell
Download get-authentication.ps1
function GetAuthentication($sasHost)
{
#OAuth 2.0 settings
$username = "MyEmail"
$password = "MyPassword"
$scope = "All"
$clientId = "ResourceOwnerClientId"
$clientSecret = "CLIENTSECRET"
$grantType = "password"
$authUri = -join ("https://", $sasHost, "/AuthorizationServer/sasweb/oauth/token")
#Get bearerToken
$basic = @{
"Authorization" = ("Basic", [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(($clientId, $clientSecret -join ":"))) -join " ")
}
$authBody = @{
"grant_type" = $grantType;
"username" = $username;
"password" = $password;
"scope" = $scope
}
try{
$response = Invoke-RestMethod -Method Post -Uri $authUri -ContentType "application/x-www-form-urlencoded" -Headers $basic -Body $authBody
$global:bearerToken = $response.access_token
}
catch{
$statusCode = $_.Exception.Response.StatusCode.value__
$statusDescription = $_.Exception.Response.StatusDescription
Write-Host "$($statusDescription)($($statusCode)); Uri: $($authUri); Response:$($_.Exception)"
exit
}
$global:bearer = @{ "Authorization" = ("Bearer", $global:bearerToken -join " ") }
}
GetAuthentication "my.machine1.net"
This function puts into the PowerShell global variable space the authentication token in 2 formats. variable | usage ------|----- $global:bearerToken|Just the token in base64 format $global:bearer | The token in an object that looks like @{ "Authorization" = ("Bearer", $global:bearerToken -join " ") }. This is used only in the method Invoke-RestMethod
Upload files
When uploading single files to ShareAspace a checksum is mandatory. The logic for providing checksum is demonstrated further down. The checksum is a calculated hash based on the file to be uploaded, the checksum is added to the request as a header. The uploaded file will be validated in ShareAspace using the provided checksum.
Upload Stream
The upload stream method is at times too slow and takes too much memory but is very effective for files under 100 kB. It is also relatively easy to debug due to small amount of code which makes it very advantageous.
This stream technique requires that your URL from the section Unmanaged file areas has "/stream" appended to it as well as the destination subdirectories and file-name. If you want to put the file in the root then don't include any subdirectory. For example in the collection area with subdirectory MySubDir and file MyFile.txt.
| Area | URL |
|---|---|
| collection | /api/collection/fs/file/stream/MySubDir/MyFile.txt |
An example of the HTTPS call is the following:
POST https://essevm540.es.eurostep.com/api/space/InReachAdmin/fs/file/stream/testImport.xlsx
Content-Type: application/data
Authorization: Bearer eyJ0eXAi...
Content-Length: 9440
Below is a method that will upload a file via the stream method.
CSharp
For this to work you need to include the function in the code from the section Authentication.
using System;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography;
namespace Demo.FileApi
{
public static partial class Program
{
public static void UploadUserStoreFile(string accessToken, string baseAddress, string targetFilePath, string sourceFilePath, string spaceId)
{
using (var file = new FileStream(sourceFilePath, FileMode.Open))
{
string requestUri = $"{baseAddress}/api/space/{spaceId}/user/fs/file/stream/{targetFilePath}";
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
string contentType = MimeMapping.GetMimeMapping(sourceFilePath);
int size = (int)file.Length;
byte[] buffer = new byte[size];
file.Read(buffer, 0, size);
var hasher = new MD5CryptoServiceProvider();
byte[] hash = hasher.ComputeHash(buffer);
string checksum = BitConverter.ToString(hash).Replace("-", "").ToLower();
client.DefaultRequestHeaders.Add("checksum", checksum);
file.Position = 0;
var content = new StreamContent(file);
content.Headers.ContentType = new MediaTypeHeaderValue(contentType);
HttpResponseMessage response = client.PostAsync(requestUri, content).Result;
response.EnsureSuccessStatusCode();
client.Dispose();
}
}
}
}
Usage would look something like the following:
public static void TestUploadFileStream()
{
string baseAddress = @"https://my.machine1.net";
string userName = "MyEmail";
string password = "MyPassword";
string clientName = "ResourceOwnerClientId";
string clientSecret = "CLIENTSECRET";
string spaceId = "mytestspace";
string remoteFilePath = @"MyFolder/TestFile.txt";
string localFilePath = @"..\..\..\..\files\TestFile.txt";
JObject token = Authenticate(baseAddress, userName, password, clientName, clientSecret);
string accessToken = (string)token["access_token"];
UploadUserStoreFile(accessToken, baseAddress, remoteFilePath, localFilePath, spaceId);
}
PowerShell
Download upload-file-stream.ps1
function UploadFileStream($sasHost, $destFileUrlPath, $srcFilePath, $bearerToken)
{
<#
.SYNOPSIS
Upload moderately sized files ( less then 1.3 MB ) to ShareASpace Nova
.DESCRIPTION
This will fail if the file already exists on the server.
When uploading files 1 MB or larger this will eat up ALL memory and be very slow.
.EXAMPLE
Load a file to the Collection files in the root
UploadFileStream "essevm540.es.eurostep.com" "/api/collection/fs/file/stream/smallfile.json" "C:\Users\sassetup\Desktop\smallfile.json" $global:bearerToken
.EXAMPLE
Load a file to the Collection files in a sub-directory called AutoImport. Non-existing subdirectories created automatically
UploadFileStream "essevm540.es.eurostep.com" "/api/collection/fs/file/stream/AutoImport/smallfile.json" "C:\Users\sassetup\Desktop\smallfile.json" $global:bearerToken
.EXAMPLE
Load a file to the User files in a sub-directory called AutoImport
UploadFileStream "essevm540.es.eurostep.com" "/api/space/MySpaceName/user/fs/file/stream/AutoImport/smallfile.json" "C:\Users\sassetup\Desktop\smallfile.json" $global:bearerToken
.EXAMPLE
Load a file to the space files in a sub-directory called AutoImport
UploadFileStream "essevm540.es.eurostep.com" "/api/space/MySpaceName/fs/file/stream/AutoImport/smallfile.json" "C:\Users\sassetup\Desktop\smallfile.json" $global:bearerToken
.PARAMETER sasHost
The FQDN of your host server.
.PARAMETER destFileUrlPath
A url that contains the information on where in the unmanaged file system you want to put the file. It may include the Space name. Non-existing subdirectories created automatically
It looks like either
/api/space/[SpaceName]/[user or participant or 'nothing for space' or participant/OEM]/fs/file/stream/[subdirectory(one or more)]/[filename]
or
/api/collection/fs/file/stream/[subdirectory(ies)]/[filename]
.PARAMETER srcFilePath
Where in your local machine the file to upload exists. Full path and file name
.PARAMETER bearerToken
The authentication token received from a request for autentication.
#>
try{
$chunk = [System.IO.File]::ReadAllBytes($srcFilePath)
$hasher = New-Object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider
$hash = $hasher.ComputeHash($chunk)
$checksum = [System.BitConverter]::ToString($hash).Replace("-","").ToLower()
$headers = @{"Authorization"="Bearer $bearerToken";"checksum"=$checksum;}
$requestUri = "https://{0}{1}" -f $sasHost, $destFileUrlPath
$response = Invoke-RestMethod -Uri $requestUri -Method Post -InFile $srcFilePath -Headers $headers
}
catch{
$statusCode = $_.Exception.Response.StatusCode.value__
$statusDescription = $_.Exception.Response.StatusDescription
Write-Host "$($statusDescription)($($statusCode)); Uri: $($requestUri); Response: $($_.Exception)"
}
}
UploadFileStream "my.machine1.net" "/api/space/mytestspace/user/fs/file/stream/UploadFolder/TestFile.txt" "C:\temp\TestFile.txt" $global:bearerToken
Upload Chunk
The "Chunk" technique for uploading is the only method recommended for large files over 1.5 MB. This technique works for small files. However, it is quite slow for small files since it allocates a relatively large buffer of memory for the operation. The code is also much larger and more complex and difficult to debug.
With this "chunk" technique the URLs from the section Unmanaged file areas are unchanged. The paths for subdirectories and the file-name are included in the body of the message instead of in the URL as the "Stream" method employs.
This technique has the downside that it uses multiple HTTPS calls to ShareAspace and if this technique crashes in the middle of the execution then you will probably have a partial file in ShareAspace and there is no way of knowing that it is a partial file. You may therefore want to change the code below to delete the file from ShareAspace in the case of a failure during uploading.
An example of the HTTPS call is the following:
POST https://essevm540.es.eurostep.com/api/space/InReachAdmin/user/fs/file
Authorization: Bearer eyJ0eXAiOi...
Content-Type: multipart/form-data; boundary="49e051f4-40e2-4ea6-a982-2fd0efaf8fc7"
Host: essevm540.es.eurostep.com
Content-Length: 1293315
A re-useable function to upload files using the chunk method.
CSharp
For this to work you need to include the function in the code from the section Authentication.
using Microsoft.AspNetCore.StaticFiles;
using System;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography;
namespace Demo.FileApi
{
public static partial class Program
{
public static void UploadUserStoreFileChunked(string accessToken, string baseAddress, string targetFilePath, string sourceFilePath, string spaceId)
{
using (FileStream file = new FileStream(sourceFilePath, FileMode.Open))
{
var requestUri = $"{baseAddress}/api/space/{spaceId}/user/fs/file";
string fileName = Path.GetFileName(sourceFilePath);
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
var fileExtContentTypeProvider = new FileExtensionContentTypeProvider();
if(!fileExtContentTypeProvider.TryGetContentType(fileName, out string contentType))
{
contentType = "application/octet-stream";
}
int chunkSize = 4 * 1024 * 1024;
long totalSize = file.Length;
int chunkNumber = 0;
int totalNumberOfChunks = (int)Math.Truncate((double)file.Length / (double)chunkSize);
if (Convert.ToInt32(file.Length % chunkSize) != 0)
{
totalNumberOfChunks += 1;
}
// dummy checksum to be deleted later
client.DefaultRequestHeaders.Add("checksum", "checksum");
var hasher = new MD5CryptoServiceProvider();
// initial call
while (true)
{
var buffer = new byte[chunkSize];
int read = file.Read(buffer, 0, chunkSize);
byte[] chunk;
int currentChunkSize;
if (read == 0)
{
break;
}
else if (read < chunkSize)
{
// this will be for the last chunk when we cant fill the buffer
chunk = new byte[read];
Array.Copy(buffer, chunk, read);
currentChunkSize = read;
}
else
{
chunk = buffer;
currentChunkSize = chunkSize;
}
byte[] hash = hasher.ComputeHash(chunk);
string checksum = BitConverter.ToString(hash).Replace("-", "").ToLower();
client.DefaultRequestHeaders.Remove("checksum");
client.DefaultRequestHeaders.Add("checksum", checksum);
// upload chunk
using (var bufferStream = new MemoryStream(chunk))
{
using (MultipartFormDataContent content = new MultipartFormDataContent())
{
bufferStream.Position = 0;
using (var stream = new StreamContent(bufferStream))
{
content.Add(stream, "file", fileName);
var resumableTotalSize = new StringContent(totalSize.ToString());
content.Add(resumableTotalSize, "resumableTotalSize");
var resumableChunkNumber = new StringContent(((int)chunkNumber + 1).ToString());
content.Add(resumableChunkNumber, "resumableChunkNumber");
var resumableChunkSize = new StringContent(chunkSize.ToString());
content.Add(resumableChunkSize, "resumableChunkSize");
var resumableCurrentChunkSize = new StringContent(currentChunkSize.ToString());
content.Add(resumableCurrentChunkSize, "resumableCurrentChunkSize");
var resumableTotalChunks = new StringContent(totalNumberOfChunks.ToString());
content.Add(resumableTotalChunks, "resumableTotalChunks");
var resumableType = new StringContent(contentType);
content.Add(resumableType, "resumableType");
var resumableFileName = new StringContent(fileName);
content.Add(resumableFileName, "resumableFilename");
var resumableRelativePath = new StringContent(targetFilePath);
content.Add(resumableRelativePath, "resumableRelativePath");
using (HttpResponseMessage response = client.PostAsync(requestUri, content).Result)
{
response.EnsureSuccessStatusCode();
}
}
}
}
chunkNumber += 1;
}
hasher.Dispose();
client.Dispose();
}
}
}
}
Usage would look something like the following:
public static void TestUploadFileChunked()
{
string baseAddress = @"https://my.machine1.net";
string userName = "MyEmail";
string password = "MyPassword";
string clientName = "ResourceOwnerClientId";
string clientSecret = "CLIENTSECRET";
string spaceId = "mytestspace";
string remoteFilePath = @"MyFolder/BigFile.pdf";
string localFilePath = @"..\..\..\..\files\BigFile.pdf";
JObject token = Authenticate(baseAddress, userName, password, clientName, clientSecret);
string accessToken = (string)token["access_token"];
UploadUserStoreFileChunked(accessToken, baseAddress, remoteFilePath, localFilePath, spaceId);
}
PowerShell
Download upload-file-chunked.ps1
function UploadFileChunked($sasHost, $destUrl, $destFileFullName, $srcFileFullName, $fileName, $bearerToken)
{
<#
.SYNOPSIS
Upload small to HUGE files to ShareASpace Nova using the Chunk method
.DESCRIPTION
This will fail if the file already exists on the server.
There is a path to the file System.Net.Http.dll inside this function that you may need to set for your environment
.EXAMPLE
Load a file to the Collection files in the root
UploadFileChunked "essevm540.es.eurostep.com" "/api/collection/fs/file" "BigFile.json" "C:\Users\sassetup\Desktop\BigFile.json" "BigFile.json" $global:bearerToken
.EXAMPLE
Load a file to the Collection files in a sub-directory called AutoImport. Non-existing subdirectories created automatically
UploadFileChunked "essevm540.es.eurostep.com" "/api/collection/fs/file" "AutoImport/BigFile.json" "C:\Users\sassetup\Desktop\BigFile.json" "BigFile.json" $global:bearerToken
.EXAMPLE
Load a file to the User files in a sub-directory called AutoImport
UploadFileChunked "essevm540.es.eurostep.com" "/api/space/MySpaceName/user/fs/file" "AutoImport/BigFile.json" "C:\Users\sassetup\Desktop\BigFile.json" "BigFile.json" $global:bearerToken
.EXAMPLE
Load a file to the space files in a sub-directory called AutoImport
UploadFileChunked "essevm540.es.eurostep.com" "/api/space/MySpaceName/fs/file" "AutoImport/BigFile.json" "C:\Users\sassetup\Desktop\BigFile.json" "BigFile.json" $global:bearerToken
.PARAMETER sasHost
The FQDN of your host server.
.PARAMETER destUrl
A url that contains the information on where in the unmanaged file system you want to put the file. It may include the Space name.
It looks like either
/api/space/[SpaceName]/[user or participant or 'nothing for space' or participant/OEM]/fs/file
or
/api/collection/fs/file
.PARAMETER destFileFullName
This is the destination name of your file with any relative path. If just a filename the file is put on the root. Non-existing subdirectories created automatically. It looks like
[directoryA]/[directoryB]/[filename.ext]
.PARAMETER srcFileFullName
Where in your local machine the file to upload exists. Full path and file name
.PARAMETER fileName
Just the filename. This is used to get the MIME information
.PARAMETER bearerToken
The authentication token received from a request for autentication.
#>
try{
#You must put here the correct path here to your System.Net.Http.dll . Sometimes you may also comment out this line
Add-Type -Path 'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\System.Net.Http.dll'
$md5 = new-object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider
$httpClientHandler = New-Object System.Net.Http.HttpClientHandler
$httpClient = New-Object System.Net.Http.Httpclient $httpClientHandler
$httpClient.DefaultRequestHeaders.Add("Authorization", -join("Bearer", " ", $bearerToken))
$packageFileStream = New-Object System.IO.FileStream @($srcFileFullName, [System.IO.FileMode]::Open)
$contentDispositionHeaderValue = New-Object System.Net.Http.Headers.ContentDispositionHeaderValue "form-data"
$contentDispositionHeaderValue.Name = "fileData"
$contentDispositionHeaderValue.FileName = (Split-Path $fileName -leaf)
$contentType = ""
$mimeType = [System.Web.MimeMapping]::GetMimeMapping($fileName)
if ($mimeType)
{
$contentType = $mimeType
}
else
{
$contentType = "application/octet-stream"
}
$streamContent = New-Object System.Net.Http.StreamContent $packageFileStream
$streamContent.Headers.ContentDisposition = $contentDispositionHeaderValue
$streamContent.Headers.ContentType = New-Object System.Net.Http.Headers.MediaTypeHeaderValue $contentType
$chunkSize = 4 * 1024 * 1024
$totalSize = $packageFileStream.Length
$chunkNumber = 0
$totalNumberOfChunks = [System.Convert]::ToInt32([System.Math]::Truncate($packageFileStream.Length / $chunkSize))
if ([System.Convert]::ToInt32($packageFileStream.Length % $chunkSize) -ne 0)
{
$totalNumberOfChunks += 1;
}
$httpClient.DefaultRequestHeaders.Add("checksum","123")
# initial call
while($true)
{
$buffer = New-Object byte[] $chunkSize
[System.IO.MemoryStream]$ss = New-Object -TypeName System.IO.MemoryStream -ArgumentList (,$buffer)
$r = $packageFileStream.Read($buffer, 0, $chunkSize)
if($r -eq 0)
{
break
}
$ssByteArr = $buffer
$currentChunkSize = $chunkSize
# this will be for the last chunk when we cant fill the buffer
if($r -lt $chunkSize)
{
$buffer2 = New-Object byte[] $r
[Array]::Copy($buffer, $buffer2, $r)
$ss = New-Object -TypeName System.IO.MemoryStream -ArgumentList (,$buffer2)
$ssByteArr =$buffer2
$currentChunkSize = $r
}
$hash = [System.BitConverter]::ToString($md5.ComputeHash($ssByteArr)).Replace("-","").ToLower();
$x = $httpClient.DefaultRequestHeaders.Remove("checksum")
$httpClient.DefaultRequestHeaders.Add("checksum",$hash)
$ss.Position = 0
# upload chunk
$content = New-Object System.Net.Http.MultipartFormDataContent
$sc = New-Object System.Net.Http.StreamContent($ss)
$content.Add($sc, "file", $fileName)
$resumableTotalSize = New-Object System.Net.Http.StringContent($totalSize)
$content.Add($resumableTotalSize, "resumableTotalSize")
$resumableChunkNumber = New-Object System.Net.Http.StringContent([int]$chunkNumber + 1)
$content.Add($resumableChunkNumber, "resumableChunkNumber")
$resumableChunkSize = New-Object System.Net.Http.StringContent($chunkSize)
$content.Add($resumableChunkSize, "resumableChunkSize")
$resumableCurrentChunkSize = New-Object System.Net.Http.StringContent($currentChunkSize)
$content.Add($resumableCurrentChunkSize, "resumableCurrentChunkSize")
$resumableTotalChunks = New-Object System.Net.Http.StringContent($totalNumberOfChunks)
$content.Add($resumableTotalChunks, "resumableTotalChunks")
$resumableType = New-Object System.Net.Http.StringContent($mimeType)
$content.Add($resumableType, "resumableType")
$resumableFileName = New-Object System.Net.Http.StringContent($fileName)
$content.Add($resumableFileName, "resumableFilename")
$resumableRelativePath = New-Object System.Net.Http.StringContent($destFileFullName)
$content.Add($resumableRelativePath, "resumableRelativePath")
try
{
$requestUri = "https://{0}{1}" -f $sasHost, $destUrl
$response = $httpClient.PostAsync($requestUri, $content).Result
$statusCode = $response.StatusCode
if (!$response.IsSuccessStatusCode)
{
$responseBody = $response.Content.ReadAsStringAsync().Result
$errorMessage = "Status code {0}. Reason {1}. Server reported the following message: {2}." -f $response.StatusCode, $response.ReasonPhrase, $responseBody
throw [System.Net.Http.HttpRequestException] $errorMessage
}
$result = $response.Content.ReadAsStringAsync().Result
}
catch [Exception]
{
$PSCmdlet.ThrowTerminatingError($_)
}
finally
{
if($null -ne $response)
{
$response.Dispose()
}
if($null -ne $content)
{
$content.Dispose()
}
if($null -ne $sc)
{
$sc.Dispose()
}
}
$chunkNumber += 1
}
if($null -ne $packageFileStream)
{
$packageFileStream.Dispose()
}
}
catch {
$statusCode = $_.Exception.Response.StatusCode.value__
$statusDescription = $_.Exception.Response.StatusDescription
Write-Log -Level Error -Message "UploadFileChunked - $($statusDescription)($($statusCode)); Response:$($_.Exception)"
}
}
UploadFileChunked "my.machine1.net" "/api/space/mytestspace/user/fs/file" "UploadFolder/TestFile.txt" "C:\temp\TestFile.txt" "TestFile.txt" $global:bearerToken
Upload Bulk
There are two admin routes that enable direct upload of a folder structure with files through a HTTP Post to ShareAspace at the three levels:
- collection level
- space level
- participant level
To accomplish this two admin routes are used:
admin/fileupload/admin/fileupload/{spaceId}/{participantId?}
These admin routes needs the AdministrationApi key to work.
Request body params:
sourceDirectory(mandatory) - This is where your files are on disk, the ShareAspace host needs access to that folderwriteLog(mandatory) - Set this to true to get a detailed JSON log generated at sourceDirectory when upload is finishedtargetDirectory(defaulted to "/") - This is the relative path in ShareAspace where the file structure should end upreplaceFiles(optional bool, defaulted to false) - If set, the files that already exist in ShareAspace will be replaceddeleteFilesAfterUpload(optional bool, defaulted to false) - If set, the files on disk will be deleted from the sourceDirectory, use with caution
If writeLog setting is set to true a JSON log is automatically created at the sourceDirectory root when upload is finished. FileUpload_timestamp.log. This will contain the settings used for upload, details about each file, the eventual errors and some file upload statistics.
CSharp
using Newtonsoft.Json.Linq;
using System;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography;
using System.Text;
namespace Demo.FileApi
{
public static partial class Program
{
public static void BulkUploadFiles(string baseAddress, string sourceDirectory, string apiKey, bool writeLog, string spaceId = null, string participantId = null, string targetDirectory = "/", bool replaceFiles = false, bool deleteFilesAfterUpload = false)
{
string path;
if (spaceId == null)
{
path = $"/admin/fileupload";
}
else if (spaceId != null && participantId != null)
{
path = $"/admin/fileupload/{spaceId}/{participantId}";
}
else
{
path = $"/admin/fileupload/{spaceId}";
}
JObject data = new JObject();
data["sourceDirectory"] = sourceDirectory;
data["targetDirectory"] = targetDirectory;
data["replaceFiles"] = replaceFiles;
data["deleteFilesAfterUpload"] = deleteFilesAfterUpload;
data["writeLog"] = writeLog;
//Hash the data
byte[] apiKeys = Convert.FromBase64String(apiKey);
HMACSHA512 hmacsha = new HMACSHA512(apiKeys);
byte[] hashed = hmacsha.ComputeHash(Encoding.UTF8.GetBytes(path));
// Convert to proper format
string output = Convert.ToBase64String(hashed);
output = output.Split('=')[0]; // Remove any trailing '='s
output = output.Replace('+', '-'); // 62nd char of encoding
output = output.Replace('/', '_'); // 63rd char of encoding
HttpClient client = new HttpClient();
client.BaseAddress = new Uri(baseAddress);
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", output);
StringContent postParams = new StringContent(data.ToString(), Encoding.UTF8, "application/json");
HttpResponseMessage response = client.PostAsync(path, postParams).Result;
response.EnsureSuccessStatusCode();
client.Dispose();
}
}
}
Usage would look something like the following:
public static void TestBulkUploadFiles()
{
string baseAddress = @"http://localhost:5000";
string sourceDirectory = @"C:\UploadFolder";
string adminApiSymmetricKey = "8JcfLlO+usfLslIIUNIYofT8FI17l93je/QK6jE/Yhm+cKd/N2iaV26qElc9qpuLNP/ySy/2lAxeIVKd9ONt+w==";
bool writeLog = true;
string spaceId = "mytestspace";
string participantId = "OEM";
string targetDirectory = "/";
bool replaceFiles = true;
bool deleteFilesAfterUpload = false;
BulkUploadFiles(baseAddress, sourceDirectory, adminApiSymmetricKey, writeLog, spaceId, participantId, targetDirectory, replaceFiles, deleteFilesAfterUpload);
}
PowerShell
Download bulk-upload-files.ps1
<#
.SYNOPSIS
This PowerShell scripts uploads files from a local folder to unmanaged files for a collection, space or a paricipant
.DESCRIPTION
This script takes 3 mandatory parameters. Before executing this script please make sure service host is running.
.PARAMETER baseAddress
First parameter is baseAddress. It is a mandatory parameter. BaseAddress must be as http://localhost:5000 or http://my.machine1.net:5000.
.PARAMETER sourceDirectory
Second parameter is sourceDirectory. It is a mandatory parameter. SourceDirectory is where the files are resided that should be uploaded. Must be an absolute path.
.PARAMETER apiKey
Third parameter is apiKey. It is a mandatory parameter. Apikey can be found from bootstrap_collection.json
.PARAMETER writeLog
Fourth parameter is writeLog. It is a mandatory boolean parameter. If set to true a json log will be generated at the sourceDirectory when upload finish.
.PARAMETER spaceId
Fifth parameter is spaceId. It is an optional parameter. SpaceId is the ID of space. Files will be uploaded to collection if not specified.
.PARAMETER participantId
Sixth parameter is participantId. It is an optional parameter. ParticipantId is the id of participant. This must be used in combination with spaceId. Files will be uploaded to the participant.
.PARAMETER targetDirectory
Seventh parameter is targetDirectory. It is an optional parameter. targetDirectory is the relative folder path where the files should be uploaded. eg -targetDirectory "/targetFolder/" defaulted to "/"
.PARAMETER replaceFiles
Eighth parameter is replaceFiles. It is an optional parameter defaulted to false. If set to true it will replace existing files in targetDirectory.
.PARAMETER deleteFilesAfterUpload
Ninth parameter is deleteFilesAfterUpload. It is an optional parameter defaulted to false. DeleteFilesAfterUpload if set to true will remove the uploaded files from the sourceDirectory.
.EXAMPLE
C:\PS> .\bulk-upload-files.ps1
If no parameter provided it will ask to enter mandatory parameters. It will display both success and failure results
.EXAMPLE
C:\PS> .\bulk-upload-files.ps1 -baseAddress http://localhost:5000 -sourceDirectory "C:\UploadFolder" -apiKey apikey -writeLog $true
All mandatory parameters provided. It will display both success and failure results
#>
param (
[Parameter(Position=1, Mandatory=$true)]
[String]$baseAddress,
[Parameter(Position=2, Mandatory=$true)]
[String]$sourceDirectory,
[Parameter(Position=3, Mandatory=$true)]
[String]$apiKey,
[Parameter(Position=4, Mandatory=$true)]
[Bool]$writeLog,
[Parameter(Position=5, Mandatory=$false)]
[String]$spaceId = $null,
[Parameter(Position=6, Mandatory=$false)]
[String]$participantId = $null,
[Parameter(Position=7, Mandatory=$false)]
[String]$targetDirectory = "/",
[Parameter(Position=8, Mandatory=$false)]
[Bool]$replaceFiles = $false,
[Parameter(Position=9, Mandatory=$false)]
[Bool]$deleteFilesAfterUpload = $false
)
# Add the necessary .NET assembly
Add-Type -AssemblyName System.Net.Http
$apiKeys = [System.Convert]::FromBase64String($apiKey);
if (!$spaceId)
{
$path = "/admin/fileupload";
}
elseif ($spaceId -and $participantId)
{
$path = "/admin/fileupload/$spaceId/$participantId";
}
else
{
$path = "/admin/fileupload/$spaceId";
}
# Setup json data
$data = @{
sourceDirectory = "$sourceDirectory";
targetDirectory = "$targetDirectory";
replaceFiles = "$replaceFiles";
deleteFilesAfterUpload = $deleteFilesAfterUpload;
writeLog = $writeLog} | ConvertTo-Json
# Create the HttpClient client
$client = New-Object -TypeName System.Net.Http.Httpclient
# Set base address
$client.BaseAddress = $baseAddress;
# Hash the data
$hmacsha = New-Object System.Security.Cryptography.HMACSHA512;
$hmacsha.key = $apiKeys;
$hashed = $hmacsha.ComputeHash([system.Text.Encoding]::UTF8.GetBytes($path));
# Convert to proper format
$output = [System.Convert]::ToBase64String($hashed);
$output = $output.Split('=')[0]; # Remove any trailing '='s
$output = $output.Replace('+', '-'); # 62nd char of encoding
$output = $output.Replace('/', '_'); # 63rd char of encoding
# Setup HttpClient client for the secure call
$client.DefaultRequestHeaders.Authorization = New-Object -TypeName System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", $output);
$encoding = New-Object -TypeName system.Text.UTF8Encoding;
$postParams = New-Object -TypeName System.Net.Http.StringContent($data, $encoding, "application/json");
# PostAsync
$response = $client.PostAsync($path, $postParams).Result;
# check If it is failed
if($response.IsSuccessStatusCode -eq $false)
{
write-host $response.StatusCode -foregroundcolor "Red";
Write-Host "Failed to upload files" -ForegroundColor "Red";
return;
}
else
{
Write-Host $response.RequestMessage.RequestUri -ForegroundColor Magenta -BackgroundColor White
Write-Host "File upload request successful, check log file after completion for details" -ForegroundColor Magenta -BackgroundColor White
}
An example of usage:
.\bulk-upload-files.ps1 -baseAddress http://localhost:5000 -sourceDirectory "C:\UploadFolder" -apiKey 8JcfLlO+usfLslIIUNIYofT8FI17l93je/QK6jE/Yhm+cKd/N2iaV26qElc9qpuLNP/ySy/2lAxeIVKd9ONt+w== -writeLog $true
Download files
Downloading files is a very simple process that requires that your URL from the section Unmanaged file areas has appended to it the source subdirectories and file-name. If the file is in the root then don't include any subdirectory. For example in the collection area with subdirectory MySubDir and file MyFile.txt
| Area | URL |
|---|---|
| collection | /api/collection/fs/file/MySubDir/MyFile.txt |
An example of the HTTPS call is the following:
GET https://essevm540.es.eurostep.com/api/space/InReachAdmin/participant/OEM/fs/file/AutoImport/testImport.xlsx?token=eyJ0eXAiOiJKV1Qi...
Note that in the above call the token is in the URL. There are no headers. This is a very simple technique.
CSharp
For this to work you need to include the function in the code from the section Authentication.
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
namespace Demo.FileApi
{
public static partial class Program
{
public static void DownloadUserStoreFile(string accessToken, string baseAddress, string sourceFilePath, string targetFilePath, string spaceId)
{
string requestUri = $"{baseAddress}/api/space/{spaceId}/user/fs/file/{sourceFilePath}";
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
Stream response = client.GetStreamAsync(requestUri).Result;
using (FileStream file = new FileStream(targetFilePath, FileMode.Create))
{
response.CopyTo(file);
}
client.Dispose();
}
}
}
Usage would look something like the following:
public static void TestDownloadFile()
{
string baseAddress = @"https://my.machine1.net";
string userName = "MyEmail";
string password = "MyPassword";
string clientName = "ResourceOwnerClientId";
string clientSecret = "CLIENTSECRET";
string spaceId = "mytestspace";
string remoteFilePath = @"MyFolder/TestFile.txt";
string localFilePath_user = @"..\..\..\..\files\TestFile_user.txt";
JObject token = Authenticate(baseAddress, userName, password, clientName, clientSecret);
string accessToken = (string)token["access_token"];
DownloadUserStoreFile(accessToken, baseAddress, remoteFilePath, localFilePath_user, spaceId);
}
PowerShell
The code example below uses a System.Net.WebClient object to execute this HTTPS call. The reason this object is used is because of its speed and simplicity. However, there is no indication of downloading progress. If you want a function that shows the downloading progress then use the function "Invoke-WebRequest -Uri $url -OutFile $output" instead with the same URL as in the code below. The Invoke-WebRequest is many times slower than the technique shown below.
function DownloadFile($sasHost, $srcFileUrlPath, $localFullName, $bearerToken)
{
<#
.SYNOPSIS
Download files from ShareASpace Nova
.DESCRIPTION
fails if the file does not exist on the server
.EXAMPLE
Download a file from the Collection files in the root
DownloadFile "essevm540.es.eurostep.com" "/api/collection/fs/file/myfile.txt" "C:\Users\sassetup\Desktop\myfile.txt" $global:bearerToken
.EXAMPLE
Download a file from the Collection files in a sub-directory called AutoImport.
DownloadFile "essevm540.es.eurostep.com" "/api/collection/fs/file/AutoImport/myfile.txt" "C:\Users\sassetup\Desktop\myfile.txt" $global:bearerToken
.EXAMPLE
Download a file from the User files in a sub-directory called AutoImport
DownloadFile "essevm540.es.eurostep.com" "/api/space/MySpaceName/user/fs/file/AutoImport/myfile.txt" "C:\Users\sassetup\Desktop\myfile.txt" $global:bearerToken
.EXAMPLE
Download a file from the space files in a sub-directory called AutoImport
DownloadFile "essevm540.es.eurostep.com" "/api/space/MySpaceName/fs/file/AutoImport/myfile.txt" "C:\Users\sassetup\Desktop\myfile.txt" $global:bearerToken
.PARAMETER sasHost
The FQDN of your host server.
.PARAMETER srcFileUrlPath
A url that contains the information on where in the unmanaged file system you want to remove the file. It may include the Space name.
It looks like either
/api/space/[SpaceName]/[user or participant or 'nothing for space' or participant/OEM]/fs/file/[subdirectory(one or more)]/[filename]
or
/api/collection/fs/file/[subdirectory(one or more)]/[filename]
.PARAMETER localFullName
The full filename and path to where you want the file put on your local drive
.PARAMETER bearerToken
The authentication token received from a request for autentication.
#>
try{
$headers = @{"Authorization"="Bearer $bearerToken";}
$requestUri = "https://{0}{1}" -f $sasHost, $srcFileUrlPath
$response = Invoke-RestMethod -Method Get -Uri $requestUri -Headers $headers -OutFile $localFullName
}
catch{
$statusCode = $_.Exception.Response.StatusCode.value__
$statusDescription = $_.Exception.Response.StatusDescription
Write-Host "$($statusDescription)($($statusCode)); Uri: $($url); Response: $($_.Exception)"
}
}
DownloadFile "my.machine1.net" "/api/space/mytestspace/user/fs/file/UploadFolder/TestFile.txt" "C:\temp\TestFile.txt" $global:bearerToken
Delete files
Deleting files is a very simple process that requires that your URL from the section Unmanaged file areas has appended to it the subdirectories and file-name of the file you want to delete. If the file is in the root then don't include any subdirectory. For example in the collection area with subdirectory MySubDir and file MyFile.txt
| Area | URL |
|---|---|
| collection | /api/collection/fs/file/MySubDir/MyFile.txt |
An example of the HTTPS call is the following:
DELETE https://essevm540.es.eurostep.com/api/space/InReachAdmin/participant/OEM/fs/file/AutoImport/testImport.xlsx
Authorization: Bearer eyJ0eXAiOiJKV1Q...
CSharp
For this to work you need to include the function in the code from the section Authentication.
using System.Net.Http;
using System.Net.Http.Headers;
namespace Demo.FileApi
{
public static partial class Program
{
public static void DeleteUserStoreFile(string accessToken, string baseAddress, string targetFilePath, string spaceId)
{
string requstUri = $"{baseAddress}/api/space/{spaceId}/user/fs/file/{targetFilePath}";
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
HttpResponseMessage response = client.DeleteAsync(requstUri).Result;
response.EnsureSuccessStatusCode();
client.Dispose();
}
}
}
Usage would look something like the following:
public static void TestDeleteFile()
{
string baseAddress = @"https://my.machine1.net";
string userName = "MyEmail";
string password = "MyPassword";
string clientName = "ResourceOwnerClientId";
string clientSecret = "CLIENTSECRET";
string spaceId = "mytestspace";
string remoteFilePath = @"MyFolder/TestFile.txt";
JObject token = Authenticate(baseAddress, userName, password, clientName, clientSecret);
string accessToken = (string)token["access_token"];
DeleteUserStoreFile(accessToken, baseAddress, remoteFilePath, spaceId);
}
PowerShell
Since this technique uses the Invoke-RestMethod then remember that the format of the authentication token is $global:bearer from the section Authentication.
function RemoveFile($sasHost, $destFileUrlPath, $bearerToken)
{
<#
.SYNOPSIS
Remove files from ShareASpace Nova
.DESCRIPTION
fails if the file does not exist on the server
.EXAMPLE
Remove a file from the Collection files in the root
RemoveFile "essevm540.es.eurostep.com" "/api/collection/fs/file/eraseme.txt" $global:bearer
.EXAMPLE
Remove a file from the Collection files in a sub-directory called AutoImport.
RemoveFile "essevm540.es.eurostep.com" "/api/collection/fs/file/AutoImport/eraseme.txt" $global:bearer
.EXAMPLE
Remove a file from the User files in a sub-directory called AutoImport
RemoveFile "essevm540.es.eurostep.com" "/api/space/MySpaceName/user/fs/file/AutoImport/eraseme.txt" $global:bearer
.EXAMPLE
Remove a file from the space files in a sub-directory called AutoImport
RemoveFile "essevm540.es.eurostep.com" "/api/space/MySpaceName/fs/file/AutoImport/eraseme.txt" $global:bearer
.PARAMETER sasHost
The FQDN of your host server.
.PARAMETER destFileUrlPath
A url that contains the information on where in the unmanaged file system you want to remove the file. It may include the Space name.
It looks like either
/api/space/[SpaceName]/[user or participant or 'nothing for space' or participant/OEM]/fs/file/[subdirectory(one or more)]/[filename]
or
/api/collection/fs/file/[subdirectory(one or more)]/[filename]
.PARAMETER bearer
The authentication bearer received from a request for autentication.This must have the format @{ "Authorization" = ("Bearer", $global:bearerToken -join " ") }
#>
try{
$headers = @{"Authorization"="Bearer $bearerToken";}
$requestUri = "https://{0}{1}" -f $sasHost, $destFileUrlPath
$response = Invoke-RestMethod -Method Delete -Uri $requestUri -Headers $headers
}
catch{
$statusCode = $_.Exception.Response.StatusCode.value__
$statusDescription = $_.Exception.Response.StatusDescription
Write-Host "$($statusDescription)($($statusCode)); Uri: $($requestUri); Response: $($_.Exception)"
}
}
RemoveFile "my.machine1.net" "/api/space/mytestspace/user/fs/file/UploadFolder/TestFile.txt" $global:bearerToken
Download sample
The examples above is collected in a project that can be downloaded.