Documentation
6.11. Azure BLOB Service¶
An azure-blob location provides access over HTTPS to the Azure BLOB Service of an Azure Storage account.
6.11.1. Introduction¶
The HTTPS connections will use the default list of secure ciphers and will accept TLS v1.1 and TLS v1.2 protocols.
Note
Unsecured HTTP access is not available.
Note
The current implementation is tested using General-purpose v2 (GPv2) accounts.
Only storage account names with access keys are currently supported. Please get in touch if you plan to use Azure Active Directory or shared access signatures.
The path / url to the Azure File / BLOB storage is case-sensitive.
When using Azure File / BLOB locations, the source or destination path will be
defined as the name of the share or container (to or from where files are transferred),
followed by the targeted directory inside that share.
The path will look like /SHARE-NAME/DIR-IN-ROOT/PARENT-DIR
or /CONTAINER-NAME/DIR-IN-ROOT/PARENT-DIR
.
The requests made by SFTPPlus to the Azure Storage server are done using
the sftpplus-azure-blob-UUID
or sftpplus-azure-file-UUID
format.
Where UUID is a unique identifier for this location.
This can be used inside Azure Storage Analytics to identify the operations
done by this location.
The request ID will look like:
x-ms-client-request-id: sftpplus-azure-blob-60ec1329-cc5d-416e-81b9-7c22
of
x-ms-client-request-id: sftpplus-azure-file-60ec1329-cc5d-416e-81b9-7c22
6.11.2. username¶
- Default value:
Empty
- Optional:
No
- From version:
3.36.0
- Values:
Text.
- Description:
Name of the Azure Storage Account.
6.11.3. password¶
- Default value:
Empty
- Optional:
Yes
- From version:
3.36.0
- Values:
Plain text.
- Description:
Any of the two access keys for the Azure storage account.
It should be specified in Base64 format. This is the default format presented by the Azure Portal.
6.11.4. name¶
- Default value:
Empty
- Optional:
No
- From version:
2.8.0
- Values:
Any text.
- Description:
Human-readable short text used to identify this location.
6.11.5. description¶
- Default value:
Empty
- Optional:
Yes
- From version:
2.8.0
- Values:
Any text.
- Description:
Human-readable text that describes the purpose of this location.
6.11.6. type¶
- Default value:
''
- Optional:
No
- From version:
2.6.0
- Values:
filesystem - Local file system.
sftp - SFTP protocol v3 over SSH v2.
ftp - FTP protocol without any encryption.
ftpse - Explicit FTPS protocol.
ftpsi - Implicit FTPS protocol.
webdavs - WebDAV over HTTPS.
as2 - AS2 over HTTP or HTTPS
azure-file - Azure File Service.
smb - SMB / Windows Share
- Description:
This option specifies the type of the location. Each type has a set of specific configuration options
6.11.7. idle_connection_timeout¶
- Default value:
300
- Optional:
Yes
- From version:
3.0.0
- Values:
Number of seconds
0
- Description:
This controls the automatic disconnection from the remote server after the location has not received any file transfer operation requests for the configured number of seconds.
Keep-alive command requests are not counted as file transfer operations. The connection gets automatically disconnected if keep-alive is the only command requested in the configured interval.
Disconnected locations automatically reconnect when a new file transfer operation request is made. For example, when a new file needs to be transferred to the remote server.
If the remote peer closes the connection before the configured timeout, the connection is left closed. It gets automatically reconnected when a new file transfer operation is requested.
Set to 0 to always keep the connection active, by forcing re-connection when the remote server closes the connection.
Note
The idle_connection_timeout is the maximum number of seconds before closing an idle connection to the server. If the remote server decides that the connection is idle and closes the connection, SFTPPlus doesn't try to "challenge" the server, leaving the connection closed. The connection is automatically reopened next time a file needs to be transferred.
6.11.8. idle_connection_keepalive_interval¶
- Default value:
0
- Optional:
Yes
- From version:
3.0.0
- Values:
Number of seconds
- Description:
Send a keep-alive command every N seconds to avoid having the connection disconnected by the other peer due to inactivity.
Set to 0 to disable keep-alive commands.
The keep-alive command does not reset the idle connection timeout,
6.11.9. connection_retry_count¶
- Default value:
12
- Optional:
Yes
- From version:
3.9.0
- Values:
Number of retries
- Description:
Number of times to retry connection to the location, when the initial connection fails.
Set to 0 to not retry.
6.11.10. connection_retry_interval¶
- Default value:
300
- Optional:
Yes
- From version:
3.9.0
- Values:
Number of seconds
- Description:
Number of seconds to wait between connection attempts.
Set to 0 to retry right away without any delay.
6.11.11. Limitation¶
The Azure BLOB service is designed to work as an object storage system, and not as a file-system.
The main difference between Azure BLOB and other file storage systems, is the lack of support for folders / directories.
The directory hierarchy is created by using a naming convention, in which the file name contains a directory delimiter character. Files are still stored in a flat structure.
A major implication, is that there are no empty folders. Folders are generated based on section of a file name, thus you can't just have a folder without a file.
At the same time, getting the files from a folder is implemented by searching for all the files for which their names start with that folder name. This means that when you list a folder that doesn't exist the result is an empty list.
6.11.12. Keeping the source directory¶
When a transfer is configured with an Azure container as source and configured to delete the source file on a success transfer, the source folder might end up being empty.
In Azure BLOB, empty folders don't exist, and SFTPPlus will generate an error when the source folder of a transfer doesn't exists.
To keep the source folder in Azure BLOB, it is recommend to always keep a file in the source directory.
You can configure the SFTPPlus transfer to ignore that file.
For example, you can create a file name .keep
and configure a transfer to ignored it as:
source_path = /manual-container/source/folder
source_filter: ! .keep