13.12.1. Introduction¶

There are 3 main phases when dealing with application logs:

• Log generation

• Log centralization

• Log analysis

The focus for SFTPPlus is to provide a robust file transfer solution which can be easily integrated with any external dedicated log management tool.

In the case in which you did not find it straightforward to integrate SFTPPlus with your preferred log management tool, please get in touch and we will work towards providing a better experience.

13.12.2. Log generation¶

In SFTPPlus, the logs are generated using the event handlers. An event handler will receive the events generated by SFTPPlus and transform it into a format usable for the logging system.

This simple transformation is to generate an entry into a plain text log file.

SFTPPlus can store the logs as comma-separated values (CSV), making it easy to parse the input and transform it according to your needs.

SFTPPlus can also generate the events directly as syslog entries or as HTTP requests. In this case, these logs are stored on the local SFTPPlus instance. While this can help reduce the disk usage on the SFTPPlus instance, this approach is susceptible to data loss, in the case in which the network connection is broken between STPPlus and the remote server. This approach is not recommended for high volume installations.

13.12.3. Log rotation¶

With a default configuration SFTPPlus will rotate the log file at the end of each day. That is, it will rename the previous log file and start with an empty file.

In SFTPPlus there are 2 main types of rotations:

• size-based rotation

• time-based rotation

The size-based rotation will rotate the file as soon as the log file reaches a certain size. The files on disk will look like:

path/to/server.log
path/to/server.log.1
path/to/server.log.2
path/to/server.log.3
path/to/server.log.4
path/to/server.log.5

where path/to/server.log is the main log file and the others are rotated files containing older logs.

The time-based rotation will rotate the file at a specific time or regular time intervals, regardless of the size of the log file. The time-based rotated files will look on disk like the example below. The actual time stamp used for the rotated files differs based on the rotation rule:

path/to/server.log
path/to/server.2018-07-30_15-17-10.log
path/to/server.2018-07-30_15-17-43.log
path/to/server.2018-07-30_15-18-16.log
path/to/server.2018-07-30_15-19-01.log
path/to/server.2018-07-30_15-21-00.log

where path/to/server.log is the main log file and the others are rotated files containing older logs.

When rotating logs, you have the option to automatically delete older rotated files and keep rotated log files up to a certain number.

13.12.4. Log centralization¶

The embedded SFTPPlus database is not suitable for high volume installations. We recommend using specialized tools for log centralization.

The standard way to achieve log centralization is by using the syslog protocol. By configuring a syslog event handler, SFTPPlus can send the logs to a remote syslog server.

The syslog protocol may not be enough and for new installations, we recommend using modern log management tools like Fluentd, Splunk, Datadog, Sumologic, or Loggly (just to name a few in no particular order).

For efficient and fault tolerant operations, we recommend storing the SFTPPlus logs as a rotated local file and use the agent provided by your logging management tool to forward the logs to a centralized server.

In this way, even if the network between the SFTPPlus instance and the centralized server is down for a brief period of time, the logs are buffered inside the local file and can be forwarded once the connection is established. Having pending logs buffered in a local file provides improved performance while at the same time mitigate data losses associated to power failures, as only queues stored in memory will be lost.

SFTPPlus provides advanced functionality for rotating log files and can automatically delete older rotated files.

The default installation will rotate log files at the end of each time. For better control over disk space used by log files, you should choose to rotate files based on size.

13.12.5. Log analysis¶

For log analysis, SFTPPlus only provides a basic tool to search and filter the logs, which is not suitable for high volume installations.

The log analysis component is implemented in SFTPPlus as the "Activity Log" page from the Web Manager web console.

We recommend using the analysis and filtering functionalities provided by the dedicated logging tools.

13.12.6. Linux Service Logs¶

When running on Linux without systemd or without a custom supervisor, SFTPPlus will operate using an embedded process supervisor.

Two additional logs are created:

• log/supervisord.log - Content of the processed launched to create the SFTPPlus service / daemon.

• log/server-startup-errors.log - This is usually empty and contains unexpected error during startup.

13.12.7. Windows Service Logs¶

When running on Windows as a service, SFTPPlus will generate 3 additional log files. There are used to log the initial service startup up to the point where the SFTPPlus own logs are initialized.

Under normal operation these don't contain any important information. The purpose of these logs is to catch errors during the Windows service startup process.

Below is a description of each log file:

• sftpplus-service.wrapper.log - Logs the commands used to start the SFTPPlus Windows service. This log can be removed at any time.

• sftpplus-service.TIMESTAMP.out.log - Includes the standard output for the main SFTPPlus process during startup. This log file can only be removed when the SFTPPlus application is not running.

• sftpplus-service.TIMESTAMP.err.log - Includes the standard error content. This log file can only be removed when the SFTPPlus application is not running.