Enable log rotation for Edge components

Log rotation—the process of rotating multiple log files in and out of use— simplifies administration of systems that generate large numbers of log files. Log rotation enables automatic rotation, compression, removal, and mailing of log files.

In Edge for Private Cloud, some of the main log files on each apigee component are configured with a default rotation mechanism. For example, on the Message Processor component, the following files are configured with a default rotation mechanism using logback:

  • /opt/apigee/var/log/edge-message-processor/logs/system.log
  • /opt/apigee/var/log/edge-message-processor/logs/events.log
  • /opt/apigee/var/log/edge-message-processor/logs/startupruntimeerrors.log
  • /opt/apigee/var/log/edge-message-processor/logs/configurations.log
  • /opt/apigee/var/log/edge-message-processor/logs/transactions.log

Similar files exist for other edge-* components (whose names begin with edge-), such as edge-management-server, edge-router, edge-postgres-server, and edge-qpid-server.

Each of these edge-* components also generate an additional log file that is a redirected output of the respective component's console. In the case of the Message Processor component, this file is called /opt/apigee/var/log/edge-message-processor/edge-message-processor.log. Other edge-* components generate a similar file. These files' rotation are not done by the logback library, but rather using logrotate and crontab.

Before you begin

  • If you aren't familiar with logrotate configurations, read the logrotate manual.
  • If you aren't familiar with crontab configurations, read the crontab manual.

Enable log rotations

Starting with Edge for Private Cloud version 4.52, when you install an edge-* component, a crontab entry for rotating this log is automatically created in the apigee user's crontab. This cron job runs every hour by default, and uses the logrotate utility to rotate logs. If multiple edge-* components are installed on the same node (for example, in an all-in-one installation, or if the router and message-processor are installed on one node), every component will have its own cron entry for rotating its respective logs.

Notes:

  • If during installation of an edge-* component, you don't want to automatically set up a logrotate cronjob, set the flag “ENABLE_LOGROTATE=n” in your silent configuration file you use for installation. If the cron job is not set during installation, you can set it up later by following the steps to enable a component's logrotate cron.
  • For the component edge-router, an additional logrotate cron job is also configured by default. This cron rotates access and error logs generated by Nginx.

The following examples illustrate various configurations and administrative operations that you can perform.

View cron entries on a node

sudo crontab -u apigee -l

View a component's logrotate cron entry

$APIGEE_ROOT/apigee-service/bin/apigee-service <component-name> logrotate -s

For example, to view edge-message-processor's logrotate cron, run:

$APIGEE_ROOT/apigee-service/bin/apigee-service edge-message-processor logrotate -s

View a components' logrotate configuration

A component's logrotate configuration can be found in the file $APIGEE_ROOT/<component>/logrotate/logrotate.conf. For example, in the case of edge-message-processor, the node's logrotate configuration can be found in $APIGEE_ROOT/edge-message-processor/logrotate/logrotate.conf.

Enable a component's logrotate cron

$APIGEE_ROOT/apigee-service/bin/apigee-service <component-name> logrotate -e

For example, to enable edge-message-processor's logrotate cron, run:

$APIGEE_ROOT/apigee-service/bin/apigee-service edge-message-processor logrotate -e

Disable a component's logrotate cron

$APIGEE_ROOT/apigee-service/bin/apigee-service <component-name> logrotate -d

For example, to disable edge-message-processor's logrotate cron, run:

$APIGEE_ROOT/apigee-service/bin/apigee-service edge-message-processor logrotate -d

Change cron schedule of a particular logrotate command

On the node where you would like to change the logrotate cron schedule, open the respective component's configuration override file. The edge-* component's configuration override files are listed in the table below:

Component Configuration override file
edge-message-processor $APIGEE_ROOT/customer/application/message-processor.properties
edge-management-server $APIGEE_ROOT/customer/application/management-server.properties
edge-router $APIGEE_ROOT/customer/application/router.properties
edge-postgres-server $APIGEE_ROOT/customer/application/postgres-server.properties
edge-qpid-server $APIGEE_ROOT/customer/application/qpid-serverproperties

Open the appropriate override file and set the following configuration override. The example below overrides the cron job to run at 0th minute of every hour.

conf_logrotate_cron_duration=0 * * * *

To set the configuration override:

  1. Ensure the file is readable by the apigee user.
  2. Apply the newer cron schedule to the component: 
    $APIGEE_ROOT/apigee-service/bin/apigee-service <component-name> logrotate -e

For example, to apply newer cron schedule to edge-message-processor's logrotate, enter:

$APIGEE_ROOT/apigee-service/bin/apigee-service edge-message-processor logrotate -e

To verify that the changes have been applied, either check cron entries or view component's logrotate cron entry.

Additional logrotate configurations

Similar to steps listed in the section above for modifying the cron schedule, you can also modify additional configurations. The table below describes these configurations:

Configuration Description
conf_logrotate_bin_location Location of logrotate binary. Default: /usr/sbin/logrotate
conf_logrotate_status_file Status file capturing runtime status of logrotate - show have read/write permissions for apigee user. Default: $APIGEE_ROOT/var/run/<component>/logrotate.status
conf_logrotate_logs Which logs to rotate. Can include wildcards. Default: $APIGEE_ROOT/var/log/<component>/<component>.log
conf_logrotate_rotate Rotation count. Default: “5”
conf_logrotate_compress Whether to compress rotated files. Default: “compress”
conf_logrotate_compress_type Type of compression. Default: “delaycompress”. Postpones compression of latest rotated file to next logrotate cycle.
conf_logrotate_size Log size limit for rotation. Default: “10M”

Use your own log rotation mechanism and cron

If your requirements for rotating the <component>.log are not met by the configurations provided by Apigee, you utilize your own mechanism of log rotation using tools like logrotate or others. For example, the configurations provided by Apigee let you rotate logs by size but you want to rotate logs based on date/time or other factors. In such cases, you should disable the logrotate cron provided by Apigee and set up your own cron jobs as needed.

If you need to refer to Apigee's log rotation cron entry, you can find it in the file $APIGEE_ROOT//logrotate/logrotate.cron. For example, in the case of edge-message-processor, this file is called $APIGEE_ROOT/edge-message-processor/logrotate/logrotate.cron. This file includes both the cron entry and the command executed by Apigee for log rotation. If you want to use logrotate but don't want to use Apigee's configs, you can create your own logrotate configuration file and use a similar logrotate command to perform log rotation.