Apache Cassandra maintenance tasks

Edge for Private Cloud v4.18.05

Anti-Entropy Maintenance

The Apache Cassandra ring nodes require periodic maintenance to ensure consistency across all nodes. To perform this maintenance, use the Cassandra "nodetool -h localhost repair" command.

Note: This maintenance must be run on every Cassandra node at least every seven days in order to eliminate problems related to Cassandra "forgotten deletes". Running nodetool -h localhost repair imposes a significant load on the system, so Apigee recommends that this process only be run during periods of relatively low workload.

You should also run the command on one Cassandra node at a time. You can run it on different nodes on different days, or schedule it so that there are several hours between running it on each node.

In a single-region or single-data-center Apigee installation, use the command nodetool -h localhost repair on one Cassandra node to ensure consistency across all nodes in the ring. In a multi-region or multi-data-center Apigee installation, use nodetool -h localhost repair -pr on every node in the ring (across all regions or data centers).

For more information, see the following resources:

Log File Maintenance

Cassandra logs are stored in the /opt/apigee/var/log/cassandra directory on each node. By default, a maximum of 50 log files, each with a maximum size of 20 MB, can be created; once this limit is reached older logs are deleted when newer logs are created.

If you should find that Cassandra log files are taking up excessive space, you can modify the amount of space allocated for log files by editing the log4j settings.

Edit /opt/apigee/customer/application/cassandra.properties to set the following properties. If that file does not exist, create it:

conf_log4j-server_log4j.appender.r.maxfilesize=20MB
# max file size
conf_log4j-server_log4j.appender.r.maxbackupindex=50 # max open files

Restart Cassandra by using the following command:

/opt/apigee/apigee-service/bin/apigee-service apigee-cassandra restart