QPID operations guide

Edge for Private Cloud version 4.52.02 ships with Qpid Broker-J 8.0.6. This page describes memory and disk considerations for the apigee-qpidd component of Edge for Private Cloud (with Java-based Broker). The sections below apply specifically to Apigee's Qpid use case and explain how to tune Qpid using Apigee's configuration mechanisms.

Qpid in Apigee

In Apigee, Qpid is used in two main workflows:

  • Analytics
  • Monetization

In both of these workflows, the Message Processor generates messages, which are collected into batches and pushed to Qpid. For Analytics, the messages are read by Qpid-server; for Monetization, they are read by Rating Server (part of management-server). Once read, the batched messages are unbatched and ingested.

The following sections describe the recommended configurations for memory and disk with Qpid. These configurations are based on a single message size of 2KB for Analytics or Monetization. With a default batch size of 50, each message in Qpid would be approximately 100 kb. Apigee allows proxies to modify and add additional information to a message (Analytics or Monetization). In extreme cases, this may significantly change the size of a single message and the overall batched message in Qpid. While the general recommendations regarding memory and disk in the sections below hold true even in such cases, you may need to perform additional tweaks to Qpid to ensure smooth operations.

Memory

Qpid Broker uses two types of memory: off-heap memory and heap memory. Qpid stores messages in off-heap memory, and uses heap memory for its own internal tasks, direct messages to queues and exchanges, etc. Under most circumstances, Apigee recommends that you keep a 3:1 ratio between off-heap and heap memory for Qpid Broker.

Messages in the off-heap memory persist on disk. Additionally, if total utilization of off-heap memory breaches a certain threshold, Qpid is configured to overflow messages to disk.

By default, Apigee calculates heap and off-heap memory of Qpid Broker based on available memory on the node. Apigee's scripts have a soft limit of 4 GB combined for Qpid Broker's heap and off-heap memory. If you would like to override these limits, you can use the two configurations below to set your own values for Qpid's heap and off-heap memory.

  • Configuration for heap memory: bin_run-qpidd_qpid.run.jvm.max_vm_heap=1024m
  • Configuration for off-heap memory: bin_run-qpidd_qpid.run.jvm.max_direct_memory_size=3072m

See How to change a Qpid configuration for information on how to override configurations.

For more information, see Apache Qpid Reference for Memory.

Disk

Qpid Broker's data directory is $APIGEE_ROOT/data/apigee-qpidd. Qpid Broker's logs are stored in $APIGEE_ROOT/var/log/apigee-qpidd.

The performance of the file system hosting the data directory is key to the performance of Broker as a whole. For best performance, choose a device that has low latency and that is not used by other applications.

The Qpid Broker persists its messages in the data directory. By default, the Broker will accept messages until the file system hosting the data directory is 80% full. You can modify this threshold by overriding the configuration shown in the example below on each Qpid node.

bin_run-qpidd_qpid.store.filesystem.maxUsagePercent=60

In this example, the Broker rejects new messages when the underlying file system is 60% full.

How to change a Qpid configuration

To change a Qpid configuration:

  1. Create or edit the following file: 
    $APIGEE_ROOT/customer/application/qpidd.properties
  2. Add or remove or modify configuration as necessary. For example, to change max heap memory of Qpid Broker to 6 GB, add the following configuration override in the file: 
    bin_run-qpidd_qpid.run.jvm.max_vm_heap=6144M
  3. Save the changes and ensure file is owned and readable by the apigee user: 
    chown apigee:apigee $APIGEE_ROOT/customer/application/qpidd.properties
  4. Restart the qpidd process: 
    apigee-service apigee-qpidd restart