You're viewing **Apigee Edge** documentation. [Go to the **Apigee X** documentation](http://cloud.google.com/apigee/docs). info *Edge Microgateway v. 3.3.x* ## Audience This topic is intended for Edge Microgateway operators who wish to use existing plugins that are installed with the microgateway. It also discusses the spike arrest and quota plugins in detail (both are included with the installation). If you're a developer who wants to develop new plugins, see [Develop custom plugins](https://docs.apigee.com/api-platform/microgateway/3.3.x/develop-custom-plugins). ## What is an Edge Microgateway plugin? A plugin is a Node.js module that adds functionality to Edge Microgateway. Plugin modules follow a consistent pattern and are stored in a location known to Edge Microgateway, enabling the microgateway to discover and load them automatically. Edge Microgateway includes several existing plugins and you can also create custom plugins, as explained in [Develop custom plugins](https://docs.apigee.com/api-platform/microgateway/3.3.x/develop-custom-plugins). ## Existing plugins bundled with Edge Microgateway A number of plugins are provided with Edge Microgateway at installation. The following table describes some of the most commonly used plugins. > [!NOTE] > **Note:** For a complete list of plugins and supporting documentation, see [`microgateway-plugins`](https://github.com/apigee/microgateway-plugins) on GitHub. | Plugin | Enabled by default | Description | |---|---|---| | **analytics** | Yes | Sends analytics data from Edge Microgateway to Apigee Edge. | | **oauth** | Yes | Adds OAuth token and API Key validation to Edge Microgateway. See [Setting up and configuring Edge Microgateway](https://docs.apigee.com/api-platform/microgateway/3.3.x/setting-and-configuring-edge-microgateway). | | **quota** | No | Enforces quota on requests to Edge Microgateway. Uses Apigee Edge to store and manage the quotas. See [Using the quota plugin](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#Using%20the%20quota%20plugin). | | **spikearrest** | No | Protects against traffic spikes and DoS attacks. See [Using the spike arrest plugin](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#Using%20the%20spike%20arrest%20plugin). | | **header-uppercase** | No | A commented, sample proxy intended as a guide to help developers write custom plugins. See [Edge Microgateway sample plugin](https://docs.apigee.com/api-platform/microgateway/3.3.x/develop-custom-plugins#sampleplugins-headeruppercaseplugin). | | **accumulate-request** | No | Accumulates request data into a single object before passing the data to the next handler in the plugin chain. Useful for writing transform plugins that need to operate on a single, accumulated request content object. | | **accumulate-response** | No | Accumulates response data into a single object before passing the data to the next handler in the plugin chain. Useful for writing transform plugins that need to operate on a single, accumulated response content object. | | **transform-uppercase** | No | Transforms request or response data. This plugin represents a best practice implementation of a transform plugin. The example plugin performs a trivial transform (converts request or response data to uppercase); however, it can easily be adapted to perform other kinds of transformations, such as XML to JSON. | | **json2xm**l | No | Transforms request or response data based on accept or content-type headers. For details, [refer to the plugin documentation in GitHub](https://github.com/apigee/microgateway-plugins/tree/master/json2xml). | | **quota-memory** | No | Enforces quota on requests to Edge Microgateway. Stores and manages quotas in local memory. | | **healthcheck** | No | Returns information about the Edge Microgateway process -- memory usage, cpu usage, etc. To use the plugin, call the URL **/healthcheck** on your Edge Microgateway instance. This plugin is intended to be an example that you can use to implement your own health checking plugin. | ## Where to find existing plugins Existing plugins bundled with Edge Microgateway are located here, where `[prefix]` is the `npm` prefix directory. See [Where is Edge Microgateway installed](https://docs.apigee.com/api-platform/microgateway/3.3.x/installing-edge-microgateway#whereisedgemicrogatewayinstalled) if you can't locate this directory. ``` [prefix]/lib/node_modules/edgemicro/node_modules/microgateway-plugins ``` ## Adding and configuring plugins Follow this pattern to add and configure plugins: 1. Stop Edge Microgateway. 2. Open an Edge Microgateway configuration file. For details, see [Making configuration changes](https://docs.apigee.com/api-platform/microgateway/3.3.x/operation-and-configuration-reference-edge-microgateway#makingconfigurationchanges) for options. 3. Add the plugin to the `plugins:sequence` element of the config file, as follows. Plugins are executed in the order they appear in this list. ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - plugin-name ``` 4. Configure the plugin. Some plugins have optional parameters that you can configure in the config file. For example, you can add the following stanza to configure the spike arrest plugin. See [Using the spike arrest plugin](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#Using%20the%20spike%20arrest%20plugin) for more information. ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - spikearrest spikearrest: timeUnit: minute allow: 10 ``` > [!NOTE] > **Note:** You can also create a plugin-specific configuration file. See [Plugin-specific configuration](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#Plugin-specific%20configuration). 5. Save the file. 6. Restart or reload Edge Microgateway, depending on which configuration file you edited. ## Plugin-specific configuration You can override the plugin parameters specified in the config file by creating a plugin-specific configuration in this directory: ``` [prefix]/lib/node_modules/edgemicro/node_modules/microgateway-plugins/config ``` where `[prefix]` is the `npm` prefix directory. See [Where is Edge Microgateway installed](https://docs.apigee.com/api-platform/microgateway/3.3.x/installing-edge-microgateway#whereisedgemicrogatewayinstalled) if you can't locate this directory. `plugins//config/default.yaml`. For example, you could put this block in `plugins/spikearrest/config/default.yaml`, and they will override any other config settings. ``` spikearrest: timeUnit: hour allow: 10000 buffersize: 0 ``` ## Using the spike arrest plugin The spike arrest plugin protects against traffic spikes. It throttles the number of requests processed by an Edge Microgateway instance. ### Adding the spike arrest plugin See [Adding and configuring plugins](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#Adding%20and%20configuring%20plugins). ### Sample configuration for spike arrest ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - spikearrest spikearrest: timeUnit: minute allow: 10 bufferSize: 5 ``` ### Configuration options for spike arrest - **timeUnit**: How often the spike arrest execution window resets. Valid values are second or minute. - **allow** : The maximum number of requests to allow during the timeUnit. See also [If you're running multiple Edge Micro processes](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#multipleprocesses). - **bufferSize** : (optional, default = 0) if bufferSize \> 0, spike arrest stores this number of requests in a buffer. As soon as the next execution "window" occurs, the buffered requests will be processed first. See also [Adding a buffer](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#Adding%20a%20buffer). ### How does spike arrest work? Think of spike arrest as a way to generally protect against traffic spikes rather than as a way to limit traffic to a specific number of requests. Your APIs and backend can handle a certain amount of traffic, and the spike arrest policy helps you smooth traffic to the general amounts you want. The runtime spike arrest behavior differs from what you might expect to see from the literal per-minute or per-second values you enter. For example, say you specify a rate of 30 requests per minute, like this: ``` spikearrest: timeUnit: minute allow: 30 ``` In testing, you might think you could send 30 requests in 1 second, as long as they came within a minute. But that's not how the policy enforces the setting. If you think about it, 30 requests inside a 1-second period could be considered a mini spike in some environments. What actually happens, then? To prevent spike-like behavior, spike arrest smooths the allowed traffic by dividing your settings into smaller intervals, as follows: #### Per-minute rates Per-minute rates get smoothed into requests allowed intervals of seconds. For example, 30 requests per minute gets smoothed like this: 60 seconds (1 minute) / 30 = 2-second intervals, or about 1 request allowed every 2 seconds. A second request inside of 2 seconds will fail. Also, a 31st request within a minute will fail. #### Per-second rates Per-second rates get smoothed into requests allowed in intervals of milliseconds. For example, 10 requests/second gets smoothed like this: 1000 milliseconds (1 second) / 10 = 100-millisecond intervals, or about 1 request allowed every 100 milliseconds . A second request inside of 100ms will fail. Also, an 11th request within a second will fail. #### When the limit is exceeded If the number of requests exceeds the limit within the specified time interval, spike arrest returns this error message with an HTTP 503 status: ``` {"error": "spike arrest policy violated"} ``` #### Adding a buffer You have an option of adding a buffer to the policy. Let's say you set the buffer to 10. You'll see that the API does not return an error immediately when you exceed the spike arrest limit. Instead, requests are buffered (up to the number specified), and the buffered requests are processed as soon as the next appropriate execution window is available. The default **bufferSize** is 0. ### If you're running multiple Edge Micro processes The number of allowed requests depends on the number of Edge Micro worker processes that are running. Spike arrest calculates the allowable number of requests per worker process. By default, the number of Edge Micro processes equals the number of CPUs on the machine where Edge Micro is installed. However, you can configure the number of worker processes when you start Edge Micro using the `--processes` option on the `start` command. For example, if you want spike arrest to trigger at 100 requests in a given time period, and if you start Edge Microgateway with the option `--processes 4`, then set `allow: 25` in the spike arrest config. In summary, the rule of thumb is to set the `allow` config parameter to the value "desired spike arrest count / number of processes". ## Using the quota plugin A quota specifies the number of request messages that an app is allowed to submit to an API over the course of an hour, day, week, or month. When an app reaches its quota limit, subsequent API calls are rejected. See also [What's the difference between spike arrest and quota](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#usingthequotaplugin-whatsthedifferencebetweenspikearrestandquota)?. > [!NOTE] > **Note:** To use quota, OAuth authentication must be enabled in the Edge Microgateway config file **AND** the quota plugin must be placed **after** the oauth plugin in the sequence list. Quotas are only enforced when client authentication is enabled. The [Setting up and > configuring Edge Microgateway](https://docs.apigee.com/api-platform/microgateway/3.3.x/setting-and-configuring-edge-microgateway) covers all the steps required to make authenticated API calls. ### Adding the quota plugin See [Adding and configuring plugins](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#Adding%20and%20configuring%20plugins). ### Product configuration in Apigee Edge You configure quotas in the Apigee Edge UI where you configure API products. You need to know which product contains the microgateway-aware proxy that you want to limit with a quota. This product must be added to a developer app. When you make API calls that are authenticated using keys in the developer app, the quota will be applied to those API calls. 1. Log in to your Apigee Edge organization account. 2. In the Edge UI, open the product associated with the microgateway-aware proxy to which you want to apply the quota. 1. In the UI, select **Products** from the Publish menu. 2. Open the product containing the API to which you want to apply the quota. 3. Click **Edit**. 4. In the Quota field, specify the quota interval. For example, 100 requests every one minute. Or 50000 requests every 2 hours. ![](https://docs.apigee.com/static/api-platform/images/em-admin-fig-7.png) 3. Click **Save**. 4. Be sure the product is added to a developer app. You'll need the keys from this app to make authenticated API calls. ### Sample configuration for quota > [!NOTE] > **Note:** You must place the **quota** plugin **after** the **oauth** plugin in the sequence list. ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - quota ``` ### Configuration options for quota To configure the quota plugin, add the `quotas` element to your configuration file, as shown in the following example: ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - quota quotas: bufferSize: hour: 20000 minute: 500 month: 1 default: 10000 useDebugMpId: true failOpen: true isHTTPStatusTooManyRequestEnabled: true ... ``` | Option | Description | |---|---| | `bufferSize` | (Integer) The `bufferSize` configuration allows you to adjust how frequently Edge Microgateway syncs its quota count with Apigee Edge. To understand `bufferSize`, consider the following example configuration: ``` quotas: bufferSize: minute: 500 default: 10000 useDebugMpId: true failOpen: true ``` By default, the microgateway syncs its quota counter with Apigee Edge every 5 seconds if the quota interval is set to "minute". The above configuration says that if the quota interval is set in the API product to "minute", Edge Microgateway will sync with Edge to obtain the current quota count after every 500 requests or after 5 seconds, whichever comes first. For more information, see [Understanding how quotas are counted](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#understanding-how-quotas-are-counted). Allowed time units include: `minute`, `hour`, `day`, `week`, `month`, and `default`. | | `isHTTPStatusTooManyRequestEnabled` | Configures the quota plugin to return an HTTP 429 response status instead of status 403 if there is a quota violation. Default: `false`. By default, or if the flag is set to `false`, quota returns the HTTP status `403` when the quota is exceeded. If the flag is set to `true`, quota returns the HTTP status `429` when the quota is exceeded. To change the default HTTP return status to `429`, use the following configuration: ``` edgemicro: ... quotas: isHTTPStatusTooManyRequestEnabled: true ``` ... | | `failOpen` | When this feature is enabled, if a quota-processing error occurs or if the "quota apply" request to Edge fails to update remote quota counters, the quota will be processed based on local counts only until the next successful remote quota sync happens. In both of these cases, a `quota-failed-open` flag is set in the request object. To enable the quota "fail open" feature, set the following configuration: ``` edgemicro: ... quotas: failOpen: true ``` ... | | `useDebugMpId` | Set this flag to `true` to enable the logging of the MP (message processor) ID in quota responses. To use this feature, you must set the following configuration: ``` edgemicro: ... quotas: useDebugMpId: true ... ``` When `useDebugMpId` is set, quota responses from Edge will contain the MP id and will be logged by Edge Microgateway. For example: ``` { "allowed": 20, "used": 3, "exceeded": 0, "available": 17, "expiryTime": 1570748640000, "timestamp": 1570748580323, "debugMpId": "6a12dd72-5c8a-4d39-b51d-2c64f953de6a" } ``` | | `useRedis` | If set to `true` the plugin uses Redis for the quota backing store. For details, see [Using a Redis backing store for quota](https://docs.apigee.com/api-platform/microgateway/3.3.x/use-plugins#using-a-redis-backing-store-for-quota). | ### Understanding how quotas are counted By default, the microgateway syncs its quota counter with Apigee Edge every 5 seconds if the quota interval is set to "minute". If the interval is set to a level higher than "minute", such as "week" or "month", the default refresh period is 1 minute. It's important to note that you specify quota intervals in the API products that are defined on Apigee Edge. Quota intervals specify how many requests are allowed for a minute, hour, day, week, or month. For example, Product A might have a quota interval of 100 requests per minute and Product B might have a quota interval of 10,000 requests per hour. The Edge Microgateway `quota` plugin's YAML configuration does **not** set the quota interval; rather, it provides a way to adjust the frequency at which a local Edge Microgateway instance syncs its quota count with Apigee Edge. For example, assume there are three API products defined in Apigee Edge with the following quota intervals specified: - Product A has a quota of 100 requests per minute - Product B has a quota of 5000 requests per hour - Product C has a quota of 1000000 requests per month With those quota settings in mind, how should the Edge Microgateway `quota` plugin be configured? The best practice is to configure Edge Microgateway with sync intervals that are lower than the quota intervals defined in the API products. For example: ``` quotas: bufferSize: hour: 2000 minute: 50 month: 1 default: 10000 ``` This configuration defines the following sync intervals for the API products described previously: - Product A is set to the "minute" interval. Edge Microgateway will sync to Edge after every 50th request or 5 seconds, whichever comes first. - Product B is set to the "hour" interval. Edge Microgateway will sync to Edge after every 2000th request or 1 minute, whichever comes first. - Product C is set to the "month" interval. Edge Microgateway will sync to Edge after every single request or 1 minute, whichever comes first. Every time a microgateway instance syncs with Edge, the microgateway's quota count is set to the retrieved quota count. The `bufferSize` settings allow you adjust how the quota counter is synced with Edge. In high-traffic situations, the `bufferSize` settings allow the buffer counter to sync before the default time-based sync is triggered. ### Understanding quota scope The quota count is scoped to an environment in an organization. To achieve this scope, Edge Microgateway creates a quota identifier that is a combination of "org + env + appName + productName". ### Using a Redis backing store for quota To use a Redis backing store for quota, use the same configuration used for the Synchronizer feature. Following is the basic configuration required to use Redis for quota storage: ``` edgemicro: redisHost: localhost redisPort: 6379 redisDb: 2 redisPassword: codemaster quotas: useRedis: true ``` For details on the `edgemicro.redis*` parameters, see [Using the synchronizer](https://docs.apigee.com/api-platform/microgateway/3.3.x/operation-and-configuration-reference-edge-microgateway#synchronizer). > [!NOTE] > You can use the environment variable EDGEMICRO_REDIS_PASSWORD to override the `edgemicro.redisPassword` parameter See also [Setting configuration attributes with environment variable values](https://docs.apigee.com/api-platform/microgateway/3.3.x/operation-and-configuration-reference-edge-microgateway#setting-configuration-attributes-with-environment-variable-values). ### Testing the quota plugin When the quota is exceeded, an HTTP 403 status is returned to the client, along with the following message: ``` {"error": "exceeded quota"} ``` ### What's the difference between spike arrest and quota? It's important to choose the right tool for the job at hand. Quota policies configure the number of request messages that a client app is allowed to submit to an API over the course of an hour, day, week, or month. The quota policy enforces consumption limits on client apps by maintaining a distributed counter that tallies incoming requests. Use a quota policy to enforce business contracts or SLAs with developers and partners, rather than for operational traffic management. For example, a quota might be used to limit traffic for a free service, while allowing full access for paying customers. Use spike arrest to protect against sudden spikes in API traffic. Typically, spike arrest is used to head off possible DDoS or other malicious attacks.