You're viewing **Apigee Edge** documentation. [Go to the **Apigee X** documentation](http://cloud.google.com/apigee/docs). info *Edge Microgateway v. 3.0.x* ## Audience This topic is intended for Edge Microgateway operators who wish to use existing plugins that are installed with the microgateway. It also discusses the spike arrest and quota plugins in detail (both are included with the installation). If you're a developer who wants to develop new plugins, see [Develop custom plugins](https://docs.apigee.com/api-platform/microgateway/3.0.x/develop-custom-plugins). ## What is an Edge Microgateway plugin? A plugin is a Node.js module that adds functionality to Edge Microgateway. Plugin modules follow a consistent pattern and are stored in a location known to Edge Microgateway, enabling the microgateway to discover and load them automatically. Edge Microgateway includes several existing plugins and you can also create custom plugins, as explained in [Develop custom plugins](https://docs.apigee.com/api-platform/microgateway/3.0.x/develop-custom-plugins). ## Existing plugins bundled with Edge Microgateway Several existing plugins are provided with Edge Microgateway at installation. These include: | Plugin | Enabled by default | Description | |---|---|---| | **analytics** | Yes | Sends analytics data from Edge Microgateway to Apigee Edge. | | **oauth** | Yes | Adds OAuth token and API Key validation to Edge Microgateway. See [Setting up and configuring Edge Microgateway](https://docs.apigee.com/api-platform/microgateway/3.0.x/setting-and-configuring-edge-microgateway). | | **quota** | No | Enforces quota on requests to Edge Microgateway. Uses Apigee Edge to store and manage the quotas. See [Using the quota plugin](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#Using%20the%20quota%20plugin). | | **spikearrest** | No | Protects against traffic spikes and DoS attacks. See [Using the spike arrest plugin](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#Using%20the%20spike%20arrest%20plugin). | | **header-uppercase** | No | A commented, sample proxy intended as a guide to help developers write custom plugins. See [Edge Microgateway sample plugin](https://docs.apigee.com/api-platform/microgateway/3.0.x/develop-custom-plugins#sampleplugins-headeruppercaseplugin). | | **accumulate-request** | No | Accumulates request data into a single object before passing the data to the next handler in the plugin chain. Useful for writing transform plugins that need to operate on a single, accumulated request content object. | | **accumulate-response** | No | Accumulates response data into a single object before passing the data to the next handler in the plugin chain. Useful for writing transform plugins that need to operate on a single, accumulated response content object. | | **transform-uppercase** | No | Transforms request or response data. This plugin represents a best practice implementation of a transform plugin. The example plugin performs a trivial transform (converts request or response data to uppercase); however, it can easily be adapted to perform other kinds of transformations, such as XML to JSON. | | **json2xm**l | No | Transforms request or response data based on accept or content-type headers. For details, [refer to the plugin documentation in GitHub](https://github.com/apigee/microgateway-plugins/tree/master/json2xml). | | **quota-memory** | No | Enforces quota on requests to Edge Microgateway. Stores and manages quotas in local memory. | | **healthcheck** | No | Returns information about the Edge Microgateway process -- memory usage, cpu usage, etc. To use the plugin, call the URL **/healthcheck** on your Edge Microgateway instance. This plugin is intended to be an example that you can use to implement your own health checking plugin. | ## Where to find existing plugins Existing plugins bundled with Edge Microgateway are located here, where `[prefix]` is the `npm` prefix directory. See [Where is Edge Microgateway installed](https://docs.apigee.com/api-platform/microgateway/3.0.x/installing-edge-microgateway#whereisedgemicrogatewayinstalled) if you can't locate this directory. ``` [prefix]/lib/node_modules/edgemicro/node_modules/microgateway-plugins ``` ## Adding and configuring plugins Follow this pattern to add and configure plugins: 1. Stop Edge Microgateway. 2. Open an Edge Microgateway configuration file. For details, see [Making configuration changes](https://docs.apigee.com/api-platform/microgateway/3.0.x/operation-and-configuration-reference-edge-microgateway#makingconfigurationchanges) for options. 3. Add the plugin to the `plugins:sequence` element of the config file, as follows. Plugins are executed in the order they appear in this list. ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - plugin-name ``` 4. Configure the plugin. Some plugins have optional parameters that you can configure in the config file. For example, you can add the following stanza to configure the spike arrest plugin. See [Using the spike arrest plugin](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#Using%20the%20spike%20arrest%20plugin) for more information. ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - spikearrest spikearrest: timeUnit: minute allow: 10 ``` > [!NOTE] > **Note:** You can also create a plugin-specific configuration file. See [Plugin-specific configuration](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#Plugin-specific%20configuration). 5. Save the file. 6. Restart or reload Edge Microgateway, depending on which configuration file you edited. ## Plugin-specific configuration You can override the plugin parameters specified in the config file by creating a plugin-specific configuration in this directory: ``` [prefix]/lib/node_modules/edgemicro/node_modules/microgateway-plugins/config ``` where `[prefix]` is the `npm` prefix directory. See [Where is Edge Microgateway installed](https://docs.apigee.com/api-platform/microgateway/3.0.x/installing-edge-microgateway#whereisedgemicrogatewayinstalled) if you can't locate this directory. `plugins//config/default.yaml`. For example, you could put this block in `plugins/spikearrest/config/default.yaml`, and they will override any other config settings. ``` spikearrest: timeUnit: hour allow: 10000 buffersize: 0 ``` ## Using the spike arrest plugin The spike arrest plugin protects against traffic spikes. It throttles the number of requests processed by an Edge Microgateway instance. ### Adding the spike arrest plugin See [Adding and configuring plugins](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#Adding%20and%20configuring%20plugins). ### Sample configuration for spike arrest ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - spikearrest spikearrest: timeUnit: minute allow: 10 bufferSize: 5 ``` ### Configuration options for spike arrest - **timeUnit**: How often the spike arrest execution window resets. Valid values are second or minute. - **allow** : The maximum number of requests to allow during the timeUnit. See also [If you're running multiple Edge Micro processes](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#multipleprocesses). - **bufferSize** : (optional, default = 0) if bufferSize \> 0, spike arrest stores this number of requests in a buffer. As soon as the next execution "window" occurs, the buffered requests will be processed first. See also [Adding a buffer](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#Adding%20a%20buffer). ### How does spike arrest work? Think of spike arrest as a way to generally protect against traffic spikes rather than as a way to limit traffic to a specific number of requests. Your APIs and backend can handle a certain amount of traffic, and the spike arrest policy helps you smooth traffic to the general amounts you want. The runtime spike arrest behavior differs from what you might expect to see from the literal per-minute or per-second values you enter. For example, say you specify a rate of 30 requests per minute, like this: ``` spikearrest: timeUnit: minute allow: 30 ``` In testing, you might think you could send 30 requests in 1 second, as long as they came within a minute. But that's not how the policy enforces the setting. If you think about it, 30 requests inside a 1-second period could be considered a mini spike in some environments. What actually happens, then? To prevent spike-like behavior, spike arrest smooths the allowed traffic by dividing your settings into smaller intervals, as follows: #### Per-minute rates Per-minute rates get smoothed into requests allowed intervals of seconds. For example, 30 requests per minute gets smoothed like this: 60 seconds (1 minute) / 30 = 2-second intervals, or about 1 request allowed every 2 seconds. A second request inside of 2 seconds will fail. Also, a 31st request within a minute will fail. #### Per-second rates Per-second rates get smoothed into requests allowed in intervals of milliseconds. For example, 10 requests/second gets smoothed like this: 1000 milliseconds (1 second) / 10 = 100-millisecond intervals, or about 1 request allowed every 100 milliseconds . A second request inside of 100ms will fail. Also, an 11th request within a second will fail. #### When the limit is exceeded If the number of requests exceeds the limit within the specified time interval, spike arrest returns this error message with an HTTP 503 status: ``` {"error": "spike arrest policy violated"} ``` #### Adding a buffer You have an option of adding a buffer to the policy. Let's say you set the buffer to 10. You'll see that the API does not return an error immediately when you exceed the spike arrest limit. Instead, requests are buffered (up to the number specified), and the buffered requests are processed as soon as the next appropriate execution window is available. The default **bufferSize** is 0. ### If you're running multiple Edge Micro processes The number of allowed requests depends on the number of Edge Micro worker processes that are running. Spike arrest calculates the allowable number of requests per worker process. By default, the number of Edge Micro processes equals the number of CPUs on the machine where Edge Micro is installed. However, you can configure the number of worker processes when you start Edge Micro using the `--processes` option on the `start` command. For example, if you want spike arrest to trigger at 100 requests in a given time period, and if you start Edge Microgateway with the option `--processes 4`, then set `allow: 25` in the spike arrest config. In summary, the rule of thumb is to set the `allow` config parameter to the value "desired spike arrest count / number of processes". ## Using the quota plugin A quota specifies the number of request messages that an app is allowed to submit to an API over the course of an hour, day, week, or month. When an app reaches its quota limit, subsequent API calls are rejected. See also [What's the difference between spike arrest and quota](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#usingthequotaplugin-whatsthedifferencebetweenspikearrestandquota)?. > [!NOTE] > **Note:** To use quota, OAuth authentication must be enabled in the Edge Microgateway config file **AND** the quota plugin must be placed **after** the oauth plugin in the sequence list. Quotas are only enforced when client authentication is enabled. The [Setting up and > configuring Edge Microgateway](https://docs.apigee.com/api-platform/microgateway/3.0.x/setting-and-configuring-edge-microgateway) covers all the steps required to make authenticated API calls. ### Adding the quota plugin See [Adding and configuring plugins](https://docs.apigee.com/api-platform/microgateway/3.0.x/use-plugins#Adding%20and%20configuring%20plugins). ### Product configuration in Apigee Edge You configure quotas in the Apigee Edge UI where you configure API products. You need to know which product contains the microgateway-aware proxy that you want to limit with a quota. This product must be added to a developer app. When you make API calls that are authenticated using keys in the developer app, the quota will be applied to those API calls. 1. Log in to your Apigee Edge organization account. 2. In the Edge UI, open the product associated with the microgateway-aware proxy to which you want to apply the quota. 1. In the UI, select **Products** from the Publish menu. 2. Open the product containing the API to which you want to apply the quota. 3. Click **Edit**. 4. In the Quota field, specify the quota interval. For example, 100 requests every one minute. Or 50000 requests every 2 hours. ![](https://docs.apigee.com/static/api-platform/images/em-admin-fig-7.png) 3. Click **Save**. 4. Be sure the product is added to a developer app. You'll need the keys from this app to make authenticated API calls. ### Sample configuration for quota > [!NOTE] > **Note:** You must place the **quota** plugin **after** the **oauth** plugin in the sequence list. ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - quota ``` ### Configuration options for quota To configure the quota plugin, add the `quotas` element to your configuration file, as shown in the following example: ``` edgemicro: home: ../gateway port: 8000 max_connections: -1 max_connections_hard: -1 logging: level: info dir: /var/tmp stats_log_interval: 60 plugins: dir: ../plugins sequence: - oauth - quota quotas: bufferSize: hour: 20000 minute: 500 month: 1 default: 10000 useDebugMpId: true failOpen: true useRedis: true redisHost: localhost redisPort: 6379 redisDb: 1 ... ``` | Option | Description | |---|---| | `buffersize` | (Integer) The buffersize to set for the specified time interval. Allowed time units include: `hour`, `minute`, `day`, `week`, `month`, and `default`. (Added: Version 3.0.9) | | `failOpen` | When this feature is enabled, if a quota-processing error occurs or if the "quota apply" request to Edge fails to update remote quota counters, the quota will be processed based on local counts only until the next successful remote quota sync happens. In both of these cases, a `quota-failed-open` flag is set in the request object. (Added: Version 3.0.9) To enable the quota "fail open" feature, set the following configuration: ``` edgemicro: ... quotas: failOpen: true ``` ... | | `useDebugMpId` | Set this flag to `true` to enable the logging of the MP (message processor) ID in quota responses. (Added: Version 3.0.9) To use this feature, you must update your `edgemicro-auth` proxy to version 3.0.7 or higher and set the following configuration: ``` edgemicro: ... quotas: useDebugMpId: true ... ```
When `useDebugMpId` is set, quota responses from Edge will contain the MP id and will be logged by Edge Microgateway. For example: ``` { "allowed": 20, "used": 3, "exceeded": 0, "available": 17, "expiryTime": 1570748640000, "timestamp": 1570748580323, "debugMpId": "6a12dd72-5c8a-4d39-b51d-2c64f953de6a" } ``` | | `useRedis` | (Boolean) Set to `true` to use the Redis quota database module. When set, the quota is restricted to only those Edge Microgateway instances that connect to Redis. Otherwise, the quota counter is global. Default: `false` (The `redis-volos-apigee` module is used) (Added: Version 3.0.10) | | `redisHost` | The host where your Redis instance is running. Default: 127.0.0.1 (Added: Version 3.0.10) | | `redisPort` | The port of the Redis instance. Default: 6379 (Added: Version 3.0.10) | | `redisDb` | The Redis DB to use. Default: 0 (Added: Version 3.0.10) | ### Understanding quota scope The quota count is scoped to an API product. If a developer app has multiple products, the quota is scoped to each one individually. To achieve this scope, Edge Microgateway creates a quota identifier that is a combination of "appName + productName". ### Testing the quota plugin When the quota is exceeded, an HTTP 403 status is returned to the client, along with the following message: ``` {"error": "exceeded quota"} ``` ### What's the difference between spike arrest and quota? It's important to choose the right tool for the job at hand. Quota policies configure the number of request messages that a client app is allowed to submit to an API over the course of an hour, day, week, or month. The quota policy enforces consumption limits on client apps by maintaining a distributed counter that tallies incoming requests. Use a quota policy to enforce business contracts or SLAs with developers and partners, rather than for operational traffic management. For example, a quota might be used to limit traffic for a free service, while allowing full access for paying customers. Use spike arrest to protect against sudden spikes in API traffic. Typically, spike arrest is used to head off possible DDoS or other malicious attacks.