SpikeArrest policy

You're viewing Apigee Edge documentation.
View Apigee X documentation.

The Spike Arrest policy protects against traffic surges with the <Rate> element. This element throttles the number of requests processed by an API proxy and sent to a backend, protecting against performance lags and downtime.

How Spike Arrest works

Think of Spike Arrest as a way to generally protect against traffic spikes rather than as a way to limit traffic to a specific number of requests. Your APIs and backend can handle a certain amount of traffic, and the Spike Arrest policy helps you smooth traffic to the general amounts you want.

The runtime Spike Arrest behavior differs from what you might expect to see from the literal per-minute or per-second values you enter.

For example, say you enter a rate of 30pm (30 requests per minute). In testing, you might think you could send 30 requests in 1 second, as long as they came within a minute. But that's not how the policy enforces the setting. If you think about it, 30 requests inside a 1-second period could be considered a mini spike in some environments.

What actually happens, then? To prevent spike-like behavior, Spike Arrest smooths the number of full requests allowed by dividing your settings into smaller intervals:

Per-minute rates get smoothed into full requests allowed in intervals of seconds.

For example, 30pm gets smoothed like this:
60 seconds (1 minute) / 30pm = 2-second intervals, or 1 request allowed every 2 seconds. A second request inside of 2 seconds will fail. Also, a 31st request within a minute will fail.
Per-second rates get smoothed into full requests allowed in intervals of milliseconds.
For example, 10ps gets smoothed like this:
1000 milliseconds (1 second) / 10ps = 100-millisecond intervals, or 1 request allowed every 100 milliseconds. A second request inside of 100ms will fail. Also, an 11th request within a second will fail.

There's more: 1 request * number of Message Processors

By default, Spike Arrest is not distributed unless you enable <UseEffectiveCount>. That means request counts are not synchronized across MPs. With more than one Message Processor, especially those with a round-robin configuration, each handles its own Spike Arrest throttling independently. With one message processor, a 30pm rate smooths traffic to 1 request every 2 seconds (60 / 30). With two message processors (the default for Edge cloud), that number doubles to 2 requests every 2 seconds. So multiply your calculated number of full requests per interval by the number of Message Processors to get your overall arrest rate.

The difference between spike arrest and quota

The Quota policy configures the number of request messages that a client app is allowed to submit to an API over the course of an hour, day, week, or month. The quota policy enforces consumption limits on client apps by maintaining a distributed counter that tallies incoming requests.

Use Quota to enforce business contracts or SLAs with developers and partners, rather than for operational traffic management. Use Spike Arrest to protect against sudden spikes in API traffic. See also Comparing Quota, Spike Arrest, and Concurrent Rate Limit Policies.

Videos

These videos show you how to protect your APIs against traffic spikes using the Spike Arrest policy:

Why You Need It

How to Configure

Attach Policy

Per Second Rate Limiting

Per Minute Rate Limiting

Unique Rate Limiting

Compare Quota Policy

Flow Variables

`<SpikeArrest>` element

Defines the Spike Arrest policy.

Default Value	See Default Policy tab, below
Required?	Optional
Type	Complex object
Parent Element	n/a
Child Elements	`<Identifier>` `<MessageWeight>` `<Rate>` (Required) `<UseEffectiveCount>`

Syntax

The <SpikeArrest> element uses the following syntax:

<SpikeArrest
  continueOnError="[false|true]"
  enabled="[true|false]"
  name="policy_name"
>
  <DisplayName>display_name</DisplayName>
  <Properties/>
  <Identifier ref="flow_variable"/>
  <MessageWeight ref="flow_variable"/>
  <Rate ref="flow_variable">rate[pm|ps]</Rate>
  <UseEffectiveCount>[false|true]</UseEffectiveCount>
</SpikeArrest>

Default Policy

The following example shows the default settings when you add a Spike Arrest policy to your flow in the Edge UI:

<SpikeArrest async="false" continueOnError="false" enabled="true" name="Spike-Arrest-1">
  <DisplayName>Spike Arrest-1</DisplayName>
  <Properties/>
  <Identifier ref="request.header.some-header-name"/>
  <MessageWeight ref="request.header.weight"/>
  <Rate>30ps</Rate>
  <UseEffectiveCount>true</UseEffectiveCount>
</SpikeArrest>

This element has the following attributes that are common to all policies:

Attribute	Default	Required?	Description
`name`	N/A	Required	The internal name of the policy. The value of the `name` attribute can contain letters, numbers, spaces, hyphens, underscores, and periods. This value cannot exceed 255 characters. Optionally, use the `<DisplayName>` element to label the policy in the management UI proxy editor with a different, natural-language name.
`continueOnError`	false	Optional	Set to "false" to return an error when a policy fails. This is expected behavior for most policies. Set to "true" to have flow execution continue even after a policy fails.
`enabled`	true	Optional	Set to "true" to enforce the policy. Set to "false" to "turn off" the policy. The policy will not be enforced even if it remains attached to a flow.
`async`	false	Deprecated	This attribute is deprecated.

Examples

The following examples show some of the ways in which you can use the Spike Arrest policy:

Example 1

The following example sets the rate to five per second:

<SpikeArrest name="Spike-Arrest-1">
  <Rate>5ps</Rate>
</SpikeArrest>

The policy smoothes the rate to one request allowed every 200 milliseconds (1000/5).

Example 2

The following example sets the rate to 12 per minute:

<SpikeArrest name="Spike-Arrest-1">
  <Rate>12pm</Rate>
</SpikeArrest>

This example policy smoothes the rate to one request allowed every five seconds (60/12).

Example 3

The following example restricts requests to 12 per minute (one request allowed every five seconds, or 60/12):

<SpikeArrest name="Spike-Arrest-1">
  <Rate>12pm</Rate>
  <Identifier ref="client_id" />
  <MessageWeight ref="request.header.weight" />
</SpikeArrest>

In addition, the <MessageWeight> element accepts a custom value (the weight header) that adjusts message weights for specific apps or clients. This provides additional control over throttling for entities that are identified with the <Identifier> element.

Example 4

The following example instructs Spike Arrest to look for a runtime value set via the request that is passed in as the request.header.runtime_rate flow variable:

<SpikeArrest name="Spike-Arrest-1">
  <Rate ref="request.header.runtime_rate" />
</SpikeArrest>

The value of the flow variable must be in the form of intpm or intps.

To try this example, execute a request like the following:

curl http://myorg-myenv.apigee.net/price -H 'runtime_rate:30ps'

Child element reference

This section describes the child elements of <SpikeArrest>.

`<DisplayName>`

Use in addition to the name attribute to label the policy in the management UI proxy editor with a different, more natural-sounding name.

The <DisplayName> element is common to all policies.

Default Value	n/a
Required?	Optional. If you omit `<DisplayName>`, the value of the policy's `name` attribute is used
Type	String
Parent Element	<`PolicyElement`>
Child Elements	None

The <DisplayName> element uses the following syntax:

Syntax

<PolicyElement>
  <DisplayName>policy_display_name</DisplayName>
  ...
</PolicyElement>

Example

<PolicyElement>
  <DisplayName>My Validation Policy</DisplayName>
</PolicyElement>

The <DisplayName> element has no attributes or child elements.

`<Identifier>`

Lets you choose how to group the requests so that the Spike Arrest policy can be applied based on the client. For example, you can group requests by developer ID, in which case each developer's requests will count towards their own Spike Arrest rate and not all requests to the proxy.

Use in conjunction with <MessageWeight> element for more fine-grained control over request throttling.

If you leave the <Identifier> element empty, one rate limit is enforced for all requests into that API proxy.

Default Value	n/a
Required?	Optional
Type	String
Parent Element	`<SpikeArrest>`
Child Elements	None

Syntax

<SpikeArrest
  continueOnError="[false|true]"
  enabled="[true|false]"
  name="policy_name"
>
  <Identifier ref="flow_variable"/>
</SpikeArrest>

Example 1

The following example applies the Spike Arrest policy per developer ID:

<SpikeArrest name="Spike-Arrest-1">
  <Identifier ref="developer.id"/>
  <Rate>42pm</Rate/>
</SpikeArrest>

The following table describes the attributes of <Identifier>:

Attribute	Description	Default	Presence
`ref`	Identifies the variable by which Spike Arrest groups incoming requests. You can use any flow variable to indicate a unique client, such those available with the VerifyAPIKey policy. You can also set custom variables using the JavaScript policy or the AssignMessage policy.	n/a	Required

This element is also discussed in the following Apigee Community post: http://community.apigee.com/questions/2807/how-does-the-edge-quota-policy-work-when-no-identi.html.

`<MessageWeight>`

Specifies the weighting defined for each message. Message weight modifies the impact of a single request on the calculation of the Spike Arrest rate. Message weight can be any flow variable, such as an HTTP header, query parameter, form parameter, or message body content. You can also use custom variables using the JavaScript policy or the AssignMessage policy.

Use in conjunction with <Identifier> to further throttle requests by specific clients or apps.

For example, if the Spike Arrest <Rate> is 10pm, and an app submits requests with a weight of 2, then only five messages per minute are permitted from that client because each request counts as 2.

Default Value	n/a
Required?	Optional
Type	Integer
Parent Element	`<SpikeArrest>`
Child Elements	None

Syntax

<SpikeArrest
  continueOnError="[false|true]"
  enabled="[true|false]"
  name="policy_name"
>
  <MessageWeight ref="flow_variable"/>
</SpikeArrest>

Example 1

The following example restricts requests to 12 per minute (one request allowed every five seconds, or 60/12):

<SpikeArrest name="Spike-Arrest-1">
  <Rate>12pm</Rate>
  <Identifier ref="client_id" />
  <MessageWeight ref="request.header.weight" />
</SpikeArrest>

In this example, <MessageWeight> accepts a custom value (the weight header in the request) that adjusts message weights for specific clients. This provides additional control over throttling for entities that are identified with the <Identifier> element.

The following table describes the attributes of <MessageWeight>:

Attribute	Description	Presence	Default
`ref`	Identifies the flow variable that contains the message weight for the specific client. This can be any flow variable, such as an HTTP query param, header, or message body content. For more information, see Flow variables reference. You can also set custom variables using the JavaScript policy or the AssignMessage policy.	Required	N/A

`<Rate>`

Specifies the rate at which to limit traffic spikes (or bursts) by setting the number of requests that are allowed in per minute or per second intervals. You can also use this element in conjunction with <Identifier> and <MessageWeight> to smoothly throttle traffic at runtime by accepting values from the client.

Default Value	n/a
Required?	Required
Type	Integer
Parent Element	`<SpikeArrest>`
Child Elements	None

Syntax

You can specify rates in one of the following ways:

A static rate that you specify as the body of the <Rate> element
A variable value, which can be passed by the client; identify the name of the flow variable using the ref attribute

<SpikeArrest
  continueOnError="[false|true]"
  enabled="[true|false]"
  name="policy_name"
>
  <Rate ref="flow_variable">rate[pm|ps]</Rate>
</SpikeArrest>

Valid rate values (either defined as a variable value or in the body of the element) must conform to the following format:

intps (number of requests per second, smoothed into intervals of milliseconds)
intpm (number of requests per minute, smoothed into intervals of seconds)

The value of int must be a positive, non-zero integer.

Example 1

The following example sets the rate to five requests per second:

<SpikeArrest name="Spike-Arrest-1">
  <Rate>5ps</Rate>
</SpikeArrest>

The policy smoothes the rate to one request allowed every 200 milliseconds (1000/5).

Example 2

The following example sets the rate to 12 requests per minute:

<SpikeArrest name="Spike-Arrest-1">
  <Rate>12pm</Rate>
</SpikeArrest>

This example policy smoothes the rate to one request allowed every five seconds (60/12).

The following table describes the attributes of <Rate>:

Attribute Description Presence Default

Attribute	Description	Presence	Default
`ref`	Identifies a flow variable that specifies the rate. This can be any flow variable, such as an HTTP query parameter, header, or message body content, or a value such as a KVM. For more information, see Flow variables reference. You can also use custom variables using the JavaScript policy or the AssignMessage policy. If you define both `ref` and the body of this element, the value of `ref` is applied and takes precedence when the flow variable is set in the request. (The reverse is true when the variable identified in `ref` is not set in the request.) For example: <Rate ref="request.header.custom_rate">1pm</Rate> In this example, if the client does not pass a "custom_rate" header, then the rate for the API proxy is 1 request per minute for all clients. If the client passes a "custom_rate" header, then the rate limit becomes 10 requests per second for all clients on the proxy — until a request without the "custom_rate" header is sent. You can use `<Identifier>` to group requests to enforce custom rates for different type of client. If you specify a value for `ref` but do not set the rate in the body of the `<Rate>` element and the client does not pass a value, then the Spike Arrest policy throws an error.	Optional	n/a

ref

Identifies a flow variable that specifies the rate. This can be any flow variable, such as an HTTP query parameter, header, or message body content, or a value such as a KVM. For more information, see Flow variables reference.

You can also use custom variables using the JavaScript policy or the AssignMessage policy.

If you define both ref and the body of this element, the value of ref is applied and takes precedence when the flow variable is set in the request. (The reverse is true when the variable identified in ref is not set in the request.)

For example:

<Rate ref="request.header.custom_rate">1pm</Rate>

In this example, if the client does not pass a "custom_rate" header, then the rate for the API proxy is 1 request per minute for all clients. If the client passes a "custom_rate" header, then the rate limit becomes 10 requests per second for all clients on the proxy — until a request without the "custom_rate" header is sent.

You can use <Identifier> to group requests to enforce custom rates for different type of client.

If you specify a value for ref but do not set the rate in the body of the <Rate> element and the client does not pass a value, then the Spike Arrest policy throws an error.

Optional

n/a

`<UseEffectiveCount>`

Distributes your Spike Arrest counts across Message Processors (MPs) when using auto-scaling groups.

Note: The <UseEffectiveCount> element is available for Edge Public Cloud and in Edge for the Private Cloud versions 4.18.05 and later.

Syntax

<SpikeArrest
  continueOnError="[false|true]"
  enabled="[true|false]"
  name="policy_name"
>
  <UseEffectiveCount>[false|true]</UseEffectiveCount>
</SpikeArrest>

Example 1

The following example sets <UseEffectiveCount> to true:

<SpikeArrest name='Spike-Arrest-1'>
  <Rate>40ps</Rate>
  <UseEffectiveCount>true</UseEffectiveCount>
</SpikeArrest>

The <UseEffectiveCount> element is optional. The default value is false when the element is omitted from your Spike Arrest policy.

Note: The <UseEffectiveCount> element is included in the default SpikeArrest policy and set to true.

Default Value	False
Required?	Optional
Type	Boolean
Parent Element	`<SpikeArrest>`
Child Elements	None

The following table describes the attributes of the <UseEffectiveCount> element:

Attribute	Description	Default	Presence
`ref`	Identifies the variable that contains the value of `<UseEffectiveCount>`. This can be any flow variable, such as an HTTP query param, header, or message body content. For more information, see Flow variables reference. You can also set custom variables using the JavaScript policy or the AssignMessage policy.	n/a	Optional

The effect of <UseEffectiveCount> depends on it's value:

true: An MP's spike rate limit is the <Rate> divided by the current number of MPs in the same pod. The aggregate limit is the value of <Rate>. When MPs are dynamically added (or removed), their individual spike rate limits will increase (or decrease), but the aggregate limit will stay the same.
false (this is the default value if omitted): Each MP's spike rate limit is simply the value of its <Rate>. The aggregate limit is the sum of the rates of all the MPs. When MPs are added (or removed), their individual spike rate limits will stay the same, but the aggregate limit will increase (or decrease).

The following table shows the effect of <UseEffectiveCount> on the effective rate limit of each MP:

	Value of `<UseEffectiveCount>`
	`false`	`false`	`false`	`true`	`true`	`true`
# of MPs	8	4	2	8	4	2
Value of `<Rate>`	10	10	10	40	40	40
Effective Rate per MP	10	10	10	5	10	20
Aggregate Limit	80	40	20	40*	40*	40*
* Same as `<Rate>`.

In this example, notice that when the number of MPs is decreased from 4 to 2, and <UseEffectiveCount> is false, the effective rate per MP stays the same (at 10). But when <UseEffectiveCount> is true, the effective rate per MP goes from 10 to 20 when the number of MPs is decreased from 4 to 2.

Tip: When you set <UseEffectiveCount> to true, set <Rate> to your target aggregate limit.

Flow variables

When a Spike Arrest policy executes, the following flow variable is populated:

Variable	Type	Permission	Description
`ratelimit.policy_name.failed`	Boolean	Read-Only	Indicates whether or not the policy failed (`true` or `false`).

For more information, see Flow variables reference.

Error reference

This section describes the fault codes and error messages that are returned and fault variables that are set by Edge when this policy triggers an error. This information is important to know if you are developing fault rules to handle faults. To learn more, see What you need to know about policy errors and Handling faults.

Runtime errors

These errors can occur when the policy executes.

Fault code	HTTP status	Cause
`policies.ratelimit.FailedToResolveSpikeArrestRate`	`500`	This error occurs if the reference to the variable containing the rate setting within the `<Rate>` element cannot be resolved to a value within the Spike Arrest policy. This element is mandatory and used to specify the spike arrest rate in the form of `intpm` or `intps`.
`policies.ratelimit.InvalidMessageWeight`	`500`	This error occurs if the value specified for the `<MessageWeight>` element through a flow variable is invalid (a non-integer value).
`policies.ratelimit.SpikeArrestViolation`	`429`	The rate limit was exceeded. Note: For Private Cloud, the default HTTP status code returned is `500`. To change the status code returned to `429`, set this property with the API call described in Example fault rule

Deployment errors

These errors can occur when you deploy a proxy containing this policy.

Error name	Cause	Fix
`InvalidAllowedRate`	If the spike arrest rate specified in the `<Rate>` element of the Spike Arrest Policy is not an integer or if the rate does not have `ps` or `pm` as a suffix, then the deployment of the API proxy fails.

Fault variables

These variables are set when a runtime error occurs. For more information, see What you need to know about policy errors.

Variables	Where	Example
`fault.name="fault_name"`	`fault_name` is the name of the fault, as listed in the Runtime errors table above. The fault name is the last part of the fault code.	`fault.name Matches "SpikeArrestViolation"`
`ratelimit.policy_name.failed`	`policy_name` is the user-specified name of the policy that threw the fault.	`ratelimit.SA-SpikeArrestPolicy.failed = true`

Example error response

Shown below is an example error response:

Note: For error handling, the best practice is to trap the errorcode part of the error response. Do not rely on the text in the faultstring, because it could change.

{  
   "fault":{  
      "detail":{  
         "errorcode":"policies.ratelimit.SpikeArrestViolation"
      },
      "faultstring":"Spike arrest violation. Allowed rate : 10ps"
   }
}

Example fault rule

Shown below is an example fault rule to handle a SpikeArrestViolation fault:

<FaultRules>
    <FaultRule name="Spike Arrest Errors">
        <Step>
            <Name>JavaScript-1</Name>
            <Condition>(fault.name Matches "SpikeArrestViolation") </Condition>
        </Step>
        <Condition>ratelimit.Spike-Arrest-1.failed=true</Condition>
    </FaultRule>
</FaultRules>

The current HTTP status code for exceeding a rate limit set by a Quota or Spike Arrest policy is 429 (Too Many Requests). To change the HTTP status code to 500 (Internal Server Error), set the features.isHTTPStatusTooManyRequestEnabled property to false using the Update organization properties API.

Private Cloud: For Private Cloud, the default HTTP status code returned is 500. To change the status code returned to 429, set this property with the following API call:

Note: When updating properties, you must pass all existing properties to the API, even if they are not being changed. If you omit properties in the payload, the properties are removed.

curl -u email:password -X POST -H "Content-type:application/xml" \
  http://edge-management-server:8080/v1/o/myorg \
  -d '<Organization type="trial" name="MyOrganization">
    <DisplayName>MyOrganization</DisplayName>
    <Environments/>
    <Properties>
        <Property name="features.isHTTPStatusTooManyRequestEnabled">true</Property>
    </Properties>
  </Organization>'

For example:

curl -u email:password -X POST -H "Content-type:application/xml" http://api.enterprise.apigee.com/v1/organizations/myorg -d \
"<Organization type="trial" name="MyOrganization">
    <Properties>
        <Property name="features.isHTTPStatusTooManyRequestEnabled">true</Property>
        . . .
    </Properties>
</Organization>"

Schemas

Each policy type is defined by an XML schema (.xsd). For reference, policy schemas are available on GitHub.

SpikeArrest policy

How Spike Arrest works

There's more: 1 request * number of Message Processors

The difference between spike arrest and quota

Videos

Why You Need It

How to Configure

Attach Policy

Per Second Rate Limiting

Per Minute Rate Limiting

Unique Rate Limiting

Compare Quota Policy

Flow Variables

<SpikeArrest> element

Syntax

Default Policy

Examples

Example 1

Example 2

Example 3

Example 4

Child element reference

<DisplayName>

Syntax

Example

<Identifier>

Syntax

Example 1

<MessageWeight>

Syntax

Example 1

<Rate>

Syntax

Example 1

Example 2

<UseEffectiveCount>

Syntax

Example 1

Flow variables

Error reference

Runtime errors

Deployment errors

Fault variables

Example error response

Example fault rule

Schemas

Related topics

`<SpikeArrest>` element

`<DisplayName>`

`<Identifier>`

`<MessageWeight>`

`<Rate>`

`<UseEffectiveCount>`