ConcurrentRateLimit policy

Public and Private Cloud Deprecation Notice:

Support for the Concurrent Rate Limit feature

The Concurrent Rate Limit policy was designed to be used with slow backends, however, the architecture used by it affected the performance of the APIs.

Since the Concurrent Rate Limit's announcement, Apigee has added newer capabilities such as chaining proxies together and shared flows. If you are experiencing slower performance with Concurrent Rate Limit policy, you can try using Spike Arrest instead.

You can read more details in our usage notes page.

Using such features along with policy, will allow you to protect slow/sluggish backends without impacting the performance of the APIs.

Migrate proxies using the Concurrent Rate Limit policy to the Quota policy and/or the Spike Arrest policies.

What

Throttles inbound connections from your API proxies running on Apigee Edge to your backend services.

Need help deciding which rate limiting policy to use? See Comparing Quota, Spike Arrest, and Concurrent Rate Limit Policies.

Video

To learn how to restrict concurrent connections to the backend using the Concurrent Rate Limit policy, watch this video.

Sample

<ConcurrentRatelimit name="ConnectionThrottler" >
   <AllowConnections count="200" ttl="5" />
   <Distributed>true</Distributed>
   <StrictOnTtl>false</StrictOnTtl>
   <TargetIdentifier name="MyTargetEndpoint"  ref="header/qparam/flow variables" /> 
</ConcurrentRatelimit>

Element reference

The element reference describes the elements and attributes of the ConcurrentRatelimit policy.

<ConcurrentRatelimit async="false" continueOnError="false" enabled="true" name="Concurrent-Rate-Limit-1">
   <DisplayName>Concurrent Rate Limit 1</DisplayName>
   <AllowConnections count="200" ttl="5"/>
   <Distributed>true</Distributed>
   <StrictOnTtl>false</StrictOnTtl>
   <TargetIdentifier name="default"></TargetIdentifier> 
</ConcurrentRatelimit>

<ConcurrentRatelimit> attributes

<ConcurrentRatelimit async="false" continueOnError="false" enabled="true" name="Concurrent-Rate-Limit-1">

The following table describes attributes that are common to all policy parent elements:

Attribute	Description	Default	Presence
`name`	The internal name of the policy. The value of the `name` attribute can contain letters, numbers, spaces, hyphens, underscores, and periods. This value cannot exceed 255 characters. Optionally, use the `<DisplayName>` element to label the policy in the management UI proxy editor with a different, natural-language name.	N/A	Required
`continueOnError`	Set to `false` to return an error when a policy fails. This is expected behavior for most policies. Set to `true` to have flow execution continue even after a policy fails.	false	Optional
`enabled`	Set to `true` to enforce the policy. Set to `false` to "turn off" the policy. The policy will not be enforced even if it remains attached to a flow.	true	Optional
`async`	This attribute is deprecated.	false	Deprecated

<DisplayName> element

Use in addition to the name attribute to label the policy in the management UI proxy editor with a different, natural-language name.

<DisplayName>Policy Display Name</DisplayName>

Default

Default	N/A If you omit this element, the value of the policy's `name` attribute is used.
Presence	Optional
Type	String

N/A

If you omit this element, the value of the policy's name attribute is used.

Presence Optional

Type String

<AllowConnections> element

Provides the number of concurrent connections between Apigee Edge and a backend service that are allowed at any given time.

<AllowConnections count="200" ttl="5"/>

Default:	N/A
Presence:	Required
Type:	N/A

Attributes

Attribute	Description	Default	Presence
count	Specifies the number of concurrent connections between Apigee Edge and a backend service that are allowed at any given time.	N/A	Required
ttl	Include to automatically decrement the counter after the number of seconds specified. This can help to clean up any connections that were not decremented properly in the response path.	N/A	Required

<Distributed> element

Specify whether counter values are shared across instances of Apigee Edge's server infrastructure.

<Distributed>true</Distributed>

Default:	false
Presence:	Optional
Type:	Boolean

<StrictOnTtl> element

Set to true to honor the <AllowConnections> ttl attribute setting regardless of backend server throughput. Consider setting this property to true for high throughput or low latency backend services.

<StrictOnTtl>false</StrictOnTtl>

Default:	false
Presence:	Optional
Type:	Boolean

<TargetIdentifier> element

Provides the name of the TargetEndpoint to which the throttling should be applied.

<TargetIdentifier name="default"></TargetIdentifier>

Default:	N/A
Presence:	Optional
Type:	N/A

Attributes

Attribute	Description	Default	Presence
name	Specifies the name of the TargetEndpoint to which the throttling should be applied.	N/A	Optional
ref		N/A	Optional

Flow variables

A set of predefined flow variables are populated each time the policy executes:

concurrentratelimit.{policy_name}.allowed.count
concurrentratelimit.{policy_name}.used.count
concurrentratelimit.{policy_name}.available.count
concurrentratelimit.{policy_name}.identifier

Error codes

This section describes the fault codes and error messages that are returned and fault variables that are set by Edge when this policy triggers an error. This information is important to know if you are developing fault rules to handle errors. To learn more, see What you need to know about policy errors and Handling faults.

Runtime errors

These errors can occur when the policy executes.

Fault code HTTP status Occurs when

Fault code	HTTP status	Occurs when
`policies.concurrentratelimit.ConcurrentRatelimtViolation`	503	ConcurrentRatelimit connection exceeded. Connection limit : {0} Note: The Fault code shown on the left is correct, although it contains a misspelling ("limt"). Be sure to use the code exactly as shown here when creating fault rules to trap this error.

policies.concurrentratelimit.ConcurrentRatelimtViolation

503

ConcurrentRatelimit connection exceeded. Connection limit : {0}

Note: The Fault code shown on the left is correct, although it contains a misspelling ("limt"). Be sure to use the code exactly as shown here when creating fault rules to trap this error.

Deployment errors

Error name	Occurs when
`InvalidCountValue`	ConcurrentRatelimit invalid count value specified.
`ConcurrentRatelimitStepAttachment\ NotAllowedAtProxyEndpoint`	Concurrent Ratelimit policy {0} attachment is not allowed at proxy request/response/fault paths. This policy must be placed on the Target Endpoint.
`ConcurrentRatelimitStepAttachment\ MissingAtTargetEndpoint`	Concurrent Ratelimit policy {0} attachment is missing at target request/response/fault paths. This policy must be placed in the Target Request Preflow, Target Response Postflow, and DefaultFaultRule.
`InvalidTTLForMessageTimeOut`	ConcurrentRatelimit invalid ttl value specified for message timeout. It must be a positive integer.

Fault variables

These variables are set when this policy triggers an error. For more information, see What you need to know about policy errors.

Variables	Where	Example
`fault.name="fault_name"`	`fault_name` is the name of the fault, as listed in the Runtime errors table above. The fault name is the last part of the fault code.	`fault.name Matches "ConcurrentRatelimtViolation"` Note: The Error code shown in the example is correct, although it contains a misspelling ("limt"). Please be sure to use the code exactly as shown here when creating fault rules to trap this error.
`concurrentratelimit.policy_name.failed`	`policy_name` is the user-specified name of the policy that threw the fault.	`concurrentratelimit.CRL-RateLimitPolicy.failed = true`

Example error response

If the rate limit is exceeded, the policy returns only an HTTP status 503 to the client.

Example fault rule

<faultrule name="VariableOfNonMsgType"></faultrule><FaultRules>
    <FaultRule name="Quota Errors">
        <Step>
            <Name>JavaScript-1</Name>
            <Condition>(fault.name Matches "ConcurrentRatelimtViolation") </Condition>
        </Step>
        <Condition>concurrentratelimit.CRL-RateLimitPolicy.failed=true</Condition>
    </FaultRule>
</FaultRules>

Schemas

Usage notes

In distributed environments, app traffic may be managed by many replicated API proxies. While each API proxy might be handling just a few connections, collectively, a set of API proxies, running in redundant message processors across multiple datacenters, all of which point to the same backend service, might swamp the capacity of that backend service. Use this policy to limit the aggregate traffic to a manageable number of connections.

When the number of requests being processed by Apigee Edge exceeds the connection limit configured in the policy, Apigee Edge will return the HTTP response code 503: Service Unavailable.

This policy is known to slow performance in API proxies that handle a high number of transactions per second (TPS). For high-TPS API proxies, if Concurrent Rate Limit slows performance to unacceptable levels, try using Spike Arrest instead.

Policy attachment details

The ConcurrentRatelimit policy must be attached as a Step to three Flows on a TargetEndpoint: request, response, and DefaultFaultRule. (A validation error will be thrown at deployment time if the policy is attached to any other Flows, including any ProxyEndpoint Flows.) See also this Apigee Community post.

For example to attach a ConcurrentRatelimit policy called ConnectionThrottler to a TargetEndpoint called MyTargetEndpoint, create the following TargetEndpoint configuration. Note that the policy must also be included in the DefaultFaultRule to ensure that the rate limit counters are maintained correctly in the event of an error.

<TargetEndpoint name="MyTargetEndpoint">
  <DefaultFaultRule name="DefaultFaultRule">
    <Step><Name>ConnectionThrottler</Name></Step>
    <AlwaysEnforce>true</AlwaysEnforce>
  </DefaultFaultRule>
  <PreFlow name="PreFlow">
    <Request>
      <Step><Name>ConnectionThrottler</Name></Step>
    </Request>
  </PreFlow>
  <PostFlow name="PostFlow">
    <Response>
      <Step><Name>ConnectionThrottler</Name></Step>
    </Response>
  </PostFlow>
  <HTTPTargetConnection>
    <URL>http://api.mybackend.service.com</URL>
  </HTTPTargetConnection>
</TargetEndpoint>

ConcurrentRateLimit policy

What

Video

Sample

Element reference

<ConcurrentRatelimit> attributes

<DisplayName> element

<AllowConnections> element

Attributes

<Distributed> element

<StrictOnTtl> element

<TargetIdentifier> element

Attributes

Flow variables

Error codes

Runtime errors

Deployment errors

Fault variables

Example error response

Example fault rule

Schemas

Usage notes

Policy attachment details

Related topics