ConcurrentRateLimit policy

What

Throttles inbound connections from your API proxies running on Apigee Edge to your backend services.

Need help deciding which rate limiting policy to use? See Comparing Quota, Spike Arrest, and Concurrent Rate Limit Policies.

Video

To learn how to restrict concurrent connections to the backend using the Concurrent Rate Limit policy, watch this video.

Sample

<ConcurrentRatelimit name="ConnectionThrottler" >
   <AllowConnections count="200" ttl="5" />
   <Distributed>true</Distributed>
   <StrictOnTtl>false</StrictOnTtl>
   <TargetIdentifier name="MyTargetEndpoint"  ref="header/qparam/flow variables" /> 
</ConcurrentRatelimit>

Element reference

The element reference describes the elements and attributes of the ConcurrentRatelimit policy.

<ConcurrentRatelimit async="false" continueOnError="false" enabled="true" name="Concurrent-Rate-Limit-1">
   <DisplayName>Concurrent Rate Limit 1</DisplayName>
   <AllowConnections count="200" ttl="5"/>
   <Distributed>true</Distributed>
   <StrictOnTtl>false</StrictOnTtl>
   <TargetIdentifier name="default"></TargetIdentifier> 
</ConcurrentRatelimit>

<ConcurrentRatelimit> attributes

<ConcurrentRatelimit async="false" continueOnError="false" enabled="true" name="Concurrent-Rate-Limit-1">

The following table describes attributes that are common to all policy parent elements:

Attribute Description Default Presence
name

The internal name of the policy. The value of the name attribute can contain letters, numbers, spaces, hyphens, underscores, and periods. This value cannot exceed 255 characters.

Optionally, use the <DisplayName> element to label the policy in the management UI proxy editor with a different, natural-language name.

 N/A Required
continueOnError

Set to false to return an error when a policy fails. This is expected behavior for most policies.

Set to true to have flow execution continue even after a policy fails.

 false Optional
enabled

Set to true to enforce the policy.

Set to false to "turn off" the policy. The policy will not be enforced even if it remains attached to a flow.

 true Optional
async

This attribute is deprecated.

 false Deprecated

<DisplayName> element

Use in addition to the name attribute to label the policy in the management UI proxy editor with a different, natural-language name.

<DisplayName>Policy Display Name</DisplayName>
Default

N/A

If you omit this element, the value of the policy's name attribute is used.
Presence Optional
Type String

<AllowConnections> element

Provides the number of concurrent connections between Apigee Edge and a backend service that are allowed at any given time.

<AllowConnections count="200" ttl="5"/>
Default: N/A
Presence: Required
Type: N/A

Attributes

Attribute Description Default Presence
count Specifies the number of concurrent connections between Apigee Edge and a backend service that are allowed at any given time. N/A Required
ttl

Include to automatically decrement the counter after the number of seconds specified. This can help to clean up any connections that were not decremented properly in the response path.

N/A Required

<Distributed> element

Specify whether counter values are shared across instances of Apigee Edge's server infrastructure.

<Distributed>true</Distributed>
Default: false
Presence: Optional
Type: Boolean

<StrictOnTtl> element

Set to true to honor the <AllowConnections> ttl attribute setting regardless of backend server throughput. Consider setting this property to true for high throughput or low latency backend services.

<StrictOnTtl>false</StrictOnTtl>
Default: false
Presence: Optional
Type: Boolean

<TargetIdentifier> element

Provides the name of the TargetEndpoint to which the throttling should be applied.

<TargetIdentifier name="default"></TargetIdentifier>
Default: N/A
Presence: Optional
Type: N/A

Attributes

Attribute Description Default Presence
name Specifies the name of the TargetEndpoint to which the throttling should be applied. N/A Optional
ref N/A Optional

Flow variables

A set of predefined flow variables are populated each time the policy executes:

  • concurrentratelimit.{policy_name}.allowed.count
  • concurrentratelimit.{policy_name}.used.count
  • concurrentratelimit.{policy_name}.available.count
  • concurrentratelimit.{policy_name}.identifier

Error codes

This section describes the fault codes and error messages that are returned and fault variables that are set by Edge when this policy triggers an error. This information is important to know if you are developing fault rules to handle errors. To learn more, see What you need to know about policy errors and Handling faults.

Runtime errors

These errors can occur when the policy executes.

Fault code HTTP status Occurs when
policies.concurrentratelimit.ConcurrentRatelimtViolation 503

ConcurrentRatelimit connection exceeded. Connection limit : {0}

Note: The Fault code shown on the left is correct, although it contains a misspelling ("limt"). Be sure to use the code exactly as shown here when creating fault rules to trap this error.

Deployment errors

Error name Occurs when
InvalidCountValue ConcurrentRatelimit invalid count value specified.
ConcurrentRatelimitStepAttachment\
NotAllowedAtProxyEndpoint		 Concurrent Ratelimit policy {0} attachment is not allowed at proxy request/response/fault paths. This policy must be placed on the Target Endpoint.
ConcurrentRatelimitStepAttachment\
MissingAtTargetEndpoint		 Concurrent Ratelimit policy {0} attachment is missing at target request/response/fault paths. This policy must be placed in the Target Request Preflow, Target Response Postflow, and DefaultFaultRule.
InvalidTTLForMessageTimeOut ConcurrentRatelimit invalid ttl value specified for message timeout. It must be a positive integer.

Fault variables

These variables are set when this policy triggers an error. For more information, see What you need to know about policy errors.

Variables Where Example
fault.name="fault_name" fault_name is the name of the fault, as listed in the Runtime errors table above. The fault name is the last part of the fault code. fault.name Matches "ConcurrentRatelimtViolation"

Note: The Error code shown in the example is correct, although it contains a misspelling ("limt"). Please be sure to use the code exactly as shown here when creating fault rules to trap this error.

concurrentratelimit.policy_name.failed policy_name is the user-specified name of the policy that threw the fault. concurrentratelimit.CRL-RateLimitPolicy.failed = true

Example error response

If the rate limit is exceeded, the policy returns only an HTTP status 503 to the client.

Example fault rule

<faultrule name="VariableOfNonMsgType"></faultrule><FaultRules>
    <FaultRule name="Quota Errors">
        <Step>
            <Name>JavaScript-1</Name>
            <Condition>(fault.name Matches "ConcurrentRatelimtViolation") </Condition>
        </Step>
        <Condition>concurrentratelimit.CRL-RateLimitPolicy.failed=true</Condition>
    </FaultRule>
</FaultRules>

Schemas

Usage notes

In distributed environments, app traffic may be managed by many replicated API proxies. While each API proxy might be handling just a few connections, collectively, a set of API proxies, running in redundant message processors across multiple datacenters, all of which point to the same backend service, might swamp the capacity of that backend service. Use this policy to limit the aggregate traffic to a manageable number of connections.

When the number of requests being processed by Apigee Edge exceeds the connection limit configured in the policy, Apigee Edge will return the HTTP response code 503: Service Unavailable.

This policy is known to slow performance in API proxies that handle a high number of transactions per second (TPS). For high-TPS API proxies, if Concurrent Rate Limit slows performance to unacceptable levels, try using Spike Arrest instead.

Policy attachment details

The ConcurrentRatelimit policy must be attached as a Step to three Flows on a TargetEndpoint: request, response, and DefaultFaultRule. (A validation error will be thrown at deployment time if the policy is attached to any other Flows, including any ProxyEndpoint Flows.) See also this Apigee Community post.

For example to attach a ConcurrentRatelimit policy called ConnectionThrottler to a TargetEndpoint called MyTargetEndpoint, create the following TargetEndpoint configuration. Note that the policy must also be included in the DefaultFaultRule to ensure that the rate limit counters are maintained correctly in the event of an error.

<TargetEndpoint name="MyTargetEndpoint">
  <DefaultFaultRule name="DefaultFaultRule">
    <Step><Name>ConnectionThrottler</Name></Step>
    <AlwaysEnforce>true</AlwaysEnforce>
  </DefaultFaultRule>
  <PreFlow name="PreFlow">
    <Request>
      <Step><Name>ConnectionThrottler</Name></Step>
    </Request>
  </PreFlow>
  <PostFlow name="PostFlow">
    <Response>
      <Step><Name>ConnectionThrottler</Name></Step>
    </Response>
  </PostFlow>
  <HTTPTargetConnection>
    <URL>http://api.mybackend.service.com</URL>
  </HTTPTargetConnection>
</TargetEndpoint>

Related topics

Quota policy

Spike Arrest policy

Comparing Quota, Spike Arrest, and Concurrent Rate Limit Policies