Send Docs Feedback

Key Value Map Operations policy

What

Provides policy-based access to a key/value map (KVM) store available in Apigee Edge. Key/value pairs can be stored, retrieved, and deleted from named existing maps by configuring KeyValueMapOperations policies that specify PUT, GET, or DELETE operations. (At least one of these operations must be performed by the policy.)

Where

This policy can be attached in the following locations. 

ProxyEndpoint TargetEndpoint
    PreFlow Flow PostFlow PreFlow Flow PostFlow    
Request    
    Response
    PostFlow Flow PreFlow PostFlow Flow PreFlow    

Samples

When the following policy runs, it creates a Key Value Map named FooKVM, then creates a key named FooKey_1 with two values set with literal strings (not set with values extracted from variables): foo,bar. (When you GET the key in the next example, you specify an index number to retrieve the value you want.)

<KeyValueMapOperations async="false" continueOnError="false" enabled="true" name="CreateFooKVM" mapIdentifier="FooKVM">
  <DisplayName>CreateFooKVM</DisplayName>
  <ExpiryTimeInSecs>86400</ExpiryTimeInSecs>
  <Scope>environment</Scope>
  <Put>
    <Key>
      <Parameter>FooKey_1</Parameter>
    </Key>
    <Value>foo</Value>
    <Value>bar</Value>
  </Put>
</KeyValueMapOperations>

Notice that the scope is "environment". That means you can see the KVM in the management UI under APIs > Environment Configuration > Key Value Maps. The KVMs shown on that page are all scoped to the selected environment.

This policy looks at the FooKVM map from the previous example, gets the second value (index="2") from the FooKey_1 key, and stores it in a variable called foo_variable.

<KeyValueMapOperations mapIdentifier="FooKVM" async="false" continueOnError="false" enabled="true" name="GetKVM">
  <DisplayName>GetKVM</DisplayName>
  <ExpiryTimeInSecs>86400</ExpiryTimeInSecs>
  <Scope>environment</Scope>
  <Get assignTo="foo_variable" index="2">
    <Key>
      <Parameter>FooKey_1</Parameter>
    </Key>
  </Get>
</KeyValueMapOperations>
<KeyValueMapOperations name="putUrl" mapIdentifier="urlMapper">
   <Scope>apiproxy</Scope>
   <Put override="true">
      <Key>
         <Parameter ref="urlencoding.requesturl.hashed"/>
      </Key>
      <Value ref="urlencoding.longurl.encoded"/>
      <Value ref="request.queryparam.url"/>
   </Put>
</KeyValueMapOperations>

A simple example of a useful KeyValueMap is a URL shortening service. The KeyValueMap could be configured to store shortened URLs along with corresponding full URLs.

This policy sample creates a KeyValueMap. The policy PUTs a key with two associated values into a key/value map named "urlMapper".

The key in this example, urlencoding.requesturl.hashed, is an example of a custom variable. The hashed request URL would be generated by code (JavaScript or Java, for example) and then stored in this variable, where the KeyValueMapOperations policy can access it.

For each key, requesturl.hashed, two values are stored:

  • The contents of the custom variable named urlencoding.longurl.encoded
  • The contents of the predefined variable request.queryparam.url

For example, when the policy executes at runtime, the values of the variables may be as follows:

  • urlencoding.requesturl.hashed: ed24e12820f2f900ae383b7cc4f2b31c402db1be
  • urlencoding.longurl.encoded: http://tinyurl.com/38lwmlr
  • request.queryparam.url: http://apigee.com

The following key/value map and entry would be generated in Edge's key/value store and scoped to the API proxy to which the policy is attached:

{
    "entry" :[ 
        {
            "name" : "ed24e12820f2f900ae383b7cc4f2b31c402db1be",
            "value" : "http://tinyurl.com/38lwmlr,http://apigee.com"
        }
    ],
    "name" : "urlMapper"
}

The entry will persist until it is deleted. Key/value store entries are distributed across instances of Edge that are running the cloud.

<KeyValueMapOperations name="getUrl" mapIdentifier="urlMapper">
   <Scope>apiproxy</Scope>
   <Get assignTo="urlencoding.shorturl" index='1'>
      <Key>
         <Parameter ref="urlencoding.requesturl.hashed"/> 
      </Key>
   </Get>
</KeyValueMapOperations>

A simple example of a useful KeyValueMap is a URL 'shortening' service. The KeyValueMap could be configured to store shortened URLs along with corresponding full URLs.

To retrieve the value of key/value map entry, such as the one covered on the KeyValueMapOperations PUT tab, configure a policy to GET the KeyValueMap, as shown above.

When this policy is executed, if the value of the urlencoding.requesturl.hashed variable is ed24e12820f2f900ae383b7cc4f2b31c402db1be, then the custom variable named urlencoding.shorturl will be set with the value http://tinyurl.com/38lwmlr.

Now that the data has been retrieved, other policies and code can access it by extracting the value from those variables.

<KeyValueMapOperations name="getEncrypted" mapIdentifier="encrypted_map">
   <Scope>apiproxy</Scope>
   <Get assignTo="private.encryptedVar" index='1'>
      <Key>
         <Parameter>foo</Parameter> 
      </Key>
   </Get>
</KeyValueMapOperations>

If a key value map is encrypted, retrieve values by using the "private." prefix in the assignTo attribute value. In the example above, the variable private.encryptedVar holds the decrypted value of the key value map's foo key. For information on creating encrypted key value maps, see the "create" topics of the Key/Value Maps management API.

Now that the data has been retrieved, other policies and code can access it by extracting the value from that variable.


Element reference

The element reference describes the elements and attributes of the KeyValueMapOperations policy.

<KeyValueMapOperations async="false" continueOnError="false" 
    enabled="true" name="Key-Value-Map-Operations-1" 
    mapIdentifier="urlMapper" >
   <DisplayName>Key Value Map Operations 1</DisplayName>
   <Scope>environment</Scope>
   <ExpiryTimeInSecs>300</ExpiryTimeInSecs>
   <InitialEntries>
      <Entry>
         <Key>
            <Parameter>key_name_literal</Parameter>
         </Key>
         <Value>value_literal</Value>
      </Entry>
      <Entry>
         <Key>
            <Parameter ref="variable_name"></Parameter>
         </Key>
         <Value>value_literal</Value>
         <Value ref="variable_name"/>
      </Entry>
   </InitialEntries>
   <Put override="false">
      <Key>
         <Parameter>key_name_literal</Parameter>
      </Key>
      <Value ref="variable_name"/>
   </Put>
   <Get assignTo="myvar" index="1">
      <Key>
         <Parameter ref="variable_name"></Parameter>
      </Key>
   </Get>
   <Delete>
      <Key>
         <Parameter>key_name_literal</Parameter>
      </Key>
   </Delete>
</KeyValueMapOperations>

<KeyValueMapOperations> attributes

<KeyValueMapOperations async="false" continueOnError="false" enabled="true" name="Key-Value-Map-Operations-1" mapIdentifier="map_name">

This attribute specifies the name of the KVM you want to interact with.

Attribute Description Default Presence
mapIdentifier

Specifies an identifier to be used when accessing, via APIs, a map created by this policy or in the management UI.

If you exclude this attribute, a KVM named kvmap is used.

Within a scope of organization/environment/apiproxy, you can use the mapIdentifier attribute to specify your own map name.

N/A Optional.

The following attributes are common to all policy parent elements.

Attribute Description Default Presence
name

The internal name of the policy. Characters you can use in the name are restricted to: A-Z0-9._\-$ %. However, the Edge management UI enforces additional restrictions, such as automatically removing characters that are not alphanumeric.

Optionally, use the <DisplayName> element to label the policy in the management UI proxy editor with a different, natural-language name.

N/A Required
continueOnError

Set to false to return an error when a policy fails. This is expected behavior for most policies.

Set to true to have flow execution continue even after a policy fails.

false Optional
enabled

Set to true to enforce the policy.

Set to false to "turn off" the policy. The policy will not be enforced even if it remains attached to a flow.

true Optional
async

This attribute is deprecated.

false Deprecated

<DisplayName> element

Use in addition to the name attribute to label the policy in the management UI proxy editor with a different, natural-language name.

<DisplayName>Policy Display Name</DisplayName>
Default:

N/A

If you omit this element, the value of the policy's name attribute is used.

Presence: Optional
Type: String

 

<Delete> element

Deletes the specified key/value pair. (At least one of <Get>, <Put>, or <Delete> must be used.)

Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element.

<Delete>         
   <Key>             
      <Parameter>key_name_literal</Parameter>         
   </Key>     
</Delete> 
Default: N/A
Presence: Required if <Get> or <Put> are not present.
Type: N/A

<Entry> element

Seed values for KeyValueMaps, which are populated in the KeyValueMap when it is initialized.

<InitialEntries>         
   <Entry>             
      <Key>
         <Parameter>key_name_literal</Parameter>
      </Key>
      <Value>v1</Value>
   </Entry>
   <Entry>
      <Key>
         <Parameter>key_name_variable</Parameter>
      </Key>
      <Value>v3</Value>
      <Value>v4</Value>
   </Entry>
</InitialEntries>
Default: N/A
Presence: Optional
Type: N/A

<ExclusiveCache> element

Deprecated. Use the <Scope> element instead.

<ExpiryTimeInSecs> element

Specifies the duration in seconds after which Edge refreshes its cached value from the specified KVM. 

A value of 0 or -1, or excluding this element, means the default value of 300 seconds is used. 

<ExpiryTimeInSecs>600</ExpiryTimeInSecs>
Default: 300 (5 minutes)
Presence: Optional
Type: Integer

A KVM is a long-term persistence mechanism that stores keys and values in a NoSQL database. Because of this, reading from a KVM at runtime can potentially slow proxy performance. To improve performance, Edge has a built-in mechanism for caching KVM keys/values in memory during runtime. This KVM Operations policy always reads from cache for GET operations.

The <ExpiryTimeInSecs> element lets you control how long the keys/values used in the policy are stored in cache before they're refreshed again from the KVM. However, there are some differences between how GET and PUT operations affect cache expiration.

GET - The very first time a KVM GET operation executes, the requested keys/values from the KVM (whose name is specified in the policy's root mapIdentifier attribute) are loaded into cache, where they remain for subsequent GET operations until one of the following occurs:

  • The number of seconds specified in <ExpiryTimeInSecs> expires.
    or
  • A PUT operation in a KVM policy overwrites the existing values (explained next).

PUT - A PUT operation writes keys/values to the specified KVM. If the PUT writes to a key that already exists in cache, that cache is immediately refreshed and now holds the new value for the number of seconds specified in the policy's <ExpiryTimeInSecs> element.

Example - A KVM named "MyKVM" has a key : value of "rating" : "10"

  1. A GET operation retrieves the value of "rating," which adds the value "10" to cache. The <ExpiryTimeInSecs> on the policy is 60.
  2. 30 seconds later, the GET policy executes again and retrieves "10" from the cache.
  3. 5 seconds later, a PUT policy updates the value of "rating" to "8", and the <ExpiryTimeInSecs> on the PUT policy is 20. The cache is immediately refreshed with the new value, which is now set to remain in cache for 20 seconds. (If the PUT hadn't happened, the cache originally populated by the first GET would still exist for another 30 seconds, left over from the original 60 seconds.)
  4. 15 seconds later, another GET executes and retrieves a value of "8".

Notes

  • For best performance, balance cache expiry time with the amount of API traffic you expect. With heavy traffic, for example, you may want to set longer expiry times to avoid frequent cache-population reads from the NoSQL database.
  • There is no way to manually reset the KVM cache. However, you can create a temporary policy that executes a one-time PUT operation with a short <ExpiryTimeInSecs>. This will reset the cache, which contains the value you PUT, for the number of seconds you specify.
  • Updating a KVM with the management UI or API does not reset the KVM cache.
  • Don't add the PUT and GET operations in the same policy, because the PUT automatically resets the cache expiration, which doesn't allow the GET operation to read from a cache with a desired expiry time.

<Get> element

Retrieves the value for the key specified. (At least one of <Get>, <Put>, or <Delete> must be used.)

Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element.

Don't add PUT and GET operations in the same policy, because the PUT automatically resets the cache expiration, which doesn't allow the GET operation to read from a cache with a desired expiry time. See the ExpiryTimeInSecs element for more information.

<Get assignTo="myvar" index="1">         
   <Key>             
      <Parameter>key_name_literal</Parameter>         
   </Key>     
</Get>
Default: N/A
Presence: Required if <Put> or <Delete> are not present.
Type: N/A

Attributes

Attribute Description Default Presence
assignTo

The variable to which the retrieved value should be assigned.

If the key value map is encrypted, begin the assignTo name with "private.". For example:

<Get assignTo="private.myvar">

The policy throws an error if you try to retrieve an encrypted key value map without using that prefix. The prefix, which is required for basic security purposes during debugging, hides the encrypted values from API proxy Trace and debug sessions.

For information on creating encrypted key value maps, see the "create" topics of the Key/Value Maps management API and Creating and editing environment key value maps.

N/A Required
index

The index number (in a 1-based index) of the item to fetch from a multi-valued key. For example, specifying index=1 will return the first value and assign it to the assignTo variable. If no index value is specified, all the values of that entry are assigned to the variable as a java.util.List.

For an example, see the KeyValueMapOperations GET tab in Samples.

N/A Optional

<InitialEntries> element

Seed values for KeyValueMaps, which are populated in the KeyValueMap when it is initialized. Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element.

When using this element, when you save the policy in the management UI on a deployed version of the proxy, or deploy the API proxy bundle containing the policy with this element, the key(s) are automatically created in the KVM (as unencrypted). If the values for existing keys change in the policy, the existing values are overwritten with the new values. Any new keys/values are added to the existing KVM alongside the existing keys/values.

To create an encrypted key value map, use the management API.

<InitialEntries>         
   <Entry>             
      <Key>
         <Parameter>key_name_literal</Parameter>
      </Key>
      <Value>v1</Value>
   </Entry>
   <Entry>
      <Key>
         <Parameter>key_name_variable</Parameter>
      </Key>
      <Value>v3</Value>
      <Value>v4</Value>
   </Entry>
</InitialEntries>
Default: N/A
Presence: Optional
Type: N/A

<Key> element

Specifies the key in a key/value map entry. A key can be composite, which means that more than one parameter can be appended to create the key. For example, userID and role might be combined to create a key.

Be sure to see <Parameter> for specifics about how to set the key name.

The name ratelimit is reserved. For more information, see this Apigee Community post.

<Key>
    <Parameter>key_name_literal</Parameter>
</Key>
Default: N/A
Presence: Optional
Type: N/A

<Parameter> element

Specifies the key name in a key/value pair. This element specifies the name when creating, putting, retrieving, or deleting the key/value pair.

A key can be composite, which means that more than one parameter can be appended to make the key. For example, userID and role might be combined constitute a key.

You can specify the name as either a literal string or, using the ref attribute, as a variable to be retrieved at run time.

<!-- Specify a literal value -->
<Parameter>literal</Parameter>

OR

<!-- Specify the name of variable value to be populated at run time. -->
<Parameter ref="variable_name"></Parameter>

OR

<!-- Specify the name of variable value to be populated at run time. -->
<Parameter ref="variable_name"/>

Whether you're getting, updating, or deleting a key/value entry, the key name must match the name of the key in the key value map, whether you specify the name literally or use a variable that contains the exact key name. See Specifying and retrieving key names for guidelines.

Default: N/A
Presence: Required
Type: String

Attributes

Attribute Description Default Presence
ref Specifies the name of a variable whose value contains the exact name of the key you want to create, get, or delete. N/A Required if no literal value is given between the opening and closing tags. Prohibited if a literal value is given.

<Put> element

Writes a key/value pair to a key value map, whether the key value map is encrypted or unencrypted. If the key value map specified in the mapIdentifier attribute on the parent element doesn't exist, the map is automatically created (as unencrypted). If the key value map already exists, the key/value are added to it. To create an encrypted key value map, use the management API; or use the management UI to create encrypted environment-scoped KVMs.

Don't add PUT and GET operations in the same policy, because the PUT automatically resets the cache expiration, which doesn't allow the GET operation to read from a cache with a desired expiry time. See the ExpiryTimeInSecs element for more information.

<Put override="false">         
   <Key>             
      <Parameter ref="mykeyvar"></Parameter>         
   </Key>         
   <Value ref="myvalvar1"/>     
</Put>
Default: N/A
Presence: Required if <Get> or <Delete> are not present.
Type: N/A

If the policy PUTs a key/value in a KVM using a null key, the following key is auto-created: __$$_EDGE_KVM_EMPTY_KEY_$$__

This situation would happen, for example, if you're populating a key using a variable reference and the referenced variable is null. As new PUTs with null keys are attempted, only that key is used, so existing values for that key are potentially overwritten each time the policy is executed.

Attributes

Attribute Description Default Presence
override

If set to true, it overrides the value for a key.

false Optional

<Scope> element

Defines the boundary of accessibility for KeyValueMaps. The default scope is environment, meaning that, by default, maps entries are shared by all API proxies running in an environment (for example, test or prod). If you set the scope to apiproxy, then entries in the KeyValueMap are accessible only by the API proxy that writes the values to the map.

Note that when accessing a map or map entry, you must specify the same scope value you used when the map was created. For example, if the map was created with a scope of apiproxy, you must use the apiproxy scope when retrieving its values, putting changes, or deleting entries.

<Scope>environment</Scope>
Default: environment
Presence: Optional
Type: String
Valid values:
  • organization
  • environment
  • apiproxy
  • policy

<Value> element

Specifies the value of a key. You can specify the value as either a literal string or, using the ref attribute, as a variable to be retrieved at run time.

<!-- Specify a literal value -->
<Value>literal<Value>

OR

<!-- Specify the name of variable value to be populated at run time. -->
<Value ref="variable_name"/>

You can also include multiple <Value> elements to specify a multi-part value. Values are combined at run time.

In the following example, two keys are added to the KVM:

  • Key k1 with values v1,v2
  • Key k2 with values v3,v4
<InitialEntries>         
   <Entry>             
      <Key>
         <Parameter>k1</Parameter>
      </Key>
      <Value>v1</Value>
      <Value>v2</Value>	
   </Entry>
   <Entry>
      <Key>
         <Parameter>k2</Parameter>
      </Key>
      <Value>v3</Value>
      <Value>v4</Value>
   </Entry>
</InitialEntries>

In the following example, one key is created with two values. Let's assume the organization name is foo_org, the API proxy name is bar, and the environment is test:

  • Key foo_org with values bar,test
<Put>
    <Key>
        <Parameter ref="organization.name"/>
    </Key>
    <Value ref="apiproxy.name"/>
    <Value ref="apiproxy.name"/>
</Put>
Default: N/A
Presence: Required
Type: String

Attributes

Attribute Description Default Presence
ref Specifies the name of a variable whose value contains the key value(s) you want to set. N/A Required if no literal value is given between the opening and closing tags. Prohibited if a literal value is given.

Error codes

Errors returned from Edge policies follow a consistent format as described in the Error code reference.

This policy uses the following error codes:

Error Code Message
SetVariableFailed

Failed to set variable {0} in KeyValueMapStepDefinition {1}

When getting values in encrypted key value maps, this error occurs if you fail to prefix the assignTo variable with "private." For example:

<Get assignTo="private.encryptedVar" index="1">
    <Key>
      <Parameter>foo</Parameter>
    </Key>
</Get>
RemoveVariableFailed Failed to remove variable {0} in KeyValueMapStepDefinition {1}
InvalidIndex Invalid index {0} in KeyValueMapStepDefinition {1}
KeyIsMissing Key element is missing in KeyValueMapStepDefinition {0}
ValueIsMissing Value element is missing in KeyValueMapStepDefinition {0}

 

Schemas

See our GitHub repository samples for the most recent schemas.

Usage notes

For an overview of key value maps, see Working with key value maps.

A key value map store provides a lightweight persistence mechanism for data formatted as key/value pairs. You can access these at runtime through policies or code. A map contains any arbitrary data in the format key=value.

A key value map has a 15 MB limit. Writing to the map may fail close to or beyond this limit. If the amount of data might grow beyond 15 MB, be sure to use the ExpiryTimeInSecs element to expire entries before exceeding the limit.

For example localhost=127.0.0.1, zip_code=94110, or first_name=felix. In the first example, localhost is a key, and 127.0.0.1 is a value. Each key/value pair is stored as an entry in a key value map. A key value map can store many entries.

For example, say you need to store a list of IP addresses associated with various backend environments. You could create a key value map called ipAddresses that contains a list of key/value pairs as entries. For example, this JSON can represent such a map:

{
  "entry" : [ {
    "name" : "Development",
    "value" : "65.87.18.18"
  }, {
    "name" : "Staging",
    "value" : "65.87.18.22"
  } ],
  "name" : "ipAddresses"
}

You could use this structure to create a store of IP addresses that could be used by policies at runtime to enforce IP whitelisting or blacklisting, to dynamically select a backend target address, and so on. Typically, the KeyValueMapOperations policy is used to store or retrieve long-lived information that needs to be reused over multiple request/response transactions.

Key/value maps can be manipulated via the KeyValueMapOperations policy, or directly via the Apigee Edge management API. Refer to the management API reference for details on the Organization key/value maps API API. You can use the API to, for example, upload large data sets to the key/value store, or creating scripts to manage key/value map entries. You will need to create a key/value map with the API before accessing it with the KeyValueMapOperations policy.

Specifying and retrieving key names

With the <Parameter> and <Value> elements, you can specify either a literal value (where the value is between the opening and closing tags) or use the ref attribute to specify the name of a variable whose value should be used at runtime.

The Parameter element deserves special mention, because it determines the name of the key that gets created, as well as the key name you want to retrieve or delete. Following are two examples. The first specifies a key name literally, and the second specifies a key name using a variable. Let's assume the following are used to create keys in a KVM:

<Parameter>key_name_literal</Parameter>
<Parameter ref="key.name.variable"></Parameter>

In the first instance, the literal value of "key_name_literal" is stored in the KVM as the key name. In the second instance, whatever value is in the key.name.variable becomes the name of the key in the KVM. For example, if the key.name.variable contained the value foo, the key would be named "foo".

When you want to retrieve the key and a key value with a GET operation (or delete with a DELETE operation), the <Parameter> setting needs to match the key name in the KVM. For example, if the key name in the KVM is "foo", you can either specify the literal value with <Parameter>foo</Parameter> or specify a variable that contains the exact value "foo", like this: <Parameter ref="variable.containing.foo"/>.

The name ratelimit is reserved. For more information, see this Apigee Community post.

Related topics

Organization key/value maps API

 

Help or comments?