KeyValueMapOperations policy

You're viewing Apigee Edge documentation.
Go to the Apigee X documentation. info

Key Value Map Operations icon from the Edge UI

What

Provides policy-based access to a Key Value Map (KVM) store available in Apigee Edge. Key/value pairs can be stored, retrieved, and deleted from named existing maps by configuring KeyValueMapOperations policies that specify PUT, GET, or DELETE operations. (At least one of these operations must be performed by the policy.)

Videos

Watch the following videos to learn more about KVMs.

Video Description
Why Key Value Maps? Learn why you need KVMs and how they work.
Create KVM using UI and retrieve KVM at runtime Create a KVM, retrieve its value using a KVM policy, and inject the value into the API request using flow variables.
Create and update a KVM at API runtime Create a KVM at API runtime using a KVM policy.
Cache KVM to increase performance Improve performance of KVM Policy by caching the data.
Store encrypted KVM Store sensitive information in KVM in an encrypted format and retrieve the value at runtime using KVM policy and private variables.
Manage access using KVM scope Restrict KVM to organization, environment, API proxy, or API proxy revision using KVM policy scope attribute.
Delete KVM entries at API runtime Delete KVM entries at API runtime using the KVM policy DELETE operation.

Samples

PUT KVM with a literal

When the following policy runs, it creates an encrypted KVM named FooKVM, then creates a key named FooKey_1 with two values set with literal strings foo and bar (not set with values extracted from variables). When you GET the key in the next example, you specify an index number to retrieve the value you want.

<KeyValueMapOperations async="false" continueOnError="false" enabled="true" name="FooKVM" mapIdentifier="FooKVM">
  <DisplayName>FooKVM</DisplayName>
  <ExpiryTimeInSecs>86400</ExpiryTimeInSecs>
  <Scope>environment</Scope>
  <Put>
    <Key>
      <Parameter>FooKey_1</Parameter>
    </Key>
    <Value>foo</Value>
    <Value>bar</Value>
  </Put>
</KeyValueMapOperations>

Notice that the scope is "environment". That means you can see the KVM in the management UI under APIs > Environment Configuration > Key Value Maps. The KVMs shown on that page are all scoped to the selected environment.

GET KVM from a literal

This policy looks at the FooKVM map from the previous example, gets the second value (index="2") from the FooKey_1 key, and stores it in a variable called foo_variable.

<KeyValueMapOperations mapIdentifier="FooKVM" async="false" continueOnError="false" enabled="true" name="GetKVM">
  <DisplayName>GetKVM</DisplayName>
  <ExpiryTimeInSecs>86400</ExpiryTimeInSecs>
  <Scope>environment</Scope>
  <Get assignTo="foo_variable" index="2">
    <Key>
      <Parameter>FooKey_1</Parameter>
    </Key>
  </Get>
</KeyValueMapOperations>

PUT KVM with a variable

A simple example of a useful key value map is a URL shortening service. The key value map could be configured to store shortened URLs along with corresponding full URLs.

This policy sample creates a key value map. The policy PUTs a key with two associated values into a key/value map named "urlMapper".

<KeyValueMapOperations name="putUrl" mapIdentifier="urlMapper">
   <Scope>apiproxy</Scope>
   <Put override="true">
      <Key>
         <Parameter ref="urlencoding.requesturl.hashed"/>
      </Key>
      <Value ref="urlencoding.longurl.encoded"/>
      <Value ref="request.queryparam.url"/>
   </Put>
</KeyValueMapOperations>

The key in this example, urlencoding.requesturl.hashed, is an example of a custom variable. The hashed request URL would be generated by code (JavaScript or Java, for example) and then stored in this variable, where the KeyValueMapOperations policy can access it.

For each key, requesturl.hashed, two values are stored:

  • The contents of the custom variable named urlencoding.longurl.encoded
  • The contents of the predefined variable request.queryparam.url

For example, when the policy executes at runtime, the values of the variables may be as follows:

  • urlencoding.requesturl.hashed: ed24e12820f2f900ae383b7cc4f2b31c402db1be
  • urlencoding.longurl.encoded: http://tinyurl.com/38lwmlr
  • request.queryparam.url: http://apigee.com

The following key/value map and entry would be generated in Edge's key/value store and scoped to the API proxy to which the policy is attached:

{
    "entry" :[ 
        {
            "name" : "ed24e12820f2f900ae383b7cc4f2b31c402db1be",
            "value" : "http://tinyurl.com/38lwmlr,http://apigee.com"
        }
    ],
    "name" : "urlMapper"
}

The entry will persist until it is deleted. Key/value store entries are distributed across instances of Edge that are running the cloud.

GET KVM from a variable

A simple example of a useful key value map is a URL 'shortening' service. The key value map could be configured to store shortened URLs along with corresponding full URLs.

To retrieve the value of key/value map entry, such as the one covered on the KeyValueMapOperations PUT tab, configure a policy to GET the key value map:

<KeyValueMapOperations name="getUrl" mapIdentifier="urlMapper">
   <Scope>apiproxy</Scope>
   <Get assignTo="urlencoding.shorturl" index='1'>
      <Key>
         <Parameter ref="urlencoding.requesturl.hashed"/> 
      </Key>
   </Get>
</KeyValueMapOperations>

When this policy is executed, if the value of the urlencoding.requesturl.hashed variable is ed24e12820f2f900ae383b7cc4f2b31c402db1be, then the custom variable named urlencoding.shorturl will be set with the value http://tinyurl.com/38lwmlr.

Now that the data has been retrieved, other policies and code can access it by extracting the value from those variables.

GET encrypted value from KVM

If a key value map is encrypted, retrieve values by using the "private." prefix in the assignTo attribute value. In this example, the variable private.encryptedVar holds the decrypted value of the key value map's foo key. For information on creating encrypted key value maps, see the "create" topics of the Key/Value Maps management API.

<KeyValueMapOperations name="getEncrypted" mapIdentifier="encrypted_map">
   <Scope>apiproxy</Scope>
   <Get assignTo="private.encryptedVar" index='1'>
      <Key>
         <Parameter>foo</Parameter> 
      </Key>
   </Get>
</KeyValueMapOperations>

Now that the data has been retrieved, other policies and code can access it by extracting the value from that variable.

Element reference

The element reference describes the elements and attributes of the KeyValueMapOperations policy:

<KeyValueMapOperations async="false" continueOnError="false" 
    enabled="true" name="Key-Value-Map-Operations-1" 
    mapIdentifier="urlMapper" >
   <DisplayName>Key Value Map Operations 1</DisplayName>
   <Scope>environment</Scope>
   <ExpiryTimeInSecs>300</ExpiryTimeInSecs>
   <InitialEntries>
      <Entry>
         <Key>
            <Parameter>key_name_literal</Parameter>
         </Key>
         <Value>value_literal</Value>
      </Entry>
      <Entry>
         <Key>
            <Parameter>variable_name</Parameter>
         </Key>
         <Value>value_1_literal</Value>
         <Value>value_2_literal</Value>
      </Entry>
   </InitialEntries>
   <Put override="false">
      <Key>
         <Parameter>key_name_literal</Parameter>
      </Key>
      <Value ref="variable_name"/>
   </Put>
   <Get assignTo="myvar" index="1">
      <Key>
         <Parameter ref="variable_name"/>
      </Key>
   </Get>
   <Delete>
      <Key>
         <Parameter>key_name_literal</Parameter>
      </Key>
   </Delete>
</KeyValueMapOperations>

<KeyValueMapOperations> attributes

The following example shows the attributes on the <KeyValueMapOperations> tag:

<KeyValueMapOperations async="false" continueOnError="false" enabled="true" name="Key-Value-Map-Operations-1" mapIdentifier="map_name">

The following table describes the attributes specific to the <KeyValueMapOperations> tag:

Attribute Description Default Presence
mapIdentifier

Specifies an identifier to be used when accessing a map created by this policy or in the management UI.

The KVM name is case sensitive with Apigee Edge for Public Cloud. For example, foobar is different than FooBar.

If you exclude this attribute, a KVM named kvmap is used.

Within a scope of organization/environment/apiproxy, you can use the mapIdentifier attribute to specify your own map name.

 N/A Optional

The following table describes attributes that are common to all policy parent elements:

Attribute Description Default Presence
name

The internal name of the policy. The value of the name attribute can contain letters, numbers, spaces, hyphens, underscores, and periods. This value cannot exceed 255 characters.

Optionally, use the <DisplayName> element to label the policy in the management UI proxy editor with a different, natural-language name.

 N/A Required
continueOnError

Set to false to return an error when a policy fails. This is expected behavior for most policies.

Set to true to have flow execution continue even after a policy fails.

 false Optional
enabled

Set to true to enforce the policy.

Set to false to turn off the policy. The policy will not be enforced even if it remains attached to a flow.

 true Optional
async

This attribute is deprecated.

 false Deprecated

<DisplayName> element

Use in addition to the name attribute to label the policy in the management UI proxy editor with a different, natural-language name.

<DisplayName>Policy Display Name</DisplayName>
Default

N/A

If you omit this element, the value of the policy's name attribute is used.
Presence Optional
Type String

<Delete> element

Deletes the specified key/value pair. At least one of <Get>, <Put>, or <Delete> must be used.

Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element. For example:

<Delete>
   <Key>
      <Parameter>key_name_literal</Parameter>
   </Key>
</Delete>
Default N/A
Presence Required if <Get> or <Put> are not present.
Type N/A

<Entry> element

Seed values for key value maps, which are populated in the key value map when it is initialized.

For Edge for Public Cloud, the key size is limited to 2 KB. For example:

<InitialEntries>
   <Entry>
      <Key>
         <Parameter>key_name_literal</Parameter>
      </Key>
      <Value>v1</Value>
   </Entry>
   <Entry>
      <Key>
         <Parameter>key_name_variable</Parameter>
      </Key>
      <Value>v3</Value>
      <Value>v4</Value>
   </Entry>
</InitialEntries>
Default N/A
Presence Optional
Type N/A

<ExclusiveCache> element

Deprecated. Use the <Scope> element instead.

<ExpiryTimeInSecs> element

Specifies the duration in seconds after which Edge refreshes its cached value from the specified KVM.

A value of 0 or -1, or excluding this element, means the default value of 300 seconds is used. For example: 

<ExpiryTimeInSecs>600</ExpiryTimeInSecs>
Default 300 (5 minutes)
Presence Optional
Type Integer

A KVM is a long-term persistence mechanism that stores keys and values in a NoSQL database. Because of this, reading from a KVM at runtime can potentially slow proxy performance. To improve performance, Edge has a built-in mechanism for caching KVM keys/values in memory during runtime. This KVM Operations policy always reads from cache for GET operations.

The <ExpiryTimeInSecs> element lets you control how long the keys/values used in the policy are stored in cache before they're refreshed again from the KVM. However, there are some differences between how GET and PUT operations affect cache expiration.

GET - The very first time a KVM GET operation executes, the requested keys/values from the KVM (whose name is specified in the policy's root mapIdentifier attribute) are loaded into cache, where they remain for subsequent GET operations until one of the following occurs:

  • The number of seconds specified in <ExpiryTimeInSecs> expires.
    or
  • A PUT operation in a KVM policy overwrites the existing values (explained next).

PUT - A PUT operation writes keys/values to the specified KVM. If the PUT writes to a key that already exists in cache, that cache is immediately refreshed and now holds the new value for the number of seconds specified in the policy's <ExpiryTimeInSecs> element.

Example - Caching a KVM

  1. A GET operation retrieves the value of "rating," which adds the value "10" to cache. The <ExpiryTimeInSecs> on the policy is 60.
  2. 30 seconds later, the GET policy executes again and retrieves "10" from the cache.
  3. 5 seconds later, a PUT policy updates the value of "rating" to "8", and the <ExpiryTimeInSecs> on the PUT policy is 20. The cache is immediately refreshed with the new value, which is now set to remain in cache for 20 seconds. (If the PUT hadn't happened, the cache originally populated by the first GET would still exist for another 30 seconds, left over from the original 60 seconds.)
  4. 15 seconds later, another GET executes and retrieves a value of "8".

<Get> element

Retrieves the value for the key specified. At least one of <Get>, <Put>, or <Delete> must be used.

Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element.

You can include multiple Get blocks in the policy to retrieve multiple items from a KVM.

Default N/A
Presence Required if <Put> or <Delete> are not present.
Type N/A

Get a single item from a KVM

<Get assignTo="myvar" index="1">         
   <Key>             
      <Parameter>key_name_literal</Parameter>         
   </Key>     
</Get>

Get multiple items from a KVM

In the following example, assume a KVM with the following keys and values. In addition to storing a running list of the most popular movies of all time, the KVM stores the director name for all major films.

Key Value
top_movies Princess Bride,The Godfather,Citizen Kane
Citizen Kane Orson Welles
Princess Bride Rob Reiner
The Godfather Francis Ford Coppola

Here's a KVM policy configuration we can use to retrieve the current most popular movie and the name of its director:

<Get assignTo="top.movie.pick" index="1">
   <Key>
      <Parameter>top_movies</Parameter>
   </Key>
</Get>
<Get assignTo="movie.director">
   <Key>
      <Parameter ref="top.movie.pick"/>
   </Key>
</Get>

When the API proxy is called, Edge creates the following variables that you can use in the API proxy flow:

  • top.movie.pick=Princess Bride
  • movie.director=Rob Reiner

Attributes

The following table describes the attributes of the <Get> element:

Attribute Description Default Presence
assignTo

The variable to which the retrieved value should be assigned.

If the key value map is encrypted, begin the assignTo name with "private.". For example:

<Get assignTo="private.myvar">

The policy throws an error if you try to retrieve an encrypted key value map without using the prefix. The prefix, which is required for basic security purposes during debugging, hides the encrypted values from API proxy Trace and debug sessions.

For information on creating encrypted key value maps, see the "create" topics of the Key/Value Maps management API and Creating and editing environment key value maps.

 N/A Required
index

The index number (in a 1-based index) of the item to fetch from a multi-valued key. For example, specifying index=1 will return the first value and assign it to the assignTo variable. If no index value is specified, all the values of that entry are assigned to the variable as a java.util.List.

For an example, see the "Get Encrypted Value from KVM" tab in Samples.

 N/A Optional

<InitialEntries> element

Seed values for key value maps, which are populated in the key value map when it is initialized. Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element. for example:

<InitialEntries>
   <Entry>
      <Key>
         <Parameter>key_name_literal</Parameter>
      </Key>
      <Value>v1</Value>
   </Entry>
   <Entry>
      <Key>
         <Parameter>key_name_variable</Parameter>
      </Key>
      <Value>v3</Value>
      <Value>v4</Value>
   </Entry>
</InitialEntries>

When using this element, when you save the policy in the management UI on a deployed version of the proxy, or deploy the API proxy bundle containing the policy with this element, the key(s) are automatically created in the KVM (as unencrypted). If the values in the policy are different than the values in the KVM, the values in the KVM are overwritten when the proxy is deployed. Any new keys/values are added to the existing KVM alongside the existing keys/values.

Keys and values populated by this element must be literals. For example, <Parameter ref="request.queryparam.key"> isn't supported within this element.

The key size is limited to 2 KB for both Edge for the Public Cloud and Edge for the Private Could. The KVM value is limited to 2 KB.

To create an encrypted KVM, use the Key/Value Maps management API.

Default N/A
Presence Optional
Type N/A

<Key> element

Specifies the key in a key/value map entry. A key can be composite, which means that more than one parameter can be appended to create the key. For example, userID and role might be combined to create a key. For example:

<Key>
    <Parameter>key_name_literal</Parameter>
</Key>

Be sure to see the <Parameter> element for specifics about how to set the key name.

With Edge for Public Cloud, the key size is limited to 2 KB. See Differences between the Edge for Public Cloud API and Private Cloud API for more.

Default N/A
Presence Optional
Type N/A

<Parameter> element

Specifies the key in a key/value pair. This element specifies the name when creating, putting, retrieving, or deleting the key/value pair.

You can specify the name by using:

  • A literal string

    <Key>
  <Parameter>literal</Parameter>
</Key>

  • A variable to be retrieved at run time, using the ref attribute

    <Key>
  <Parameter ref="variable_name"/>
</Key>

  • A combination of literals and variable references

    <Key>
  <Parameter>targeturl</Parameter>
  <Parameter ref="apiproxy.name"/>
  <Parameter>weight</Parameter>
</Key>

When the Key element includes multiple Parameter elements, the effective key string is the concatenation of the values of each parameter, joined with a double underscore. For example, in the above example, if the apiproxy.name variable has the value "abc1", then the effective key will be targeturl__abc1__weight.

Whether you're getting, updating, or deleting a key/value entry, the key name must match the name of the key in the key value map. See Specifying and retrieving key names for guidelines.

Default N/A
Presence Required
Type String

Attributes

The following table describes the attributes of the <Parameter> element:

Attribute Description Default Presence
ref Specifies the name of a variable whose value contains the exact name of the key you want to create, get, or delete. N/A Required if no literal value is given between the opening and closing tags. Prohibited if a literal value is given.

<Put> element

Writes a key/value pair to a key value map, whether the key value map is encrypted or unencrypted. If the key value map specified in the mapIdentifier attribute on the parent element doesn't exist, the map is automatically created (as unencrypted). If the key value map already exists, the key/value are added to it.

To create an encrypted key value map, use the Key/Value Maps management API; or see Creating and editing environment key value maps to create encrypted environment-scoped KVMs in the UI.

<Put override="false">         
   <Key>             
      <Parameter ref="mykeyvar"/>         
   </Key>         
   <Value ref="myvalvar1"/>     
</Put>
Default N/A
Presence Required if <Get> or <Delete> are not present.
Type N/A

Attributes

The following table describes the attributes of the <Put> element:

Attribute Description Default Presence
override

If set to true, it overrides the value for a key.

 false Optional

<Scope> element

Defines the boundary of accessibility for key value maps. The default scope is environment, meaning that, by default, maps entries are shared by all API proxies running in an environment (for example, test or prod). If you set the scope to apiproxy, then entries in the key value map are accessible only by the API proxy that writes the values to the map.

Note that when accessing a map or map entry, you must specify the same scope value you used when the map was created. For example, if the map was created with a scope of apiproxy, you must use the apiproxy scope when retrieving its values, putting changes, or deleting entries.

<Scope>environment</Scope>
Default environment
Presence Optional
Type String
Valid values:
  • organization
  • environment
  • apiproxy
  • policy (API proxy revision)

<Value> element

Specifies the value of a key. You can specify the value as either a literal string or, using the ref attribute, as a variable to be retrieved at run time:

<!-- Specify a literal value -->
<Value>literal<Value>

or:

<!-- Specify the name of variable value to be populated at run time. -->
<Value ref="variable_name"/>

You can also include multiple <Value> elements to specify a multi-part value. Values are combined at run time.

In the following example, two keys are added to the KVM:

  • Key k1 with values v1,v2
  • Key k2 with values v3,v4
<InitialEntries>         
   <Entry>             
      <Key>
         <Parameter>k1</Parameter>
      </Key>
      <Value>v1</Value>
      <Value>v2</Value>     
   </Entry>
   <Entry>
      <Key>
         <Parameter>k2</Parameter>
      </Key>
      <Value>v3</Value>
      <Value>v4</Value>
   </Entry>
</InitialEntries>

In the following example, one key is created with two values. Let's assume the organization name is foo_org, the API proxy name is bar, and the environment is test:

  • Key foo_org with values bar,test
<Put>
    <Key>
        <Parameter ref="organization.name"/>
    </Key>
    <Value ref="apiproxy.name"/>
    <Value ref="environment.name"/>
</Put>
Default N/A
Presence Required
Type String

Attributes

The following table describes the attributes of the <Value> element:

Attribute Description Default Presence
ref Specifies the name of a variable whose value contains the key value(s) you want to set. N/A Required if no literal value is given between the opening and closing tags. Prohibited if a literal value is given.

Error reference

Errors returned from Edge policies follow a consistent format as described in the Error code reference.

This section describes the fault codes and error messages that are returned and fault variables that are set by Edge when this policy triggers an error. This information is important to know if you are developing fault rules to handle faults. To learn more, see What you need to know about policy errors and Handling faults.

Runtime errors

These errors can occur when the policy executes.

Fault code HTTP status Cause Fix
steps.keyvaluemapoperations.SetVariableFailed 500

This error occurs if you try to retrieve a value from an encrypted key value map and set the value to a variable whose name does not have the prefix private. The prefix, which is required for basic security purposes during debugging, hides the encrypted values from API proxy Trace and debug sessions.
steps.keyvaluemapoperations.UnsupportedOperationException 500

This error occurs if the mapIdentifier attribute is set to empty string in the Key Value Map Operations policy.

Deployment errors

These errors can occur when you deploy a proxy containing this policy.

Error name Cause Fix
InvalidIndex If the index attribute specified in the <Get> element of Key Value Map Operations policy is zero or a negative number, then the deployment of the API proxy fails. The index starts from 1, so an index of zero or negative integer is considered as invalid.
KeyIsMissing This error occurs if the <Key> element is completely missing or <Parameter> element is missing within <Key> element underneath the <Entry> of the <InitialEntries> element of the Key Value Map Operations policy.
ValueIsMissing This error occurs if the <Value> element is missing underneath the <Entry> element of the <InitialEntries> element of the Key Value Map Operations policy.

Schemas

Usage notes

For an overview of key value maps, see Working with key value maps.

A key value map store provides a lightweight persistence mechanism for data formatted as key/value pairs. You can access these at runtime through policies or code. A map contains any arbitrary data in the format key=value.

For example localhost=127.0.0.1, zip_code=94110, or first_name=felix. In the first example, localhost is a key, and 127.0.0.1 is a value. Each key/value pair is stored as an entry in a key value map. A key value map can store many entries.

Here's an example of using key value maps. Suppose you need to store a list of IP addresses associated with various backend environments. You could create a key value map called ipAddresses that contains a list of key/value pairs as entries. For example, this JSON can represent such a map:

{
  "entry" : [ {
    "name" : "Development",
    "value" : "65.87.18.18"
  }, {
    "name" : "Staging",
    "value" : "65.87.18.22"
  } ],
  "name" : "ipAddresses"
}

You could use this structure to create a store of IP addresses that could be used by policies at runtime to enforce IP allowlisting or denylisting, to dynamically select a backend target address, and so on. Typically, the KeyValueMapOperations policy is used to store or retrieve long-lived information that needs to be reused over multiple request/response transactions.

Key/value maps can be manipulated via the KeyValueMapOperations policy, or directly via the Apigee Edge management API. Refer to the management API reference for details on the Organization key/value maps API API. You can use the API to, for example, upload large data sets to the key/value store, or creating scripts to manage key/value map entries. You will need to create a key/value map with the API before accessing it with the KeyValueMapOperations policy.

Specifying and retrieving key names

With the <Parameter> and <Value> elements, you can specify either a literal value (where the value is between the opening and closing tags) or use the ref attribute to specify the name of a variable whose value should be used at runtime.

The Parameter element deserves special mention, because it determines the name of the key that gets created, as well as the key name you want to retrieve or delete. Following are two examples. The first specifies a key name literally, and the second specifies a key name using a variable. Let's assume the following are used to create keys in a KVM:

<Parameter>key_name_literal</Parameter>
<Parameter ref="key.name.variable"/>

In the first instance, the literal value of "key_name_literal" is stored in the KVM as the key name. In the second instance, whatever value is in the key.name.variable becomes the name of the key in the KVM. For example, if the key.name.variable contained the value foo, the key would be named "foo".

When you want to retrieve the key and a key value with a GET operation (or delete with a DELETE operation), the <Parameter> setting needs to match the key name in the KVM. For example, if the key name in the KVM is "foo", you can either specify the literal value with <Parameter>foo</Parameter> or specify a variable that contains the exact value "foo", like this: <Parameter ref="variable.containing.foo"/>.

Related topics