Configuring Message Processors to allow duplicate headers

You're viewing Apigee Edge documentation.
Go to the Apigee X documentation. info

As per the HTTP Specification RFC 7230, section 3.2.2: Field Order, Apigee Edge expects that the HTTP request from the client or HTTP response from the backend server do not contain the same header being passed more than once with the same or different values, unless the specific header has an exception and is allowed to have duplicates.

By default, Apigee Edge allows duplicates and multiple values to be passed to most of the HTTP headers. However, it does not allow certain headers which are listed in Headers that are not allowed to have duplicates and multiple values. Therefore:

You will get 400 Bad Request with error code protocol.http.DuplicateHeader if the client sends an HTTP request with particular header more than once or with multiple values for the HTTP headers which are not allowed to have duplicates/multiple values in Apigee Edge.
Similarly, you will get 502 Bad Gateway with error code protocol.http.DuplicateHeader if the backend server sends an HTTP response with particular header more than once or with multiple values for the HTTP headers which are not allowed to have duplicates or multiple values in Apigee Edge

The recommended solution to address these errors is to fix the client application and backend server to not send duplicate headers and adhere to the specification RFC 7230, section 3.2.2: Field Order as explained in the following troubleshooting playbooks:

However, in some cases you may want to add an exception to include duplicates and multiple values for some HTTP headers. In such situations, you can allow duplicate headers and multiple values for a specific HTTP header by setting a property HTTPHeader.HEADER_NAME at the Message Processor level.

This document provides information about this property, explains how to enable this property to avoid the above-mentioned errors and shares best practices around the same.

HTTP header properties to allow duplicates and multiple values

Apigee Edge provides the following two properties to control the behaviour of allowing duplicates and multiple values for HTTP headers. Note that these can be configured only on the Message Processors using the token syntax explained in How to configure Edge.

Property name Description Values allowed

Property name	Description	Values allowed
`HTTPHeader.ANY`	This property indicates whether duplicates or multiple values are allowed for all HTTP headers including the custom headers sent as part of HTTP request made by the client or HTTP response sent by the backend server to Apigee Edge. Default value: `multiValued, allowDuplicates,` Note: It is recommended NOT to modify this property.	`blank`: Duplicates and multiple values for HTTP headers are not allowed. `multiValued`: Split the multi-valued header into multiple headers. Multiple values are allowed for HTTP headers, but duplicates are not allowed. The value `multiValued` is enabled, which implies that `test-header=a,b` would be converted to `test-header=a` and `test-header=b.` `allowDuplicates`: Allows multiple (duplicate) HTTP headers with the same name. `multiValued, allowDuplicates`: Both multiple values and duplicates are allowed for HTTP headers.
`HTTPHeader.HEADER_NAME`	This property is used to override the behavior of a specific header from what is specified by `HTTPHeader.ANY` Note: You can modify this property as explained in this document.	Same as above.

HTTPHeader.ANY

This property indicates whether duplicates or multiple values are allowed for all HTTP headers including the custom headers sent as part of HTTP request made by the client or HTTP response sent by the backend server to Apigee Edge.

Default value:

multiValued, allowDuplicates,

blank: Duplicates and multiple values for HTTP headers are not allowed.
multiValued: Split the multi-valued header into multiple headers. Multiple values are allowed for HTTP headers, but duplicates are not allowed. The value multiValued is enabled, which implies that test-header=a,b would be converted to test-header=a and test-header=b.
allowDuplicates: Allows multiple (duplicate) HTTP headers with the same name.
multiValued, allowDuplicates: Both multiple values and duplicates are allowed for HTTP headers.

HTTPHeader.HEADER_NAME

This property is used to override the behavior of a specific header from what is specified by HTTPHeader.ANY

Same as above.

Headers that are not allowed to have duplicates and multiple values

As explained earlier, Apigee Edge allows duplicates and multiple values for most of the HTTP headers by default. This is because the property HTTPHeader.ANY is configured with the value multiValued, allowDuplicates.

Configuration overwritten

For some specific headers, the default configuration is overwritten using one of the following methods:

HTTPHeader.HEADER_NAME=multiValued, allowDuplicates

This configuration does not change the default behaviour. That is, the specific header is allowed to have duplicates and multiple values
.
HTTPHeader.HEADER_NAME=

This configuration changes the default behaviour. That is, the specific header is not allowed to have duplicates and multiple values.

Determining headers that are not allowed to have duplicates and multiple values

This section describes how to identify the following:

The specific headers that are not allowed to have duplicates and multiple values on your Apigee Edge Private Cloud setup, and
The specific headers with pre-existing config

On the Message Processor machine, search for the property HTTPHeader. in the /opt/apigee/edge-message-processor/conf directory as shown below:

grep -ri "HTTPHeader." /opt/apigee/edge-message-processor/conf

Sample output:

# grep -ri "HTTPHeader" /opt/apigee/edge-message-processor/conf
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.ANY=allowDuplicates, multiValued
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.Connection=allowDuplicates, multiValued
… <snipped>
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.Host=
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.Expires=
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.Date=allowDuplicates
…
<snipped>

As explained in the Configuration overwritten section, note the following information in the sample output above:
1. The HTTP header Connection is overwritten, but allowed to have duplicates and multiple values
2. The HTTP headers Host and Expires are overwritten and not allowed to have duplicates and multiple values
3. The HTTP header Date is overwritten and allowed to have duplicates but not allowed to have multiple values
4. All the headers which appear here (Connection, Host, Expires, and Date in the above sample) are referred to as headers with pre-existing config in this document.
Note: Using the above steps, you can determine which headers are not allowed to have duplicates and multiple values in your Private Cloud setup.

Behaviour of Apigee Edge

The following table describes the behaviour of Apigee Edge when the headers are sent as duplicates and with multiple values depending on how the HTTPHeader properties are configured on the Message Processors with an example HTTPHeader of test-header.

Request Outgoing HEADERS based on value of conf/http.properties+HTTPHeader.test-header=

<Blank> allowDuplicates multiValued allowDuplicates, multiValued (DEFAULT)

Request	Outgoing HEADERS based on value of conf/http.properties+HTTPHeader.test-header=
`test‑header=a,b`	`test‑header=a,b`	`test‑header=a,b`	`protocol.http. DuplicateHeader` Internally we split `test-header=a,b` into: `test-header=a`, and `test-header=b`, and then the `DuplicateHeader` error is thrown.	`test‑header=a,b` Internally we split `test-header=a,b` into: `test-header=a`, and `test-header=b`, but then the original form is sent to target.
`test‑header=a` `test‑header=b`	`protocol.http. DuplicateHeader`	`test‑header=a` `test‑header=b`	`protocol.http. DuplicateHeader`	`test‑header=a` `test‑header=b`

test‑header=a,b

protocol.http. DuplicateHeader

Internally we split test-header=a,b into:

test-header=a, and
test-header=b,

and then the DuplicateHeader error is thrown.

test‑header=a,b

Internally we split test-header=a,b into:

test-header=a, and
test-header=b,

but then the original form is sent to target.

test‑header=a
test‑header=b protocol.http. DuplicateHeader test‑header=a
test‑header=b protocol.http. DuplicateHeader test‑header=a
test‑header=b

Note: The property should be enabled only after careful consideration. To troubleshoot the protocol.http.DuplicateHeader error, refer to the following playbooks:

This guide should be followed only as an alternative approach. Wherever possible, it's advisable to follow the recommended approach in the above playbooks.

Before you begin

Before you use the steps in this document, be sure you understand configuring properties for Edge on Private Cloud, described in How to configure Edge.

Configuring allowDuplicates and multiple values for headers

As explained in HTTP header properties to allow duplicates and multiple values, the value of the property HTTPHeader.ANY = allowDuplicates, multiValued implies that all headers are allowed to have duplicates and multiple values in Apigee Edge. However, there are certain headers whose values are overwritten explicitly to not allow duplicate headers or multiple values for thi using the property HTTPHeader.HEADER_NAME.

This section explains how to configure the property HTTPHeader.HEADER_NAME to allow duplicates and multiple values for any such HTTP headers on the Message Processors, using the corresponding token as per the syntax described in How to configure Edge.

Note: If you modify the default value of the property HTTPHeaderHEADER_NAME on the Message Processors, then it is important to note that it affects all the API proxies associated with the organizations and environments served by the specific Message Processors.

In this section, we will use Expires (and myheader) as an example header for which we want to allow duplicates and multiple values as explained below:

Determine the current value of the property HTTPHeaderHEADER_NAME to make sure it is not already enabled to allow duplicates and multiple values using the following command:
```
grep -ri "HTTPHeader.HEADER_NAME" /opt/apigee/edge-message-processor/conf
```
For example, if you are trying to set the property for the Expires header, then check the current value of the property HTTPHeader.Expires token on the Message Processor:
```
grep -ri "HTTPHeader.Expires" /opt/apigee/edge-message-processor/conf
```
The output of the above command results in one of the following:
1. The property is set to blank, then it implies that the value is overwritten (and this is a header with pre-existing config) to NOT allow duplicate headers and multiple values. That is, you are not allowed to send the Expires header more than once as part of the HTTP request or HTTP response to Apigee.
2. There are no hits for the specific property, then that means that value is not overwritten (and this is NOT a header with pre-existing config). This means that the specific header can be sent more than once (duplicates are allowed) as part of the HTTP request or HTTP response to Apigee Edge.
3. The property is set with the value allowDuplicates, multiValued then that means that value is overwritten explicitly (and this is a header with pre-existing config). This means that the specific header can be sent more than once (duplicates are allowed) as part of the HTTP request or HTTP response to Apigee.
Sample output of the search command:
```
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.Expires=
```
The above sample output shows that the property HTTPHeader.Expires is set to blank. This means that the property is overwritten to not allow duplicate or multiple values for the header Expires.
If you notice that the property corresponding to the specific header is explicitly overwritten to not allow duplicate or multiple values as in the example output above, only then perform the following steps. If it is not explicitly overwritten, then skip the rest of the steps in this section.

Edit. If it does not exist, you can create it:

/opt/apigee/customer/application/message-processor.properties

For example, to open the file using vi, enter the following:

vi /opt/apigee/customer/application/message-processor.properties

Add a line in the following format:

conf_http_HTTPHeader.Expires=allowDuplicates, multiValued

Save your changes.
Ensure that the properties file is owned by the apigee user. If it is not, execute the following command:
```
chown apigee:apigee /opt/apigee/customer/application/message-processor.properties
```
Restart the Message Processor:
```
/opt/apigee/apigee-service/bin/apigee-service edge-message-processor restart
```
To restart without traffic impact, see Rolling restart of Message Processors without traffic impact.
If you have more than one Message Processor, repeat the above steps on all the Message Processors.

Verifying the header is configured to have duplicates and multiple values

This section explains how to verify that the property HTTPHeader.HEADER_NAME for a specific header has been updated successfully to allow duplicates on the Message Processors.

We will use Expires as an example header and check if the corresponding property HTTPHeader.Expires has been updated.

Even though you use the token conf_http_HTTPHeader.Expires to update the value on the Message Processor, you need to verify if the actual property HTTPHeader.Expires has been set with the new value.

On the Message Processor machine, search for the property HTTPHeader.HEADER_NAME in the /opt/apigee/edge-message-processor/conf directory and check to see if it has been set with the new value as shown below:
```
grep -ri "HTTPHeader.HEADER_NAME" /opt/apigee/edge-message-processor/conf
```
For example, if you want to check that the property HTTPHeader.Expires is set with the new value, then run the following command:
```
grep -ri "HTTPHeader.Expires" /opt/apigee/edge-message-processor/conf
```
If the new value is successfully set for HTTPHeader.HEADER_NAME on the Message Processor, then the above command shows the new value in the http.properties file.
The sample result from the above command after you have configured allowDuplicates and multiValued is as follows:
```
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.Expires=allowDuplicates, multiValued
```
In the example output above, note that the property HTTPHeader.Expires has been set with the new value allowDuplicates, multiValued in http.properties. This indicates that the behaviour to allow duplicates and multiple values in HTTPHeader is successfully configured on the Message Processor.
If you still see the old value for the property HTTPHeader.HEADER_NAME, then verify that you have followed all the steps outlined in Configuring allowDuplicates and multiple values for headers correctly. If you have missed any step, repeat all the steps again correctly.
Note: Once the above changes are complete and reflected in each Message Processor, you should not see the error protocol.http.DuplicateHeader even if duplicate headers are passed to Apigee Edge as described in Behaviour of Apigee Edge.

Make sure your proxies are working as expected, especially if there is a functional logic to get and set the headers in the proxy.
If you are still not able to modify the property, then contact Apigee Edge Support

Disabling allowDuplicates for headers

This section explains how to configure the property HTTPHeader.{Headername} to not allow duplicates and multiple values for a specific HTTP header on the Message Processors, using the corresponding token according to the syntax described in How to configure Edge.

Note: If you modify the default value of the property HTTPHeader{Headername} on the Message Processors, then it is important to note that it affects all the API proxies associated with the organizations and environments served by the specific Message Processors.

In this section, we will use Expires (and myheader) as an example header for which we do not want to allow duplicates as explained below:

Determine the current value of the property HTTPHeaderHEADER_NAME to make sure it is not already disabled to allow duplicates and multiple values using the following command:
```
grep -ri "HTTPHeader.HEADER_NAME" /opt/apigee/edge-message-processor/conf
```
For example if you are trying to set the property for Expires header then check the current value of the property HTTPHeader.Expires token on the Message Processor:
```
grep -ri "HTTPHeader.Expires" /opt/apigee/edge-message-processor/conf
```
The output of the above command results in one of the following:
1. The property is set to blank, then it implies that the value is overwritten to NOT to allow duplicate headers and multiple values. That is you are not allowed to send the Expires header more than once as part of the HTTP request or HTTP response to Apigee.
2. There are no hits for the specific property, then that means that value is not overwritten and this is a NOT header with pre-existing config. This means that the specific header can be sent more than once (duplicates are allowed) as part of the HTTP request or HTTP response to Apigee Edge.
3. The property is set with the value allowDuplicates, multiValued then that means that value is overwritten explicitly and this is an existent config. However, this means that the specific header can be sent more than once (duplicates are allowed) as part of the HTTP request or HTTP response to Apigee.
Sample output #1

Sample Output #1 of the search command:
```
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.Expires=allowDuplicates, multiValued
```
The sample output shows that the property HTTPHeader.Expires is set to allowDuplicates, multiValued. This means that the property is overwritten to allow duplicate or multiple values for the header Expires.
Sample output #2

Sample Command and Output #2 of the search command
```
grep -ri "HTTPHeader.myheader" /opt/apigee/edge-message-processor/conf
```
The sample output shows no output, which implies that the property HTTPHeader.myheader is set to allowDuplicates, multiValued by default. This also implies that the property is not overwritten for the header myheader,
If you notice one of the following, perform the rest of the steps in this section:
1. The property corresponding to the specific header is overwritten to allow duplicates and multiple values as in the Sample output #1 above (header with pre-existing config)
2. There are no hits for the property corresponding to the specific header as in the Sample output #2 above (not a header with pre-existing config)
Otherwise, skip the rest of the steps in this section.

Edit the following file. If it does not exist, you can create it.

/opt/apigee/customer/application/message-processor.properties

For example, to open the file using vi, enter the following:

vi /opt/apigee/customer/application/message-processor.properties

Add a line in the following format to the properties file:
Pre-existing config

Scenario #1: Header with pre-existing config:
```
conf_http_HTTPHeader.Expires=
```
No pre-existing config

Scenario #2: Not a header with pre-existing config:
```
conf/http.properties+HTTPHeader.myheader=
```
Save your changes.
Ensure that the properties file is owned by the apigee user. If it is not, execute the following:
```
chown apigee:apigee /opt/apigee/customer/application/message-processor.properties
```
Restart the Message Processor:
```
/opt/apigee/apigee-service/bin/apigee-service edge-message-processor restart
```
To restart without traffic impact, see Rolling restart of Message Processors without traffic impact.
If you have more than one Message Processor, repeat the above steps on all the Message Processors.

Verifying the header is configured to not allow duplicates and multiple values

This section explains how to verify that the property HTTPHeader.HEADER_NAME for a specific header has been updated successfully to not allow duplicates on the Message Processors.

We will use Expires (and myheader) as an example header and check if the corresponding property HTTPHeader.Expires (and HTTPHeader.myheader) has been updated.

Note: Even though you use the token conf_http_HTTPHeader.Expires to update the value on the Message Processor, you need to verify if the actual property HTTPHeader.Expires has been set with the new value.

On the Message Processor machine, search for the property HTTPHeader.HEADER_NAME in the /opt/apigee/edge-message- processor/conf directory and check to see if it has been set with the new value as shown below:
```
grep -ri "HTTPHeader.HEADER_NAME" /opt/apigee/edge-message-processor/conf
```
For example, if you want to check the property HTTPHeader.Expires is set with the new value, then you can run the following command:
Pre-existing config
```
grep -ri "HTTPHeader.Expires" /opt/apigee/edge-message-processor/conf
```
No pre-existing config
```
grep -ri "HTTPHeader.myheader" /opt/apigee/edge-message-processor/conf
```
If the new HTTP header value is successfully set for HTTPHeader.HEADER_NAME I on the Message Processor, then the above command shows the new value in the http.properties file.
The sample result from the above command after you have disabled allowDuplicates is as follows:
Pre-existing config

Scenario #1: Expires Header (header with pre-existing config)
```
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.Expires=
```
No pre-existing config

Scenario #2: myheader Header (not a header with pre-existing config)
```
/opt/apigee/edge-message-processor/conf/http.properties:HTTPHeader.myheader=
```
In the example output above, note that the property HTTPHeader.Expires ( and HTTPHeader.myheader ) has been set with the new value {blank} in http.properties. This indicates that the behaviour to allow duplicates and multiple values for the specific HTTP header Expires (and myheader) is successfully disabled on the Message Processor.
If you still see the old value for the property HTTPHeader.Expires (or HTTPHeader.myheader), then verify that you have followed all the steps outlined in Configuring allowDuplicates and multiple values for headers correctly. If you have missed any step, repeat all the steps again correctly.

Note: Once the above changes are done and reflected in each Message Processor, you should see the error protocol.http.DuplicateHeader if duplicate headers are passed to Apigee Edge as described in Behaviour of Apigee Edge

Make sure your proxies are working as expected, especially if there is a functional logic to get and set the headers in the proxy.
If you are still not able to modify the property, then contact Apigee Edge Support.

Configuring Message Processors to allow duplicate headers

HTTP header properties to allow duplicates and multiple values

Headers that are not allowed to have duplicates and multiple values

Configuration overwritten

Determining headers that are not allowed to have duplicates and multiple values

Behaviour of Apigee Edge

Before you begin

Configuring allowDuplicates and multiple values for headers

Verifying the header is configured to have duplicates and multiple values

Disabling allowDuplicates for headers

Sample output #1

Sample output #2

Pre-existing config

No pre-existing config

Verifying the header is configured to not allow duplicates and multiple values

Pre-existing config

No pre-existing config

Pre-existing config

No pre-existing config