500 Internal Server Error - BadFormData

You're viewing Apigee Edge documentation.
Go to the Apigee X documentation. info

Symptom

The client application gets an HTTP status code of 500 Internal Server Error with the error code protocol.http.BadFormData as a response for API calls.

Error message

Client application gets the following response code:

HTTP/1.1 500 Internal Server Error

In addition, you may observe the following error message:

{
   "fault":{
      "faultstring":"Bad Form Data",
      "detail":{
         "errorcode":"protocol.http.BadFormData"
      }
   }
}

Form data

Before we get into the details of troubleshooting this issue, let’s understand what is form data.

Form data is the information provided by the user typically through an HTML form having elements such as a text input box, button, or a check box. The form data is generally sent as a series of key-value pairs as part of HTTP requests or responses.

Form data transmission

1. Content-Type: application/x-www-form-urlencoded
   • If the size of the form data is small, then the data is sent as key-value pairs with:
     • The characters in both keys encoded as per the rules explained in Forms - Section 17.13.4.1
     • The header Content-Type: application/x-www-form-urlencoded

     Sample request with form data:

     curl https://HOSTALIAS/somepath -H "Content-Type: application/x-www-form-urlencoded" -d "username=abc@google.com&pasword=secret123"
     

   • Any non-alphanumeric characters in both keys and values are percent encoded, that is, they are represented as a character triplet %HH, consisting of a percent sign followed by two hexadecimal digits representing the ASCII code of the specific character.
   • Thus, even though the percent sign (%) is allowed in the form data, it is interpreted as the start of a special escape sequence. Therefore, if the form data needs to contain the percent sign (%) in the key or value, then it should be transmitted as %25, which represents the ASCII code for the percent sign (%) character.
2. Content-Type: multipart/form-data

   If you want to transmit large quantities of binary data or text containing non-ASCII characters, then you can send the data with the Content-Type: multipart/form-data as explained in Forms - Section 17.13.4.2

Possible causes

This error occurs if and only if all of the following conditions are met:

1. The HTTP request sent by the client to Apigee Edge contains:
   1. Content-Type: application/x-www-form-urlencoded, and
   2. Form data with the percent sign (%), or the percent sign (%) followed by invalid hexadecimal characters that are not allowed as per Forms - Section 17.13.4.1.

2. The API proxy in Apigee Edge reads the specific form parameters containing any characters that are not allowed to be used in the request flow using the ExtractVariables or the AssignMessage policy.

   For example, if the form data contains the percent sign (%) as-is (without encoding) or the percent sign (%) followed by any invalid hexadecimal characters in the key and/or value, then you get this error.

   Here are the possible causes for this error:

   Cause	Description	Troubleshooting instructions applicable for
   Form Parameters in request has characters that are not allowed	The form parameters passed as part of the HTTP request by the client contains any characters that are not allowed to be used.	Edge Public and Private Cloud users

Common diagnosis steps

Use one of the following tools/techniques to diagnose this error:

Cause: Form parameters in request has characters that are not allowed

Diagnosis

1. Determine the Fault Code, Fault Source, and Fault Policy for 500 Internal Server Error using API Monitoring, Trace tool or NGINX access logs as explained in Common diagnosis steps.
2. If the Fault Code is protocol.http.BadFormData, Fault Source has the value proxy or policy, and Fault Policy is non- empty, then this indicates that policy specified in Fault Policy failed while reading or extracting the form data (form parameters).
3. Examine the policy indicated in the Fault Policy and determine the following information:
   1. Source: Determine whether the policy is reading or extracting the data from request or response.
   2. Form parameters: Determine the specific form parameters that are being read in the policy.

      Sample #1

      Sample #1: ExtractVariables policy extracting form parameters:

            <ExtractVariables name="EV-ExtractFormParms">
               <DisplayName>EV-ExtractFormParams</DisplayName>
               <Source>request</Source>
               <FormParam name="username">
                  <Pattern ignoreCase="false">{username}</Pattern>
               </FormParam>
               <FormParam name="password">
                 <Pattern ignoreCase="false">{password}</Pattern>
               </FormParam>
               <VariablePrefix>forminfo</VariablePrefix>
             <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
            </ExtractVariables>
            

      In the above ExtractVariables policy:

      • Source: request

        This is indicated by the <Source> element

      • Form parameters: username and password

        This is indicated by the <Pattern> element within the <FormParam> element

      This indicates that the form parameters username and/or password passed as part of the HTTP request by the client to Apigee Edge contains characters that are not allowed to be used.

      Sample #2

      Sample #2: AssignMessage policy copying form parameters:

            <AssignMessage continueOnError="false" enabled="true" name="AM-CopyFormParams">
              <Copy source="request">
                <FormParams>
                  <FormParam name="username"/>
                  <FormParam name="password"/>
                </FormParams>
              </Copy>
              <AssignTo createNew="true" transport="http" type="request"/>
            </AssignMessage>
            

      In the above ExtractVariables policy:

      • Source: request

        This is indicated by the source attribute in the <Copy> element

      • Form parameters: username and password

        This is indicated by the name attribute in the <FormParam> element

      This indicates that the form parameters username or password or both passed as part of the HTTP request by the client to Apigee Edge contains any characters that are not allowed to be used.

4. Check to see if there are any characters that are not allowed to be used characters in the form parameters identified in step 3 using one of the following methods:

   Trace tool

   To validate using the Trace tool:

   1. If you have captured the trace for the failing request as explained in Common diagnosis steps, then select one of the failing requests.
   2. If you have determined that the form parameters containing any characters that are not allowed to be used are part of the HTTP request in step 3 above, then
      1. Navigate to the phase Request Received from Client.

      2. Scroll down to the Phase Details section and review the Request Content.

         ( view larger image)

         
      3. In the above example, note that the form parameter password contains the percent sign (%).
      4. Since the percent sign (%) is also used for percent encoding the special characters, it cannot be used as-is in the form data.
      5. Therefore, Apigee Edge responds with 500 Internal Server Error with the error code protocol.http.BadFormData.

   Actual request

   To validate using the actual request:

   1. If you don’t have access to the actual request made to the target server, then go to Resolution.
   2. If you have access to the actual request made to Apigee Edge, then perform the following steps:
      1. Review the form data contents and see if it contains any characters that are not allowed to be used such as the percent sign (%) or the percent sign (%) followed by invalid hexadecimal characters.

         Sample #1

         Sample Request #1: Form data as part of the request

         curl -X GET "https://HOSTALIAS/myproxy -H "Content-Type: application/x-www-form-urlencoded" -d "client_id=123456abc123&client_secret=c23578%ZY"
         

         In this example, note that the element client_secret contains the percent sign (%) followed by invalid hexadecimal characters ZY.

         Sample #2

         Sample Request #2: Form data passed in a file:

         curl -X GET "https://HOSTALIAS/myproxy -H "Content-Type: application/x-www-form-urlencoded" -d @form_data.xml
         

         Contents of form_data.xml:

         xml=<user><username>abc1234@google.com</username><password>qwerty12345!@#$%</password></user>

         In this example, note that the element password contains the percent sign (%), which should not be passed as-is in the form data.

   3. In the above two examples, the form data sent as part of HTTP request to Apigee Edge contains characters that are not allowed to be used.
   4. Therefore, Apigee Edge responds with 500 Internal Server Error with the error code protocol.http.BadFormData.

Resolution

1. Ensure that any special characters in both the keys and values of form data or parameters sent as part of HTTP request by the client are always encoded as explained in Form Data - application/x-www-form-urlencoded.
2. For the examples discussed above, you can fix the issues as follows:

   Sample #1

   Sample #1: Form data passed as part of the request:

   Use valid hexadecimal characters that match the ASCII code for a specific character. For example, if you want to send the dollar sign ($), use %24 as shown below:

   curl -X GET "https://HOSTALIAS/myproxy -H "Content-Type: application/x-www-form-urlencoded" -d "client_id=123456abc123&client_secret=c23578%24"
   

   Sample #2

   Sample Request #2: Form data passed in a file:

   curl -X GET "https://HOSTALIAS/myproxy -H "Content-Type: application/x-www-form-urlencoded" -d @form_data.xml
   

   Contents of form_data.xml:

   Use the percent-encoding for the percent (%) sign, that is modify the file to have %25 as shown below:

   xml=<user><username>abc1234@google.com</username><password>qwerty12345!!@#$%25</password></user>

Specification

Apigee Edge expects that the form data to be sent as per the following specifications:

If you still need any assistance from Apigee Support, go to Must gather diagnostic information.

Must gather diagnostic information

If the problem persists even after following the above instructions, gather the following diagnostic information, and then contact Apigee Edge Support:

If you are a Public Cloud user, provide the following information:

• Organization name
• Environment name
• API proxy name
• Complete curl command used to reproduce the 500 Internal Server Error with the error code protocol.http.BadFormData
• Trace file for the API requests

If you are a Private Cloud user, provide the following information:

• Complete error message observed for the failing requests
• Environment name
• API proxy bundle
• Trace file for the API requests

• NGINX access logs

  /opt/apigee/var/log/edge-router/nginx/ORG~ENV.PORT#_access_log

  Where: ORG, ENV and PORT# are replaced with actual values.

• Message Processor system logs

  /opt/apigee/var/log/edge-message-processor/logs/system.log

References

• Form Content types
• RFC3986, section 2.1: Percent encoding
• Percent Encoding
• Hexadecimal characters