Error Processing Updates

You're viewing Apigee Edge documentation.
Go to the Apigee X documentation. info

Symptom

Deployment of API proxy revisions via the Edge UI or Edge management API calls fails with the error "Unexpected error occurred while processing the updates".

Error Messages

Error in deployment for environment sit.

The revision is deployed and traffic can flow, but flow may be impared. com.apigee.kernel.exceptions.spi.UncheckedException{ code = messaging.runtime.UpdateFailed, message = Unexpected error occurred while processing the updates, associated contexts = []}

Possible Causes

There could be many different causes for this error. Here are a few sample causes:

Cause Details For
Out Of Memory Message Processor ran out of Java heap space resulting in API proxy deployment failure. Private Cloud users
Error in API Proxy Bundle API Proxy bundle has errors that can lead to API proxy deployment failure. Private and Public Cloud users

Common steps to try first

  1. Check if there are any deployment errors or exception stack trace in the Message Processor log /opt/apigee/var/log/apigee/edge-message-processor/logs/system.log
  2. You usually see a stack trace that provides information on the cause for the deployment error. Please read the stack trace thoroughly to understand the cause.

Out Of Memory

Diagnosis

Note: Only Edge Private Cloud users can perform the following steps. If you are on Edge Public Cloud, contact Apigee Support.

  1. You may see exception similar to the one in the Message Processor log /opt/apigee/var/log/apigee/edge-message-processor/logs/system.log as shown in the figure below: 
    Apigee-Main-4 ERROR BOOTSTRAP - RuntimeConfigurationServiceImpl.dispatchToListeners() : RuntimeConfigurationServiceImpl.dispatchToListeners : Error occurred while dispatching the request DeployEvent{organization=‘myorg', application='person-credentials-api', applicationRevision='275', deploymentSpec=basepath=/;env=dev;, deploymentID=null} to com.apigee.application.bootstrap.listeners.MessageProcessorBootstrapListener@5db88cb8
com.apigee.kernel.exceptions.spi.UncheckedException: Unexpected error occurred while processing the updates
        at com.apigee.entities.AbstractConfigurator.throwUncheckedException(AbstractConfigurator.java:280) ~[config-entities-1.0.0.jar:na]
        at com.apigee.messaging.configuration.MessageProcessorServiceImpl.configure(MessageProcessorServiceImpl.java:665) ~[message-processor-1.0.0.jar:na]
        at com.apigee.application.bootstrap.listeners.MessageProcessorBootstrapListener.configureMessageProcessorService(MessageProcessorBootstrapListener.java:54) ~[application-bootstrap-1.0.0.jar:na]
        at com.apigee.application.bootstrap.listeners.MessageProcessorBootstrapListener.deploy(MessageProcessorBootstrapListener.java:29) ~[application-bootstrap-1.0.0.jar:na]
        …<snipped>
        at com.apigee.application.bootstrap.proto.RuntimeConfig_ConfigRPCService_BlockingSkeleton$1.run(RuntimeConfig_ConfigRPCService_BlockingSkeleton.java:38) [application-bootstrap-1.0.0.jar:na]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_75]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.OutOfMemoryError: Java heap space
  2. The message Caused by: java.lang.OutofMemoryError: Java Heap space indicates that the Message Processor has run out of Java Heap space.

Resolution

Note: Only Edge Private Cloud users can perform the following steps. If you are on Edge Public Cloud, contact Apigee Support.

Increase the maximum Java Heap space on the Message Processors.

The maximum Java Heap space is controlled by the JVM property -Xmx. Here are the steps to increase the Java Heap space on the Message Processors:

  1. Determine the amount of memory by which the Java heap space can be increased:
    1. Check the current value set for max heap space, max_mem, in the file /opt/apigee/edge-message-processor/bin/setenv.sh
    2. Get MemTotal (Total amount of usable RAM), MemFree (amount of physical RAM left unused on the system) using /proc/meminfo command on the system.
      1. Ensure the above information includes the memory consumed by any other processes such as Edge Router etc that exist on the same system.
      2. Based on the above information, determine by how much the Java heap space can be increased for Message Processor.
      3. For example, the current max heap space on the Message Processor is 1024MB, MemTotal is 8GB (8192MB), MemFree is 5GB (5120MB), then you can increase the max Java heap space to 3GB (3072MB).
      4. If assistance is needed in determining how much to increase the Java heap space, contact Apigee Support.
  2. Create the following file, if it does not exist already: 
    /opt/apigee/customer/application/message-processor.properties
  3. Add the following line in the file, with the increased heap setting (for example, 3072M): 
    bin_setenv_max_mem=3072m
  4. Save the file.
  5. Restart the Message Processor: 
    /opt/apigee/apigee-service/bin/apigee-service edge-message-processor restart
  6. If you have more than one Message Processor, repeat the steps 3 through 6 on all the Message Processors.

If the problem persists, contact Apigee Support.

Error in API proxy bundle

If there are any errors in any of the policies used in the API proxy, then the deployment will fail with the error "Unexpected error occurred while processing the updates".

For example, see this community post.

Diagnosis

Note: Only Edge Private Cloud users can perform the following steps. If you are on Edge Public Cloud, contact Apigee Support.

  1. Check the Message Processor logs for exceptions or errors related to the API proxy deployment. The error should give you information on what should be changed in the API proxy bundle to address the problem.
  2. If no error messages show up on the Message Processor logs, check the diffs in the revision history of the API proxy. If older revisions deploy without a problem then review what code changes were made.

Resolution

  1. Make the necessary changes in the API proxy bundle to address the issue.
  2. Revert any code changes to the API proxy that was problematic for the deployment.

If the problem persists, contact Apigee Support for further assistance.