This section is not about troubleshooting SOA composite instance failures, but rather about troubleshooting composite instances in general. Just because a composite is not reported as failed does not mean it does not warrant investigation. For example, it is entirely possible that when dealing with asynchronous services, there might be synchronization issues between concurrent threads that are trying to access a schema and the thread locking mechanism being unable to avoid deadlocks. This would leave the transaction in a pending state and not faulted. This behavior may require further investigation.
Chapter 6, Monitoring Oracle SOA Suite 12c, described in detail how to administer and manage faults. Faulted instances, in a way, are easier to deal with than non-faulted instances, as at least you have an error to begin with. Refer to this chapter to understand how to recover faults manually, automatically, and in bulk.
In the section titled Identifying and viewing log file entries in Chapter 6, Monitoring Oracle SOA Suite 12c, we described how to search through logs for a particular ECID and how to enable Selective Tracing. This will help you trace the lifecycle of an ECID from beginning to end. You may have to increase logger levels if you would like more logging specific to a certain behavior (for example, if you only need to increase logging for adapter invocations).
How do you obtain the ECID anyway? There are a few approaches that can help.
If you are troubleshooting a particular instance in the console, the ECID is one of the columns in Search Results under Flow Instances, as shown in the following figure:
Figure 9.7: The initiating ECID is shown in the search results
Another approach is by obtaining the payload from any of the applications within the integration. For example, if the payload is still in an Oracle Advanced Queue (AQ), it is possible to perform some SQL queries to retrieve the payload. The ECID is included in the <instra:tracking.ecid> header element of the payload, as highlighted in the following code:
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsa="http://www.w3.org/2005/08/addressing">
<env:Header>
<wsa:MessageID>urn:5e0df372-0197-11e5-98b2-000c29547ca1</wsa:MessageID>
<wsa:ReplyTo>
<wsa:Address>http://www.w3.org/2005/08/addressing/anonymous</wsa:Address>
<wsa:ReferenceParameters>
<instra:tracking.ecid xmlns:instra="http://xmlns.oracle.com/sca/tracking/1.0">1063e612-47f5-4e9c-b9b1-7897053a07a7-00009ccb</instra:tracking.ecid>
<instra:tracking.FlowEventId xmlns:instra="http://xmlns.oracle.com/sca/tracking/1.0">10019</instra:tracking.FlowEventId>
<instra:tracking.FlowId xmlns:instra="http://xmlns.oracle.com/sca/tracking/1.0">10004</instra:tracking.FlowId>
<instra:tracking.CorrelationFlowId xmlns:instra="http://xmlns.oracle.com/sca/tracking/1.0">0000Kq2n3mE6ATMS6MGNsy1LMrKi000006</instra:tracking.CorrelationFlowId>
</wsa:ReferenceParameters>
</wsa:ReplyTo>
<wsa:FaultTo>
<wsa:Address>http://www.w3.org/2005/08/addressing/anonymous</wsa:Address>
</wsa:FaultTo>
</env:Header>
<env:Body>
.
.
.
</env:Body>
</env:Envelope>Now that you have the ECID, you can use it to search the logs.
We now move to an example where, perhaps, no action would be necessary on the part of the administrator. For example, the console reported a failure of a specific instance. On navigating to the faulted instance in the console, we see in Figure 9.8 how the instance appears to have retried:
Figure 9.8: The Flow Trace showing a retried composite executing successfully the second time
By clicking on CreateCustomer2 to drill down options in that component's flow, we get an output similar to the one shown in Figure 9.9. We can see here that there was an invocation to an external service which apparently took 20 seconds (compare 4:05:02 to 4:05:22). It turns out that we have a 20 second timeout configured on this reference. Excellent! This is the behavior we prefer. However, referring to Figure 9.8, we see that the instance was retried. This turns out to be a function of the fault policy attached to this particular composite. The composite has retried and the subsequent invocation went through just fine. Note that the retried instances are actually new instances. The reason why we see both the original transaction and the retried one in the same flow trace is because the same ECID is maintained between instance retries.
The following screenshot shows the Mediator component instance Flow Trace showing a timeout to an external service invocation:
Figure 9.9: The Mediator component instance Flow Trace showing a timeout to an external service
In this example, aside from trying to find out why the external service timed out the first time, there may not be much to do at this point.
More often than not, it is likely that you will encounter a generic ORABPEL-05002 error in your infrastructure. The error may be thrown for a host of common issues such as transaction errors, adapter errors, coding, configuration, and even transformation errors. The ORABPEL-05002 message is a wrapper-type generic error message and is always associated with an embedded error message, which causes the actual issue. It is therefore necessary to investigate an ORABPEL-05002 error in your log files and its entire trail to find out additional information within it. In case there is no additional message logged, set the BPEL engine loggers to FINEST, as discussed earlier. This will narrow down the subsystem or component causing the actual issue. Observe the error in the following code snippet where an ORA-01432 error, which occurs due to an invalid database insert, is wrapped inside the ORABPEL-05002 message:
Failed to handle dispatch message ... Exception ORABPEL-05002 Error while attempting to process the message "com.collaxa.cube.engine.dispatch.message.invoke.InvokeInstanceMessage"; the reported exception is: Exception not handled by the Collaxa Cube system. An unhandled exception has been thrown in the Collaxa Cube system; exception reported is "java.sql.SQLDataException: ORA-01438: value larger than specified precision allowed for this column