Troubleshooting SOA deployment issues

It is important to understand that deploying SOA composites does in fact take server resources. If you deploy tens or hundreds of composites to the server, you will see a direct impact on the size of the JVM heap (that is, the available heap will continue to go down as more and more composites are deployed within a short period of time). We therefore recommend you to not deploy during expected heavy or peak times.

Chapter 5, Deploying Code, describes how to use Ant to deploy composite projects, but here we take a closer look at the entire deployment process.

Knowing where to look

Let's observe the output of the Ant deployment of the HelloWorld composite in the following code. As you can see, the deployment failed due to a java.lang.OutOfMemoryError: PermGen space error:

deploy:
    [input] skipping input as property serverURL has already been set.
     [input] skipping input as property sarLocation has already been set.
     [deployComposite] setting user/password..., user=weblogic
     [deployComposite] Processing sar=/u01/svn/HelloWorld/deploy/sca_HelloWorld_rev1.0.jar
     [deployComposite] Adding sar file - /u01/svn/HelloWorld/deploy/sca_HelloWorld_rev1.0.jar
[deployComposite] INFO: Creating HTTP connection to host:soahost1, port:8001
     [echo]
     [echo] ERROR IN TRYCATCH BLOCK:
     [echo] /u01/scripts/build.soa.xml:112: The following error occurred while executing this line:
     [echo] /u01/scripts/build.soa.xml:138: The following error occurred while executing this line:
     [echo] /u01/share/oracle/middleware/products/fmw1213/soa/bin/ant-sca-deploy.xml:188: java.lang.OutOfMemoryError: PermGen space

Here, it appears that the SOA server has somehow run out of memory. This is not the case. In fact, this OutOfMemoryError exception is on the machine that executes the Ant command, not the SOA server itself. This could be confusing, couldn't it?

It is thus important to understand a few points regarding Ant deployments:

  • Deploying SOA composites is really a two-phase process: the compilation of code followed by the deployment itself.
  • The compilation of the composite takes place on the machine that executes Ant, not necessarily the SOA server.
  • The compilation process will validate external references. For example, if there is a schema imported in to the composite that resides on some external web server, it is validated at compilation time. If, for example, that web server is down during the compilation, the compilation will fail.
  • Although deployment failures are usually a result of some issue on the SOA server, this is not always the case.

The earlier error was thrown by the ant-sca-deploy.xml script, and therefore, we can assume that the error occurred in the local Ant runtime. The solution in this case is simple: increase the PermSize and MaxPermSize values in the ANT_OPTS environment variable prior to running Ant. To do so in Linux is simple:

export ANT_OPTS="-Xmx1536M -Xms1536M  -XX:NewSize=800M -XX:+UseConcMarkSweepGC -XX:SurvivorRatio=12 -XX:MaxNewSize=800M -XX:+UseParNewGC"

Compilation issues

The compilation of a composite takes place via the package target in the ant-sca-package.xml build script. This target calls several other targets to complete the compilation process.

It first calls the clean target, which removes any existing SAR files:

clean:
     [echo] deleting /u01/svn/HelloWorld/SOA/deploy/sca_HelloWorld_rev1.0.jar

It then creates the deploy subdirectory via the init target if it does not exist:

init:
    [mkdir] Created dir: /u01/svn/HelloWorld/SOA/deploy

The compile-source target is invoked:

compile-source:
    [echo] oracle.home = /u01/app/oracle/middleware/Oracle_SOA1/bin/..

The scac-validate target is subsequently called, which sets certain environment variables:

scac-validate:
     [echo] Running scac-validate in /u01/svn/HelloWorld/SOA/composite.xml
     [echo] oracle.home = /u01/app/oracle/middleware/Oracle_SOA1/bin/..

Finally, the scac target is called, which performs the actual compilation itself:

scac:
     [scac] Validating composite "/u01/svn/HelloWorld/SOA/composite.xml"
     [scac] error: location {/ns:composite/ns:import[@location='file:/u01/svn/HelloWorld/SOA/WSDLs/HelloWorld.wsdl']}: Load of wsdl "HelloWorldWebService.wsdl with Message part element undefined in wsdl [file:/u01/svn/HelloWorld/SOA/WSDLs/HelloWorld.wsdl] part name = parameters type = {http://soahost1/SOA/HelloWorldWebService}GetHelloWorldResponse" failed
     [scac] error: location {/ns:composite/ns:import[@location='file:/u01/svn/HelloWorld/SOA/WSDLs/HelloWorldWebService.wsdl']}: Load of wsdl "HelloWorldWebService.wsdl with Message part element undefined in wsdl [file:/u01/svn/HelloWorld/SOA/WSDLs/HelloWorldWebService.wsdl] part name = parameters  type = {http://soahost1/SOA/HelloWorldWebService}GetHelloWorldResponse" failed
     [echo]
     [echo] ERROR IN TRYCATCH BLOCK:
     [echo] /u01/app/oracle/middleware/Oracle_SOA1/bin/ant-sca-package.xml:46: The following error occurred while executing this line:
     [echo] /u01/app/oracle/middleware/Oracle_SOA1/bin/ant-sca-compile.xml:269: Java returned: 1 Check log file : /tmp/out.err for errors

A lot of times when compiling composites, the error returned on the prompt is not descriptive enough. Additional information can be found in the automatically created out.err file. This is located in /tmp/out.err on operating systems based on Linux/Unix and C:Users[user]AppDataLocalTempout.err on Windows. If the error on the standard output is not clear, we always recommend to review the out.err file for additional information.

Most compilation errors are actually due to coding issues. In the preceding example, one of the schemas in the project was to import a nonexistent schema. HelloWorldWebService.wsdl had the following import statement:

<xs:import schemaLocation="http://externalserver/schema.xsd" namespace="http://soahost1/SOA/HelloWorldWebService"/>

Trying to access http://externalserver/schema.xsd returned a HTTP Error 503. The service is unavailable error. Because this imported schema was not available and could not be validated at compilation time, the compilation failed as expected.

Another common issue at compilation time is when you use the oramds:// protocol to refer to external resources that are available in the metadata store. Here, you can see that the composite build failed when the HelloWorld.xsd that was being referenced from the MDS was not loaded during compilation:

oracle.fabric.common.wsdl.SchemaBuilder.loadEmbeddedSchemas (SchemaBuilder.java:492) Caused by: java.io.IOException: oracle.mds.exception.MDSException: MDS-00054: The file to be loaded oramds:/apps/Common/HelloWorld.xsd does not exist.

It is important to understand that at compilation time, Ant would refer to an MDS location that is configured in adf-config.xml. This file contains information to connect either a file-based or database-based MDS, and it is available in one of the following two locations:

  • Under the [Composite_Home]SCA-INFclassesMETA-INF project folder where the MDS configuration is specific to a particular composite
  • In the [Application_Home].adfMETA-INFadf-config.xml file under the application folder

An example of the database-based adf-config.xml is shown in the following code. This configuration file essentially tells your composite where your MDS is located, so that oramds:// lookups know where to look. As you can see, the highlighted text refers to the MDS database instance:

<?xml version="1.0" encoding="windows-1252" ?>
<adf-config xmlns="http://xmlns.oracle.com/adf/config" xmlns:adf="http://xmlns.oracle.com/adf/config/properties">
  <adf:adf-properties-child xmlns="http://xmlns.oracle.com/adf/config/properties">
    <adf-property name="adfAppUID" value="TemporaryMDS-2296"/>
  </adf:adf-properties-child>
  <adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace metadata-store-usage="mstore-usage_1" path="/apps"/>
        </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_1">
            <metadata-store class-name="oracle.mds.persistence.stores.db.DBMetadataStore">
              <property value="DEV_MDS" name="jdbc-userid"/>
              <property value="welcome1" name="jdbc-password"/>
              <property value="jdbc:oracle:thin:@//dbhost:1521/orcl" name="jdbc-url"/>
              <property value="soa-infra" name="partition-name"/>
            </metadata-store>
          </metadata-store-usage>
        </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config>
</adf-config>

It is recommended that developers set the MDS configuration at the application folder level, since it is not deleted after compilation and applies globally to all composite projects in the application.

Common deployment issues

Deployment issues have their own set of challenges. In some cases, the error is quite obvious:

deploy:
    [input] skipping input as property serverURL has already been set.
    [input] skipping input as property sarLocation has already been set.
[deployComposite] setting user/password..., user=welcome1
[deployComposite] Processing sar=/u01/svn/HelloWorld/deploy/sca_HelloWorld_rev1.0.jar
[deployComposite] Adding sar file - /u01/svn/HelloWorld/deploy/sca_HelloWorld_rev1.0.jar
[deployComposite] INFO: Creating HTTP connection to host:soahost1, port:8001
[deployComposite] java.net.UnknownHostException: soahost1

In other cases, it is more difficult. Consider the common error in the following code that you may face during composite deployment:

<Error> <oracle.integration.platform> <SOA-20003> <Unable to register service. oracle.fabric.common.FabricException: Error in getting XML input stream: http://soahost1:8001/soa-infra/services/default/QueryOrder?WSDL: Response: '503: Service Unavailable' for url:

This is a common deployment error that you are almost certain to encounter either when you are deploying a composite or when the composite is trying to register external references during server startup. However, you might find yourself in a predicament when the same error keeps on recurring, even when the reference service is available. What can you make of errors like these? First, let's understand what happens here. When you start a SOA server or deploy a composite, the engine tries to activate the composite by trying to register with all its endpoints (when lazy loading is not active). If unsuccessful, the composite is marked as inactive.

A target service being down can cause a composite to become inactive; the errors in this case can be prevented in the following two ways:

  1. From Oracle Enterprise Manager Fusion Middleware Control, navigate to the composite dashboard and reactivate the composite. The composite will now once again try to establish a communication with all its endpoints and attempt to start gracefully. This is often the first attempt made by the administrator.
  2. You can also altogether prevent the composite from going into a retired state in the first place by using abstract WSDLs of all the external services. When you use an abstract WSDL, the engine treats it as local and keeps the composite active. The concrete WSDL can be copied to a shared MDS location and in turn be referred from the abstract WSDL. This must be done at design time by the developer.

To briefly understand the difference between concrete and abstract WSDLs, click on the Common deployment issues icon on the composite dashboard of the referenced composite service. Refer to the following figure; this is the concrete WSDL:

Common deployment issues

Figure 9.12: The concrete WSDL as shown on the composite dashboard

If you click on the concrete WSDL, a new browser window opens, displaying its contents. In it, you can obtain the abstract WSDL, as shown in the following figure:

Common deployment issues

Figure 9.13: The abstract WSDL can be retrieved from the concrete WSDL

Figure 9.13 shows a version-specific abstract WSDL, which is:

http://soahost1:8001/soa-infra/services/default/HelloWorld!1.0/HelloWorld.wsdl

The default version can be referenced as follows:

http://soahost1:8001/soa-infra/services/default/HelloWorld/HelloWorld.wsdl

Here, it is the responsibility of the developer to make this change and ensure that references are made to abstract versus concrete WSDLs.

Consider another scenario where you have multiple composites deployed to your infrastructure and they refer to each other to achieve a business function. During server startup, if one composite A calls another composite B with a concrete WSDL file and composite A starts up before composite B, you will get the same error as shown earlier. This is because composite A is unable to load composite B's WSDL, which is not yet available. Unfortunately, it is not possible in Oracle SOA Suite 12c to control the order in which composites are loaded. In this case too, the same solutions that were discussed previously apply. The second approach is preferred, in order to save repetitive activation of composites every time.

Undeploying corrupt composites

In rare cases, the soa-infra application could cause the SOA managed server (for example, wls_soa1) to go into the ADMIN mode with a BEA-101216 error in the logs:

<Oct 4, 2015 12:45:23 PM EDT> <Error> <HTTP> <BEA-101216> <Servlet: "FabricInit" failed to preload on startup in Web application: "/soa-infra".
oracle.fabric.common.FabricException: Error in getting XML input stream: oramds:/deployed-composites/HelloWorld_rev1.0/composite.xml: oracle.mds.exception.MDSException: MDS-00054: The file to be loaded oramds:/deployed-composites/HelloWorld_rev1.0/composite.xml does not exist.

Here, the SOA Infrastructure is unable to start up due to a corrupt composite. The challenge is how to undeploy a SOA composite if the soa-infra application is not up and running. Fortunately, Oracle has provided a solution for this, which is detailed as follows:

  1. Confirm that the SOA managed servers are down (they should be).
  2. Refer to the Oracle Support document ID 1380835.1 and download ShareSoaInfraPartition.ear (this EAR file works for both 11g and 12c).
  3. Copy the ShareSoaInfraPartition.ear file to the $MW_HOME/oracle_common/common/bin folder.
  4. Start a WLST session by running the following commands:
    cd $MW_HOME/oracle_common/common/bin 
    ./wlst.sh 
    connect()
    
  5. Deploy the ShareSoaInfraPartition application:
    deploy('ShareSoaInfraPartition','ShareSoaInfraPartition.ear',upload='true')
  6. Export a list of deployed composites to an XML file under the /tmp folder:
    exportMetadata(application='ShareSoaInfraPartition',server='AdminServer',toLocation='/tmp',docs='/deployed-composites/deployed-composites.xml')
  7. In a separate window, edit the exported XML file:
    vi /tmp/deployed-composites/deployed-composites.xml
  8. Delete the five lines of the composite you want removed. For example, these following entries for the corrupt HelloWorld composite are deleted:
    <composite-series name="default/HelloWorld" default="default/HelloWorld!1.0">
    <composite-revision dn="default/HelloWorld!1.0" state="on" mode="active" location="dc/soa_9c645f3a-09eb-4878-bf05-178c62742955">
    <composite dn="default/HelloWorld!1.0*soa_9c645f3a-09eb-4878-bf05-178c62742955" compositeFileLabel="soa_cf_60902431-4473-408c-bc83-2ba9d7902903" scaEntityId="20001" deployedTime="2015-08-02T13:02:52.336-04:00"/>
    </composite-revision>
    </composite-series>
  9. Back in the WLST prompt, run this final command to import the XML file into the server:
    importMetadata(application='ShareSoaInfraPartition',server='AdminServer',fromLocation='/tmp',docs='/deployed-composites/deployed-composites.xml')
    exit()
  10. Start up the SOA managed server.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.145.179.59