How to migrate SOA and AIA Projects from 10g to 11g

Once all the initial Server Setups are completed Migrate the 10g Projects to 11g. First install the latest 11g Jdev compatible with SOA Suite Server Installed.(Steps provided in previous blog)

Local Desktop Setup:

• Oracle Jdeveloper (11.1.1.6). Download Location: http://download.oracle.com/otn/java/jdeveloper/11.1.1.6.0/jdevstudio11116install.exe

·         Update JDev with latest SOA

SOA design time in JDeveloper requires a JDeveloper extension called SOA Composite

Editor. While this is normally updated over the network when using release level software,

you can perform the update manually if you have the extension file.

http://www.oracle.com/ocom/groups/public/@otn/documents/webcontent/156082.xml#oracle.sca.modeler

Start Jdeveloper

Select Help/Check For Updates, Click Next

Select Install from Local File, Browse to <From your download location>\soa-jdev-extension.zip

Click Open, click Next

Steps to be followed before Migration of BPEL Projects

·         BPEL Projects have to be migrated in order of dependency.

·         Open BPEL Projects in Jdev10g and modify PartnerLink endpoints to refer 11g soa server and to the Project WSDL’s already deployed in 11g.

·         In 10g Release 3 (10.1.3) application upgrading includes a project where all the project artifacts are in one folder and the BPEL artifacts are not separated into their own folder in the project, then the Oracle JDeveloper Migration Wizard will not be able to upgrade those artifacts. To avoid Create a BPEL folder and move all the artifacts (except the build.xml and build.properties files) into that folder.

·         Save and open the project in 11g Jdeveloper. Migration wizard appears. Navigate through this wizard and if any error occurs refer logs.

Steps to be followed before Migration of ESB Projects

·         ESB Projects have to be migrated in order of dependency.

·         Open ESB Projects in Jdev10g and modify PartnerLink endpoints to refer 11g soa server and to the Project WSDL’s already deployed in 11g.

·         In case of ESB Projects if the routing rule does not have a SOAP Service as target manually add one.

·         Ensure that “Invocable from external webservice” is selected for all ESB Routing services.

·         Save the Project in Jdev 10g.

·         Open all .esbsvc files in text editor.

o   Delete all Parent lines which begin with the tag <parent>.

o   Change the qname references to remove system and service group names retaining only the Project name.

·         Open the project in 11g Jdeveloper.Migration wizard appears. Navigate through this wizard and if any error occurs refer logs. The ESB projects will be migrated as Mediators within the SCA composite.

Upgrade Tool

There is a command line upgrade tool in 11g which upgrades both Oracle BPEL Process Manager projects, and upgrades Oracle Enterprise Service Bus 10g projects to Oracle Mediator 11g.

With the command-line tool, can merge BPEL projects together and create a single composite out of multiple BPEL projects. This is not possible with the Oracle JDeveloper Migration Wizard.

The functionality is exactly the same as the JDeveloper mode when it comes to dealing with SOA project contents because the same codebase is used.

Post Migration Steps in BPEL Projects

Once all the BPEL projects are migrated some Post migration steps have to be performed before testing.

1. Composite.xml changes: Custom Properties in bpel.xml of 10g to be migrated to composite.xml in 11g manually. Ex:

<property name="jca.retry.count" type="xs:int" many="false” override="may">60</property>

2. DVM Changes: Open files which has DVM Lookup and modify the DVM function name and location manually. Ex:

10g Function: orcl:lookup-dvm ("Currency", "A", "B", "C", "D")

11g Function: dvm:lookupValue ("oramds:/apps/CustomMetaData/dvm/Currency.dvm", "A", "B", "C", "D").

3. Schema Imports: Every schema import should be manually modified to use ORAMDS references instead of HTTP references. Ex:

10g Import: <schema import="http://host:port/CustomMetaData/xsd/abc.xsd">

11g Import: <schema import="oramds:/apps/CustomMetaData/xsd/abc.xsd">

4. Fault Policy and Binding: Manually reference the fault policy and fault binding in composite.xml. There is no concept of global fault policy in 11g.
1. Edit Composite.xml
2. Add the following immediately below the </service> tag to refer the fault policy:

<property name="oracle.composite.faultPolicyFile">

oramds:/apps/CustomMetaData/faultPolicies/fault-policy.xml

</property>

<property name="oracle.composite.faultBindingFile">

oramds:/apps/CustomMetaData/faultPolicies/fault-bindings.xml

</property>

5. DB Adapter Migration: 1.Using the database adapter if the connection to the database does not exist then create a new connection to database in 11g Jdeveloper and configure the adapter to point to the new database connection.
6. Java Embedding: Jar files imported to the Project class path need to be referred from 11g installation directory.
7. File/FTP Adapter: In SOA Suite 11g, adapter header variables are now properties that have to be set on the Properties tab of the Invoke dialog box in Oracle JDeveloper.
8. Async Error Handling BPEL Process: If the BPEL Processes are using the AIA Async Error Handling BPEL Process, modify the partner link urls to refer to the SOA 11g server.

Post Migration Steps in ESB(Mediator) Projects

Once all the ESB projects are migrated as Mediators in 11g some Post migration steps have to be performed before testing.

1. DVM Changes: Open files which has DVM Lookup and modify the DVM function name and location manually. Ex:

10g Function: orcl:lookup-dvm ("Currency", "A", "B", "C", "D")

11g Function: dvm:lookupValue ("oramds:/apps/CustomMetaData/dvm/Currency.dvm", "A", "B", "C", "D").

2. Schema Imports: Every schema import should be manually modified to use ORAMDS references instead of HTTP references. Ex:

10g Import: <schema import="http://host:port/CustomMetaData/xsd/abc.xsd">

11g Import: <schema import="oramds:/apps/CustomMetaData/xsd/abc.xsd">

3. Mediator Filter Expressions: Review the Mediator filter expressions as the migration wizard does not migrate these expressions accurately. Check if all the operators and Special characters are present in the expression.
4. Xpath Header Functions: ESB XPath header functions are made obsolete in 11g and replaced with alternate Mediator XPath functions.Ex: ehdr:setOutboundHeader is changed to mhdr:setproperty. Hence make these changes and test these header functions.

Testing for SOA Processes

Once all the initial setup and migrations are done test the flow to verify successful Migration.

Limitations

·         BPEL process instances that are in 10g of any state (including completed/faulted/stale) cannot be migrated to 11g.

·         Many artifacts have to be manually moved from 10g repository to 11g MDS, if any of the artifacts are missed the Execution Fails.

·         In Oracle 11g OWSM, policies has to be attached to the webservice endpoints.10g Policies cannot be used.

Steps to upgrade SOA and AIA environment from 10g to 11g

SOA Suite Server Setup

11g SOA server consists of a Domain on which Admin server and Managed server is hosted. There could be many managed servers across systems but only one Admin Server can exist within a domain. Start the SOA Suite Admin server and managed server and make the Application server changes mentioned below. At Application Server level in 11g we need to make the following custom configurations manually:

• Data Source: Data sources and Connection Factories are used by SOA code to connect to a resource or target system. In 10g these custom datasources are configured in EM Console. Whereas in 11g we need to configure all the custom datasources using the same names and setting is Weblogic Console by navigating to <Domain_Name>->Service->Data Sources.
• Connection Factory: In 10g we configure Connection factory details in EM Console via: Cluster Topology->oc4j_soa->Applications->default->Adapter->Connection Factories. In 11g we should configure these custom Connection factories in Weblogic Console by navigating to<Domain_Name>->Deployments->Adapter->Configuration-Outbound connections.
• Partitions: In 10g we have BPEL Domains to group the BPEL Processes, ESB system and service groups to group ESB services. There is no concept of these domains and service groups in 11g but concept of Partition is introduced which do not have their own configuration and logging but to group BPEL and ESB processes it can be used. With the command line utility we can create partitions with the following command: wls:/mydomain/ServerConfig>sca_createPartition(‘cdh')
• BPEL Domain Properties: Since 11g has a single domain and 10g has multiple domains we need to migrate all the BPEL domain properties from 10g to 11g.Properties in the domain.xml for each domain of 10g has to be copied manually to bpel-config.xml/soa-config.xml file. For more info on this refer the Document: http://download.oracle.com/docs/cd/E17904_01/upgrade.1111/e10127/upgrade_bpel_apps.htm#CHDFDEDB with Chapter: Specifying Domain Descriptor Properties

Other Custom Configurations done in 10g also has to Moved to 11g manually.

AIA Setup

In 11g all the artifacts which have the configuration details are stored in a central repository: MDS unlike 10g where the configurations are stored in file systems. Hence to migrate the same Custom Configuration files we need to import the changes to MDS. Following steps has to carried out for AIA setup and migration from AIA FP(2.x) to AIA FP 11g.

• AIA Configuration Properties File: Get a copy of AIAConfigurationProperties.xml file from AIA(2.x) server in SOA 10g.To migrate the AIA Config Changes the AIAConfig file present in 11g server has to be exported from MDS by running the ant command: ant -f ant-sca-deploy.xml exportSharedData with all the required parameters. Manually copy the custom configurations from current AIA config file to 11g file along with port and Hostname changes and import it back into MDS.
• Login to AIA 11g Console and navigate to Setup->System and note the configuration settings there. Login to AIA 11g (11.1.1.6) Console and login as weblogic user that has “AIAApplicationUser” group and navigate to Setup->System and configure the same settings as in AIA 2.x.
• To import CAVS test cases: Login to AIA console and export all the test cases as files. Login the AIA console 11g as weblogic user that has “AIAApplicationUser” group and under CAVS->Definitions select Import and Import the testcase files one at a time.
• Importing Custom EBO’s: Checkout the Custom EBO’s from perforce and later modify it with these custom configurations and import it into MDS.

 10g Key Artifacts Setup

In 11g all the artifacts which have the configuration details are stored in a central repository: MDS. MDS is Metadata Services repository which is database based and centralized across all midtiers. However hierarchy in MDS is similar to a file system like 10g where the configurations are stored in file systems. MDS setup is very flexible as we can browse through the setup from Jdev, export and import the artifacts back to Server by using ANT commands or through EM Console. Hence the artifacts present in 10g are stored in MDS for Migrations by following the below mentioned steps:

1. DVM Upgrade:
• Export all the DVM’s from ESB Console or from ESB slide in 10g and modify each DVM by including namespace: xmlns=http://xmlns.oracle.com/dvm and removing isNew=”true” entry.
• Change the export DVM file name extension to ‘.dvm’ instead of ‘.xml’.
• Rest of the contents remains same. These steps can also be done by running the ant build file: “upgrade-xrefdvm” target in the “ant-sca-upgrade.xml”.

2. Fault Policies:
• Pick all the fault policies from all the domain of SOA 10g(fault-binding.xml and DefaultPolicy.xml).
• Modify the fault.bindings.xml by changing the version to “3.0” and <process> tag to <composite>
• Rename DefaultPolicy.xml to fault-policies.xml.
• Change the Version in fault-policies.xml “3.0” and add a higher level fault policy node: <faultPolicies xmlns="http://schemas.oracle.com/bpel/faultpolicy" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
• Create a folder in the local file system and copy both the files in this location.
• Zip up the contents and import these Fault Policies to MDS using Ant.

3. Import Shared Schemas and WSDLs: If there are shared Schema’s and WSDL’s follow the below mentioned steps:
• Export the files from SOA 10g. Please it in the local file system.
• Make sure to change the Server hostname and Port to 11g server and have the same directory structure as 10g.
• Zip the contents of these folder and import to MDS using ant command.

References and Installations required for Upgrade of SOA Suite and AIA Environment from 10g to 11g

Reference Oracle Documentation

Refer the following documents provided by Oracle to understand the complete Upgrade Process as an initial step:

• Oracle Fusion Middleware Upgrade Planning Guide 11g Release 1 (11.1.1)

http://download.oracle.com/docs/cd/E17904_01/upgrade.1111/e10125/toc.htm

• Oracle Fusion Middleware Upgrade Guide for Oracle SOA Suite, WebCenter, and ADF 11g Release 1 (11.1.1)

http://download.oracle.com/docs/cd/E17904_01/upgrade.1111/e10127/toc.htm

• Oracle Fusion Middleware Developer’s Guide for Oracle SOA Suite 11g Release 1 (11.1.1)

http://download.oracle.com/docs/cd/E12839_01/integration.1111/e10224/title.htm

• Oracle® Fusion Middleware Migration Guide for Oracle Application Integration Architecture11g Rel 1  http://www.art2dec.com/documentation/docs/fmw11g1115documentation/doc.1111/e17361/ch1.htm#sthref12

Installations

Oracle SOA Suite 10g and 11g have different infrastructure. For upgrade and migration we need to have both the Environments SOA Suite 10g and 11g with AIA 2.x and AIA 11g (11.1.1.6) running in parallel respectively. This is for easy reference as 11g is installed in a completely new stack and all the code will be deployed to a completely new environment. For Installing Oracle SOA Suite 11g and AIA FP 11g the below mentioned steps have been followed:

Ø  Server Side Installations

• Install Oracle Database 11g (11.2.0.x)
• Install Oracle WebLogic Server 11g (10.3.6)
• Create SOA database schemas using the Repository Creation Utility (RCU) 11g
• Install Oracle SOA Suite 11g
• Create SOA domain
• Install Oracle Service Bus

Ø  Developer Installations

• Install JDeveloper 11g (11.1.1.6) for SOA Artifacts development.
• Oracle Eclipse Pack for OSB Development

Refer the following document for the detailed steps: http://docs.oracle.com/cd/E23943_01/doc.1111/e13925/toc.htm

Refer the following document for the detailed steps on Installation of SOA 11g:

Refer the following document for the detailed steps on OSB Installation:

Next Install AIA FP on Oracle SOA Suite 11g. Refer the following document for the detailed steps: http://docs.oracle.com/cd/E23943_01/doc.1111/e17949/toc.htm

Convert xsd:anyType data into XML Data

Recently I encountered this requirement to Accept AnyType Data and convert it into a valid xml data.

Few important things to consider is: The AnyType data payload should be a valid one with the namespaces defined.The type to be converted to should be declared in an xsd.

In my process I had to get this anyType data and convert it to a format with which I had to invoke a service.
Though it sounds very simple I felt its better to document it in a very easily accessible way. I followed the following steps for the same in SOA 11.1.1.4:

1. Create 2 variables to which you want to convert the anytype Data to.
2. Use the following function to Parse the value:oraext:parseXML(bpws:getVariableData('inputVariable','request','/ns2:Request/ns2:Details/ns2:xmlData')) where xmlData is of datatype Any.Please note that when you are selecting the anyType data, in Assign activity this will not get copied in the expression builder, hence you need to manually add this data.
3. Once this Assign activity is completed, use Assign or Transform activity to transform the values within the xml generated in the above mentioned step to another variable which is used to invoke the service.

I hope this helps for people looking for usage of oraext:parseXML.

Dynamically Set Archive Drectory Location for FTP Transfer

In Some of the scenarios of FTP File transfer in a typical BPEL Process, we might need to set the location of Archive Directory  dynamically, either by picking from some configuration properties file or from the payload at runtime. To achieve this we need to follow the below mentioned steps:

1. In FTP WSDL file set the jca properties to select logical directories for Archive or source file location directory.Also Set the property "UseRemoteArchive"  to true.  
             <jca:operation FileType="binary"
             LogicalDirectory="directory"
             InteractionSpec="oracle.tip.adapter.ftp.outbound.FTPReadInteractionSpec"
             LogicalArchiveDirectory="FileArchiveDirectory"
             DeleteFile="true"
             FileName="fileName"
             OpaqueSchema="true"
             UseRemoteArchive="true" >
2. In the BPEL service flow, when we invoke the FTP adapter to either get/put files, we can see that bpel.xml file gets update with the partner link binding details. In the partner link binding details keep the values blank for LogicalArchiveDirectory.                                                                                                      <property name="FileArchiveDirectory" type="LogicalArchiveDirectory"></property>.
3. However we can set the outbound directory location by using ftpAdapterOutboundHeader. We cannot do the same for the Archive Directory. Hence we will use the following function in the JavaEmbedding to set the ArchiveDirectory.                                      Location.getLocator().lookupProcess("<<ProcessName>>").getDescriptor().getPartnerLinkBindings().getPartnerLinkBinding("<<PartnerLinkBindingName>>").setPropertyValue("FileArchiveDirectory",(String)getVariableData("archiveDir"));  
   }    
   catch(Throwable ex){  
   addAuditTrailEntry("Exception - " + ex.getMessage());  
   }                          This way we can set the Archive Directory Location in the bpel.xml file dynamically  where getVariableData("archiveDir") is the archive directory location variable set in the bpel flow.                                     

JTA transaction is not present or the transaction is not in active state

You might have faced this strange error in your BPEL flow which occurs when you are connecting to a particular Partner link from the BPEL process. If this occurs in 10.1.3.5 server you can resolve it by following the steps mentioned below. The error thrown will have the error details as:

ORABPEL-02182

JTA transaction is not present or the transaction is not in active state.
The current JTA transaction is not present or it is not in active state when processing activity or instance "100560639-BpInv4-BpSeq4.3-9". The reason is The execution of this instance "100560639-BpInv4-BpSeq4.3-9" for process "XYZ" is supposed to be in a jta transaction, but the transaction is not present or in active state, please turn on the application server transaction debug logs to get more information..
Please consult your administrator regarding this error.

To resolve this you have to follow the following steps:

1. Stop the SOA Server 
2. Set Transaction-timeout value to 7200 or more in the following file: $SOA_Home\j2ee\config\transaction-manager.xml
3. Set transaction-timeout value to 3600 or more in $SOA_Home\j2ee\application-deployments\orabpel\ejb_ob_engine\orion-ejb-jar.xml 
4. Set syncMaxWaitTime to 240 or more in $SOA_Home\bpel\domains\config\domain.xml.
5. Restart the SOA server.

This should resolve the JTA transaction issue

Unable to trasfer files using FTP Adapter in BPEL

IF you face issues to do an FTP transfer to the remote location from your BPEL / ESB code Please check and verify the following:

1. Check the connection details for that particular Connection factory. You would have given these connection details like host, port, username, password either in the wsdl of the process or you would have referred to a JNDI whose connection details are given in $ORACLE_HOME/j2ee/oc4j_soa/application-deployments/{domain_name}/FTPAdapter /oc4j-ra.xml file.check these connection details with some external FTP clients too.
2. If the connection is still failing change the port to either 21 or 22 and test it.
3. If the password has expired, refresh the password to the existing one or change it to a new password.
4. If the connection to the remote location is successful but the files are still not getting transferred then check the property UseSFTP in oc4j-ra.xml file.This value should be set to true in case you are making secure ftp connection.
5. If these values are set and still not able to make the FTP connections, check the permissions on the folder.
6. Last but not the least check if the file which has to be transferred exists and has access permissions,