As an administrator, you have already recognized the importance of establishing well-defined backup and recovery procedures. It is easy to write at length on this topic alone, discussing various backup, restore, failover, migration, and disaster recovery strategies. Fortunately, we will focus on the most important areas in this chapter to simplify the process for you as best as we can. As long as you understand a few core concepts regarding the overall backup and recovery strategy for Oracle SOA Suite 12c, you can implement it in any number of ways.
Establishing a backup and restore strategy is important because it provides you the ability to restore your environment in the event of a critical infrastructure or hardware failure. For instance, if you experience a hard drive failure, the disks may have to be replaced and the software restored from backup. It also provides you the ability to restore your environment to a previously working snapshot in the event of a faulty patch, faulty code deployment, or faulty configuration. In some cases, these faulty updates are not undoable and thus a restore may be needed.
In this chapter, we will cover the following key areas:
- Understanding what needs to be backed up
- The recommended backup strategy
- Implementing the backup process
- Recovery strategies
There are really two types of backups you can perform—offline backups and online backups. Offline backups are taken when the entire environment is down. This is the preferred approach, as all tiers are backed up at the same point, ensuring that a full restore will be an exact point in the time snapshot. Unfortunately, it is usually difficult to find the downtime needed for a full offline backup, and online backups are more commonly implemented. They are taken while the system is running and require no downtime. Compared to offline backups, online backups are generally more involved in their initial setup, and certain additional factors need to be considered to ensure that a proper restore is performed.
The types of data that are typically backed up are as follows:
- Static files (for example, domain configuration files, software binaries, and patches)
- Runtime artifacts (for example, application deployments, instance data and metadata, messages in queuing systems, and transaction logs)
So when should you back up your static files? As long as they don't change; technically, a single valid backup is all that is required. Configuration changes or the application of patches tend to change the contents of these static files, hence prompting the need to create another backup. Runtime artifacts or dynamic data, such as continually updated instance data in the database, may need to be backed up regularly. There are cases where both online and offline backups may be valid for these types of data, and we will discuss them in more detail later on in this chapter.
In the unfortunate event that a recovery is needed, performing a complete restore can guarantee full restoration of your environment. However, this is time consuming and the appropriate downtime may not be afforded to do so. Thus, once you understand the different types of files that need to be backed up, you will know what needs to be restored. Installing a highly available infrastructure helps reduce the risk by ensuring the continued operation of your environment in the event of a software or hardware failure, simplifying the restoration process. This is discussed in detail in Chapter 12, Clustering and High Availability.
Before describing how to back up your environment, it is important to understand what needs to be backed up first. We differentiate between static files, which are files that do not change frequently such as the software installation binaries, and dynamic data, otherwise referred to as runtime artifacts, which could include frequently updated data such as instance information and deployment metadata.
Static files and directories are those that do not change frequently. These files should be backed up when required, particularly when configuration changes, patching, or installations have been performed since the last backup. In most cases, static files can be backed up both online and offline.
System files include the oraInst.loc and beahomelist files. These files point to the location of Oracle Inventory and WebLogic Server Home, respectively. This enables future patching or installations of other Oracle products to easily recognize where your software is installed, check for their existing versions, and update these inventories accordingly, if needed. In Oracle SOA Suite 12c, Oracle inventory resides in Oracle SOA Home by default.
The beahomelist file is typically located in the users' home directory and varies according to the operating system. The BEA Home list contains the location of your Oracle WebLogic Server installation (which is also installed under Oracle SOA Home).
The oraInst.loc configuration file contains the location of your Oracle inventory. Oracle inventory contains metadata of all the installed Oracle products on your server. It is updated when new Oracle products are installed on your server or the existing software is upgraded. An example of the contents of oraInst.loc is as follows:
inventory_loc=/u01/oracle/products/Oracle_SOA1/inventory inst_group=oinstall
The locations of each of these files in a standard Linux installation are shown in the following table:
|
File |
Description |
Sample location |
|---|---|---|
|
The BEA Home List |
This points to the Oracle WebLogic Server installation directory and is typically created in the users' home directory. |
|
|
Oracle Installation Location |
This is a configuration file that points to the location of Oracle Inventory. |
|
|
Oracle Inventory |
The location is determined by |
|
These files should be backed up after initial installation or after any patch update or upgrade. A standard filesystem backup is sufficient.
A JDK is typically installed outside of Oracle SOA Home. The location of your JDK can be shared between the servers in your cluster or it may be installed on each server separately.
Regardless of its location, your JDK should be backed up after new installations and/or a patch update or upgrade. Starting with Java 7, JRockit and Hotspot are merged into a single JVM to make a best breed of JVM. In the previous version of Java, you would have to select either the Sun JDK or JRockit JDK for installation—this is no longer the case with Java 7.
Here is an example of the location of a JDK installation outside your Oracle SOA Home:
/u01/oracle/products/jdk1.7.0_79
Oracle SOA Home contains numerous files, components, and shared components. This possibly includes the binaries and configuration files for Oracle WebLogic Server, Oracle SOA Suite, OSB, OEP, BPM Suite, BAM, and Oracle Common components.
For example, Oracle SOA Home, denoted as the $ORACLE_HOME environment variable, may be set to /u01/oracle/products/Oracle_SOA1. The backup commands later on in this chapter back up everything under $ORACLE_HOME and should be run after initial installation or after any patch update or upgrade. A standard filesystem backup is sufficient.
The Oracle documentation refers to data that is dynamically updated and required for runtime operations as runtime artifacts. In essence, it is the data and/or configuration that is regularly accessed and/or updated during runtime. We will describe each of these areas in the later sections of this chapter.
A database is configured during the installation to store the SOA Infrastructure (instance-specific data) and MDS (deployment metadata) data. Oracle SOA Suite 12c, or rather, Oracle Fusion Middleware 12c (12.1.3) is currently certified only on Oracle database versions 12.1.0.1+, 11.2.0.3+, and 11.1.0.7+. The Oracle database can be backed up via a tool such as Oracle Recovery Manager (RMAN).
The database maintains data related to in-flight instances, deployed SOA composites, and configuration, among other things. For example, if you deploy a composite today and perform a database restore for the day before, that deployed composite will not be available. Also note that some configurations, such as that of the Oracle BPEL Process Manager Engine, Oracle BPM Engine, Mediator, and Common Configuration, are stored in the database.
Configuration data is typically stored in the database, ensuring that each managed server in the cluster has a consistent view of this data.
The schemas that need to be backed up are listed in the following table. It is recommended to have nightly backups of the database for critical business applications:
|
Schema |
Description |
|---|---|
|
|
Enterprise Scheduler Service |
|
|
Audit Services |
|
|
Metadata Services |
|
|
Managed File Transfer |
|
|
Business Activity Monitoring |
|
|
Oracle Platform Security Services |
|
|
SOA Infrastructure |
|
|
Services Tools Bundle |
|
|
Persistent stores host information such as JMS queues and JMS topics, also referred to collectively as JMS destinations. Persistent stores can either be file-based or JDBC-enabled (that is, saved in the database). Most Oracle SOA Suite 12c clustered installations utilize file-based stores for performance purposes. File-based stores essentially provide persistence capabilities to Oracle WebLogic Server subsystems and services through the use of a built-in, high-performance storage solution.
Because these file-based persistent stores save JMS messages, durable subscriber information, and temporary messages sent to unavailable destinations using the Store-and-Forward features, it is not possible for us to take their consistent backups. Restoring these persistent stores may result in data inconsistency, even if they were backed up offline. These persistent stores often (and should) reside on redundant fault-tolerant storage that is accessible to all nodes of the Oracle SOA Suite 12c cluster. Alternatively, you may have already used JDBC-enabled stores for your JMS destinations so that they are maintained in, and thus backed up with, the database (meaning filesystem backups of these objects are not necessary).
The following table shows an example of the location of persistent stores required by Oracle SOA Suite 12c clusters. These stores are shared and accessible to all nodes of the clusters (in this example, a two-node cluster):
|
JMS file store name |
Shared | |
|---|---|---|
|
|
|
Yes |
|
|
|
Yes |
|
|
|
Yes |
|
|
|
Yes |
|
|
|
Yes |
|
|
|
Yes |
|
|
|
Yes |
|
|
|
Yes |
By virtue of the file stores being shared, they are accessible to all nodes in the cluster. The recommendation here is to implement one of the following backup strategies for JMS file-based persistent stores:
- Move the JMS modules to a JDBC-enabled persistent store. Database backups will ensure the consistency of the data in the event of a database restore, but restoring to older backups of the database may result in data inconsistency.
- Ensure that the file-based persistent stores reside in fully redundant shared storage accessible to all nodes of the cluster, and guarantee its availability. Backing up these stores is possible, but there are implications regarding message loss and message duplication should you choose to restore them.
Perhaps the only scenario where file stores can be backed up is in non-production environments, where data consistency is not critical. In this case, it is still recommended that you take an offline backup (that is, where all midtier nodes are shut down while the backup is performed). Restoring from this backup essentially performs a point-in-time recovery, where the file stores may contain older JMS messages that have already been consumed and processed. It may also result in lost messages. For example, this may include messages that may have been enqueued but not consumed before the backup. The only advantage of backing up your JMS file store is that it allows the administrator to always take a single, working, and consistent backup of the environment, but with the risk of data duplication and/or inconsistency as a result of restoring older file stores.
Transaction logs store information about committed transactions that are coordinated by Oracle WebLogic Server that may not have been completed. The transaction logs, or TLOGs, provide Oracle WebLogic Server with a mechanism to recover from system crashes or network failures.
TLOGs can either be saved to WebLogic Server's Default Store or to a database via a JDBC store. TLOGs that are targeted to Default Store are file-based, and thus the backup behavior is similar to that of JMS file-based persistent stores. Oracle WebLogic Server 12c, however, allows transaction logs to be stored in a JDBC store, thus eliminating the need for filesystem backups. The following screenshot shows where to select the Transaction Log Store type:
Figure 10.1: Selecting the Transaction Log Store type in the WebLogic Server Administration Console
In the event that the Default Store is selected, it is recommended to set it to a directory on highly available and shared storage. This is typically a requirement for multinode installations of Oracle SOA Suite 12c anyway. This can be configured by logging in to the Oracle WebLogic Server Administration Console, navigating to Environment | Servers | soa_server1 | Configuration | Services, and setting the value of Directory.
The recommendation, therefore, is to ensure that the transaction logs that are logged to Default Store reside in fully redundant shared storage that is accessible to all nodes of the cluster and guarantee its availability. Backing up the transaction logs is suggested, but there are implications regarding message loss and message duplication should you choose to restore them.
For transaction logs that are stored in the database, ensure that the appropriate database schema they are connected to is backed up regularly via RMAN.
Typically, every Oracle SOA Suite 12c installation includes at least one domain extended with SOA extensions (for example, soa_domain), which hosts your administration and managed servers configuration. Though many of the files in your Domain Home are static in nature, several of them change periodically (for example, log files). By default, Domain Home is located under the Oracle SOA Suite installation directory (for example, $ORACLE_HOME/user_projects/domains/soa_domain), but it may reside elsewhere depending on what was specified during the domain creation. A typical SOA domain will fall within the 2 GB to 4 GB size range and will contain a multitude of file types that include the following:
- Startup scripts
- Libraries
- Domain configuration files
- Logs
- Deployed and extracted Java applications
The configuration files most relevant to your domain installation are listed in the following table. Backing up your domain is usually recommended if one or more of these files are updated:
|
Component |
Configuration file location |
|---|---|
|
Domain, JMS, BAM, B2B, Business rules, BPM, OWSM, and OPSS configuration |
|
|
Oracle WebLogic Server Startup Scripts |
|
|
Oracle WebLogic Server Node Manager |
|
|
Oracle WebLogic Server |
|
|
Oracle WebLogic Server |
|
In most cases, you do not need to back up managed server directories separately because AdminServer contains information about all the managed servers in its domain.
Managed servers do not require AdminServer to be up during normal operation. However, as an administrator, your view of the health and performance of the infrastructure may be restricted. Furthermore, it prevents you from making any changes to the domain's configuration if AdminServer is down.
A managed server maintains a local copy of the domain configuration. If you attempt to start a managed server while AdminServer is down, the managed server uses a local copy of the domain configuration and continues to periodically attempt to reconnect with AdminServer. Therefore, the managed server runs in what is called the Managed Server Independence (MSI) mode. When it connects successfully, the configuration state is synchronized.
It is, therefore, recommended that you take a full backup of your Domain Home periodically while your infrastructure, especially AdminServer, is offline. Technically, backing up Domain Home on the machine that runs AdminServer is sufficient, though it might make your life easier to back up the domain on the other servers in your cluster as well.