Looking at the Requirements for Cloud Integration
Many companies initially underestimate the challenges of integrating data across hybrid computing environments. (For more details on data integration you will want to refer to Chapter 11.) Most of the integration issues between public and private clouds will focus on SaaS applications. This may be a new experience for many IT organizations that are used to controlling the data sources they are integrating. They assume they already have the tools and expertise required to manage the integration process because of their prior experiences with integration in the data center. Traditionally, most organizations used a consistent process for moving data within a computing environment. This approach, known as Extraction, Transform, Load (ETL), is most appropriate for transferring and processing large volumes of data in fast, high-throughput transactional environments within an enterprise data center. However, ETL is inadequate for integrating across multiple delivery platforms of the cloud.
A major complexity with trying to make ETL work in a hybrid environment is the varied structure of the data. The IT staff typically doesn’t have the same level of insight into the data elements, data structures, data configurations, and database in SaaS platforms as compared to the applications managed and controlled in the data center. Therefore, it is important for IT management to spend the effort to gain as much understanding as possible into how the SaaS platform manages data to avoid confusion. This may require advanced training provided by the SaaS provider.
Accommodating SaaS platforms with information managed in the data center requires a lot of work on the part of the IT organization. With enough time and programming staff, companies can create custom-coded connections between internal and cloud applications. However, keeping a custom solution up-to-date can take significant, ongoing maintenance.
One of the characteristics of a SaaS environment is that the developer of that application often makes changes to the structure of the application without notifying customers. A SaaS vendor may suddenly change a data format that the customer is unaware of. These changes could potentially impact the integration process. In some situations, companies may facilitate the integration process between data in the public cloud and data in internal systems by creating an additional data repository. This causes unwanted delay and complexity.
You need an integration process that’s adaptable based on unexpected changes.
To maintain the benefit of using SaaS environments in concert with your line of business applications requires that you establish an effective and repeatable integration process. By leveraging new sets of integration platforms and best practices, you can overcome these integration challenges. Overall, you need a common and standardized way to link your applications wherever they’re managed — the five main requirements for creating this standardized approach to integrating data across internal data center applications and public and private clouds. These elements are covered in this section.
Connectivity
You need to be able to connect many different types of applications and data — SaaS applications, custom web applications, on-premise applications, private cloud applications, databases, flat-files — quickly and easily without requiring a lot of ongoing maintenance. You also need to consider different types of integration, including data migration, process integration, or some unique new type of integration, including taking data from an internal application, such as SAP, and then displaying the data in a SaaS application, such as salesforce.com.
You may need to make connections between two applications, or you may need to connect one application to many application endpoints. Even more important is the ability to scale quickly from a one-to-one integration to a one-to-many integration. In addition, different connectivity protocols or techniques may work better in different situations, so you must be prepared to choose different options for different business requirements.
Transformation
In a typical business, you often have to map the data about customers in your line-of-business application (such as accounting) with data about those same customers in your SaaS application. If you’re lucky, the formats of both these data sources will be the same. However, many times, these applications are designed or managed by different groups that never communicate with each other. For example, the IT organization manages the data in the ERP system, whereas the sales department has its own staff to manage the data in the SaaS CRM system.
Business management needs to ensure that the accounting system is consistent with the sales management system. Your IT staff is most likely familiar with the data format specifications in your legacy applications but doesn’t have the same level of understanding of the specifics of the data in your SaaS applications. One of the major advantages of SaaS applications is that business process owners can leverage these applications without any support from IT. All the data-management complexity is hidden from the user. However, in order to create these necessary mappings, you need to understand, for example, whether a customer identification (ID) number is numeric or if it includes alpha characters. After you understand the specific characteristics, you can graphically transform the ID number in both applications so they can be recognized and understood as the same information.
Service level integration
In a hybrid environment, you are asked to bring services together from different environments as though they are one unified environment. This type of service management is critical to the use of a hybrid environment. The goal is not simply to create combined value of public and private cloud services, but also there needs to be a way to create an overall SLA to support the combined environment. This service level must be planned to ensure the right overall security, governance, performance, monitoring, and management of this new virtual cloud world.
Business logic
The systems that have the data you value include business logic and processes that control the way that data is managed. You can’t simply connect data elements without a deep understanding of how these systems behave from a business process perspective. It is helpful, for example, if you can graphically define the flow of data between source and target applications. In this context, you can graphically define all the steps needed to extract purchase order data from your ERP specific system and send it to a different system (that is, a specific CRM system).
Unless your business is standing still, you can expect to see the SaaS vendor improve the underlying application. It may find a more efficient way to manage a certain business process that impacts how you connect the logic between various systems. However, the typical SaaS vendor doesn’t make arbitrary changes. The typical vendor bases its approach to integration on best practices in integration patterns and often reuses these common patterns. By understanding these patterns and watching for changes, your organization is better able to withstand changes in the implementation details.
One way to increase the speed of integration is to use an integration provider who’s studied metadata structure of SaaS applications. These vendors can provide a preconfigured integration pattern or template that jump-starts the effort of integration between data sources. One of the benefits of working with a standardized template is that the same template can be reused for other integration projects. The template is typically designed to cover about 60 percent of the requirements for a particular integration.
Management
Data doesn’t live in isolation. No matter what type of data you’re working with, it lives on specific hardware platforms, leverages specific storage environments, and connects with third-party services (payment services, credit verification, partner commerce systems, and so on). All these elements become part of the way you manage the flow of data between your applications in the data center and in the cloud.
From a management perspective, you must be able to monitor and manage these workloads. The approach you take largely depends on how you manage your business. For example, ask yourself these questions:
How many third-party services do I use?
Have SaaS applications, such as CRM, workforce planning services, and the like, become part of my IT strategy?
Does my business require seamless integration across these business services?
How much do my business partners and customers depend on the smooth and accurate integration across information sources and business services?
If you don’t have a well-planned way to manage these resources and services, the lack of planning can dramatically impact the overall efficiency of your hybrid environment. To be successful in breaking down data and processing silos, you have to focus on overall management of business workloads. This is especially true because these workloads are becoming increasingly fluid and mobile.