Data Ownership

Each database is managed by a service unit that owns primary responsibility for the security, integrity, recoverability, and residency (geographical location) of the data. The database contains data for which the service unit is the master source along with data obtained from other service units and data created locally to support the service unit operation.

The agile enterprise will have many service units and thus many databases. The increased number of databases and frequency of shared data exchanges are enabled by internet technology, advances in middleware technology, and increased communications bandwidth.

The master data of a service unit may be data on a specific subject, or it may be a subset of data of a particular subject. For example, order management may be performed by different implementations of an order management service unit that serves different countries. Each service unit is then the master source for orders originated in its assigned geographical area.

Data Encapsulation

Every database or other form of data storage is accessed only by a service unit that is the data owner. This hides the particular data storage structure and technology from other users of the data, and it provides the opportunity to control access and the integrity of updates to the data. Every service unit has associated shared data and includes the services necessary for accessing the data and accepting updates.

Data elements for widely referenced business entities, such as customers, suppliers, employees, products, and the organization structure, each may be maintained in one or more independent service units.

Service Unit Groups

Service units may be clustered in a service unit group where they provide similar capabilities for economies of scale in the management of resources and technology. A group may share a database. When a database is shared by multiple service units, each of the supported service units has ownership responsibility and control for a portion of the database.

The service units of a service unit group must be managed by a common parent organization. This common manager has responsibility for the sharing of data between the service units. This group organization may improve performance by supporting sharing of data between the service units without replication that requires coordination of updates, but the cluster should, nevertheless, involve service units with similar capabilities that together represent a more general capability.

Data Sharing

A service unit will respond to the authorized queries against its master data, and it will honor requests for events to be generated when selected master data elements change. These change notices may be generated by actions of the service unit, but the notices can be managed more effectively if they are initiated by database triggers.

Master data may be requested from a service unit as needed, but for operating efficiency, a receiving service unit may store and maintain a replica of selected data obtained from the master source. Notices of change will keep the replicated data current (see notification service discussion in Chapter 8). The use of replication of data will depend on the frequency at which a receiving service unit must access the data, the frequency of updates to the data, and the volume of data that meet the recipient requirements.

Note that a service unit may associate local data with a business data entity without necessarily sharing all the data associated with the business entity. For example, the line items of an order might be replicated in the production service unit, while the status and operational detail associated with each line item remains local to the production service unit.

Access Locking

If synchronization is critical, requests to the master data service may require that data be retrieved from the master service unit with a lock (or “check-out” and later “check-in”) that prevents changes to the locked data until the requester has completed the associated operation and releases the lock. A service unit may store data from another service unit without locking to reduce the need for frequent retrievals and locking of stable data elements. However, without locking, there is a risk that the retrieved copy will become out of synchronization with the master, while another operation relies on an updated version of that data. Notices of changes will provide the opportunity to update the copy, but then it is still possible that the recipient has already relied on obsolete data.

Locks should be avoided if at all possible due to the risk of performance consequences to other services that would be locked out and the risk that interactions of multiple lock requests could result in a deadlock.

For example, consider if A obtains a lock on X and B obtains a lock on Y, then B requests a lock on X which is suspended pending release by A, and A requests a lock on Y that is suspended pending a release by B. Both A and B are then suspended pending further action by the other.

Legacy Systems

Some service units may be implemented as interfaces to legacy systems or purchased software that has not been designed for compatibility with the enterprise architecture. Data from these different sources must be accessible and combinable to provide a consistent view of enterprise operations. The service unit must provide an interface appropriate to its services and the data that are associated with it as a master data owner. The service unit(s) must ensure that the legacy system respects the ownership of master data of other service units. Compatibility is achieved through message transformation, discussed in Chapter 3.

Outsourcing

Services may be outsourced at a variety of levels of granularity. For example, a service could be as simple as a stock market query about a current stock price, or it may be outsourcing of a complex set of services that provide a major portion of the human resource management business function. Where the outsourcing represents a substantial business function, it includes management of enterprise master data, such as employee records in the case of human resource management outsourcing, and it may involve one or more external service units. Those service units, for the most part, will be capability service units, delivering business services, as discussed later.

The objectives for outsourced services should be the same as for corresponding internal service units. Services should have well-defined interfaces that are independent of whether the services are internal or outsourced. If the outsourcing is a substantial business function, it conceptually includes a number of service units characterized as service unit groups.

If the outsourcing provider implements best practices, the service interfaces should be consistent with service interfaces that its competitors either have or will have in the future. The outsourcing provider must provide access to its service units for sharing data that are needed or provided by other users within the client enterprise. For example, in the case of HR outsourcing, the client will need access to employee records along with a variety of services related to employment, benefits, compensation, and so forth, depending on the scope of the outsourcing.

Implementation of business capabilities as services should minimize the changes necessary for implementation of outsourcing. However, management of master data by the outsourced services must be reconciled with the internal service units that share that data. The internal organization responsible for management of the outsourcing relationship must have ownership responsibility for the outsourced master data.

The interfaces of the outsourced service unit(s) must continue to be supported for access to the outsourced capabilities and data. It may be necessary for the interface implementation to generate events and manage locks for changes to shared master data, and process updates for data from other sources.

Legal Records

The enterprise will receive and create electronic records that have legal effect, including internal records that involve authorization of transfers of assets or expenditures. These records should be represented in XML with electronic signatures. Some will also be encrypted for confidentiality. XML is discussed further regarding message exchange later. See the discussion of encryption and electronic signatures in Chapter 7.

These records must be preserved in their original form for the signatures and encryptions to remain valid. It is the responsibility of the service unit that creates the record or receives the record from an external source to preserve the original form along with a decrypted copy for internal sharing and processing. Some records will be cumulative, with cumulative signatures. Each extended record must be viewed as a new version of the record.

Data Compatibility

Throughout the enterprise, shared data must be compliant with a shared enterprise logical data model (ELDM), discussed further in the later section on metadata. The format of messages as exchanged must be consistent, but the format can be reconciled by the message exchange middleware discussed in Chapter 3. A service unit may use different internal formats for shared data and manage additional data elements and structures that are not shared.

However, shared data must not only have consistent elements and semantics (the meanings) but also represent a consistent point or period in time. Conventional enterprise applications control the relationships between the data they manage so that, within the scope of the enterprise application, data retrieved for a particular view are consistent. In contrast, service units are loosely coupled and must ensure that their database elements, particularly measurements, are consistent in time.

For example, some measurements may be accumulated weekly, while others are monthly. These measurements cannot be converted from one interval to the other.

Export Regulation

Enterprises often have operations in multiple countries. Many countries have laws or regulations that restrict the export of certain categories of data across their borders. This is particularly true for data related to personal privacy. The development of cloud computing has added the risk that such data could be deployed to a cloud computing node in a different country for workload balancing or to compensate for failure of another node.

Standards for control of data residency are currently under consideration in the OMG (Object Management Group). One approach could be to tag potentially sensitive data so that restrictions can be applied by the message exchange service based on the regulatory requirements affecting communication from the sender location to the receiver location. An attempt to send a message in violation of an export regulation should be detected at the sender. Of course this also requires that the message exchange facility knows the current geographic locations.

Life-Cycle Ownership

Typically, a service receives a business request to produce a product or service for an external or internal customer. The request is pending until either it fails or the desired result is delivered. The service owns the request for the life-cycle of the request. This supports updates of the associated master data.

Consequently, a service retains responsibility for a request until all work is completed. Instead of transferring responsibility, the service unit delegates to service units that contribute to fulfillment of the request. The service unit that receives a request is the service unit that delivers the result.

In traditional, silo-based business systems, a request will be accepted by an organization and associated system, and when that organization's work is completed, it will transfer responsibility for continued work to the next organization. So each of these organizations has responsibility for a segment in the request life-cycle and thus responsibility for maintaining the associated master data.

Under certain circumstances, a request will be received as an asynchronous message that initiates another value stream, typically to deal with exceptions or support services. These have the form of a transfer of control. The service unit that accepts that transfer of control is then responsible for the life-cycle of that request for action. See the example in Bottom-Up Analysis in Chapter 3.

The service unit that receives a request is responsible for the master data records for that request. However, note that each of the service units engaged to contribute to the work receives its request and has life-cycle responsibility to resolve that request. As a result, a query for the status of a primary transaction is directed to the primary service unit that may, in turn, propagate the request through active delegations to provide more detail about the current status unless the delegation services send updates to the master data of the primary service. So the whole truth about a particular order, person, or product design may require propagation of a query to multiple service unit databases.

The identifier of the original request should be part of the request to each of the delegated services. This allows for data about a particular stage of processing to be requested directly from the service unit that performed that work.

Data Warehouse Support

Data on selected service events are sent to the data warehouse. These will be used for analysis of trends in performance and product or resource demand. The specific events and the scope of the event notices delivered to the data warehouse will depend on the value streams and business requirements for the data warehouse to support data analyses.

Measurement Reporting

Capability service units are the sources of value contributions. Measurements of value contributions may be taken at appropriate points in a service unit to correspond to collaborations and activities in the enterprise value delivery model. The level of granularity may vary depending on the level of concern and the ability of the process management services to capture appropriate measurements.

Potentially, detailed measurement reporting would be turned on when a particular value stream or capability is of particular interest and turned off when only high-level performance reporting is needed.

Traditional Mode of Operation

Under established business intelligence services, a data warehouse receives inputs of events from business services as well as other sources and updates its database for an integrated view of the data. Inputs are typically taken as periodic batch extracts from production systems. The general assumption is that the cross-enterprise reporting is not time critical, so periodic updates provide data that are sufficiently current.

The process for feeding a data warehouse is often called extract transform load because the relevant data must be extracted from the production systems, transformed to a common format, and loaded in an appropriate manner to be reconciled with data from other sources. Transformation may include “cleansing” the data and merger of related data from different sources. The OMG Common Warehouse Metamodel (CWM, http://www.omg.org/technology/documents/modeling_spec_catalog.htm#CWM) specification provides a standard for modeling the transformation of data from different sources and different format technologies.

Agile Enterprise Data Warehouse

The agile enterprise supports more timely data warehouse analysis by accepting events as they occur. Unlike traditional data warehouse implementations, the agile enterprise data warehouse is limited to capture of internal events provided by capability service units and data from other sources are received by the analytics service unit. Possible events include initiation of a process, completion of a process, or completion of an activity within a process. Not all service events at all levels are of interest, so output of event notices should be turned on for events of interest.

The data warehouse provides these data for analysis by the analytics service unit. We include traditional business intelligence services under the heading of analytics. The measurements service unit provides another perspective for support of analytics.

Knowledge Conversations

There are two popular forms of electronic conversations: email and group discussions. Both of these are based on communications among members of an interest group. An interest group has a web page that identifies the area of interest, the list of members, any access restrictions, and a form to apply for membership in the interest group. The web page should provide access to the email threads and discussions described below as well as the knowledge documents discussed later.

The interest group membership should be limited to enterprise employees. Members of an interest group should be self-selecting except that participation in some groups must be authorized if the interest group deals with restricted subject matter.

Knowledge Documents

Knowledge documents include two basic forms: authored documents and encyclopedia entries.

Measure Specifications

VDML captures value measurements. A Measure Library defines the measures, the Value Library defines the use of the measurements, and the VDML model defines the context in which these measurements occur. Various other VDML elements, such as participants, resources, and activities, may have associated measurements that describe their characteristics or behavior. For example, a store may record the number of business items waiting for an activity, or a deliverable flow may have a specified transit time.

The VDML model as context is important for reporting measurements of the Measurements Service Unit. Measurements received from business operations will be recorded on appropriate model elements. Users will look at the model to find relevant measurements.

Measurements may be captured in different contexts (eg, product mix or production rate) and associated with different VDML scenarios. The scenario defines the particular business context of the measurements.

These measurements may be reports of individual measures of occurrence (eg, one execution of an activity), or they may be statistical values with a mean and standard deviation representing the measurements during a period of time. Additional metadata may be associated with the VDML context element such as the source of the measurement. The association of measurements with activities, capabilities, and organization units provides identification of responsibility for performance measurements.

Business Items

Business item elements represent things that flow through activity networks and between collaborations. These may be records, orders, resources, products, parts, and so on. From a data modeling perspective, these are views based on a particular business entity. For example, a business item may be a customer order including order items, item detail, such as descriptions and prices, and a reference to the customer. The order is the key entity, identified by an order number, and the business item is represented as a nested data structure.

The VDML model does not include individual business items unless it has been extended to support simulation. The business item elements are described in the Business Item Library, and the data structure can be described to the extent it is relevant to the VDM analysis. The detail of the Business Item structure can be found in the ELDM. Additional metadata, such as data quality and range, may be included in the Business Item Definition and the data directory.

Industry Standards

From an industry perspective, understanding the data exchanged between enterprise applications was a major challenge of enterprise application integration. Suppliers of enterprise systems realized that the integration of their systems would be less costly and disruptive if they could exchange data consistent with a common data model, so they promoted the development of standards for exchange data. Many such standards were developed by the Open Applications Group, Inc. (OAGi).

Common data models are evident in industry standards such as those developed by the United Nations Centre for Trade Facilitation and Electronic Business, XML Business Reporting Language, and Human Relations XML. In the telecommunications industry, the TeleManagement Forum has developed a substantial ELDM called Shared Information/Data.

As the scope of electronic integration of the enterprise expands to include relationships with customers and suppliers, the scope of need for data consistency expands. When orders were exchanged, and coordination was accomplished by human-to-human communications, the people involved were able to compensate for inconsistencies, translating the terminology and the data formats as required. Computers are not so forgiving. Thus, there is a need for a shared specification for data exchange.

As a result of these forces, the world of information systems is converging toward a common LDM. Although new concepts emerge, we can expect this industry convergence to continue so that most differences will be only where innovative business solutions provide an enterprise with competitive advantage. For the most part, even significantly different business processes and product designs will use accepted interchange data models.

To the individual enterprise, industry convergence on a common data model reduces the cost of developing data models and database designs; reduces the cost of developing, maintaining, and executing data transformations; and improves the ability to measure and analyze operating performance and key indicators from an enterprise perspective. Consequently, it is the interest of every enterprise to align as much as possible to the emerging global logical data model.

Logical data models that should reflect the emerging common model are available as products for purchase or as a basis for consulting and integration services. Work on an ELDM should not start from scratch.

Data Modeling

In the meantime, each enterprise must establish and maintain an ELDM as the basis for exchange, storage, and retrieval of shared data. The primary form of enterprise data modeling is class models, generally associated with object-oriented programming and the Unified Modeling Language (UML, http://www.omg.org/spec/UML/2.4.1). These models provide a relatively robust way of representing business entities, their attributes, and their relationships. Implementations of computer applications usually employ relational databases, so for database technical design, entity-relationship modeling is more suitable, but we do not get into that level of detail here. We focus on class models that are the generally accepted form of logical data models.

A class defines the representation of similar persons, places, or things in the business domain. Class models rely on a concept called inheritance, whereby a class may inherit characteristics defined for a more general class. Fig. 6.2 illustrates a class diagram for a customer-order example. A class may represent a generalization of similar things, such as Customer, or it may be specialized to represent a more specific type of things, such as Retail Customer or Wholesaler. The more specialized class is said to inherit from the more general class, so characteristics of Customer are inherited by Wholesaler, as indicated by the open-headed arrow from Wholesaler to Customer in the diagram. This is a UML class diagram. UML is the generally accepted standard developed by OMG for modeling a number of aspects of an object-oriented application.

Inheritance provides a way to describe similarities between classes. Several different classes may inherit common characteristics from a shared parent class, or a specialized class may be defined as adding characteristics to a more general class. In the class diagram in Fig. 6.2, the Order, Customer, and Order Item classes show attributes of the classes within the boxes. The Wholesaler class is a specialization of Customer. Wholesaler inherits Account No, Name, and Address from Customer and has an additional attribute that is a discount specification that presumably is not meaningful for most customers. Wholesaler implicitly has all the other attributes and relationships of Customer, so a Wholesaler “is” a customer with an additional attribute. A Wholesaler can occur anywhere in a model or database that a Customer can occur. The numbers on the relationships indicate the cardinality, for example, an order must have one customer, but a customer can have zero or more orders (0..⁎).

The ELDM is represented with a class model, but the data may be expressed in different forms for storage and exchange. Generally, data exchanged between services are expressed in XML. Data stored in databases are most often in relational tables. Data warehouses have specialized data structures to support retrieval and analysis. Transformations between these and other representations can be modeled with tools based on the OMG CWM.

An ELDM must represent everything that may be shared through message exchanges, event notices, or queries. Service units may also include their local data since it will be related to the service unit master data and data it replicates from other service units. This may require thousands of classes, but not all these classes are implemented in a single database. A database stores the subset of the data that are of interest to the associated service unit or group of service units.

For example, in a manufacturing enterprise, incoming orders are captured by the order management service unit along with related sales and customer data. The order requirements are included in the delegation to the order fulfillment service unit, but without the sales and customer detail. The order fulfillment service unit will add data related to scheduling the order and may translate sales order specifications to production specifications before submitting the production request. During production, the production service unit may add data regarding the specific configuration, component serial numbers, or batches applicable to each order. So, as the order proceeds through fulfillment and closure of the sale, each of the service units will receive the data required to do their work and add data relevant to their work. Some data will ultimately be added to the completed master order in the sales organization and other data will be held as local, working data in the participating service units.

To get a comprehensive report on a customer order, it might be necessary to query several databases that hold data related to that order. When such reporting, analysis, or archiving occurs, the ELDM ensures that the data from different sources can be combined to produce a consistent representation and the meaning will be generally understood.