A concept represents a core unit of thought, such as a “person,” “product,” or “supplier.” A concept is associated with core characteristics. For example, a “residential property” is associated with a geographic region, a mailing address, an owner, an appraised value, possibly a mortgage, and a real estate tax assessment, among others. In turn, the characteristics may themselves be concepts as well.

Each environment may have a specific set of concepts that are relevant to the business processes, and some concepts transcend specific divisions, organizations, or even industries. Within the organization, a good place to seek out concepts is with the business process model, as was described in Chapter 2. At the macro level, standards for information exchange among participants within the same industry provide an ample resource from which concepts can be identified as well.

As part of the process of enumerating business concepts, participants may find that some concepts are referred to with a variety of words or phrases. At the same time, one may discover that certain words are overloaded and refer to more than a single concept. Collecting the concepts and their names leads to the next component of the business definitions layer: business terms.

How is the term “customer” used? Many words and terms are used so frequently within a business environment that they eventually lose their precise meaning in deference to a fuzzy understanding of the core concept. Alternatively, many organizations have a regimented lingo that confuses almost everyone except for the hardcore organizational veterans. Both ends of this spectrum reflect different aspects of the same problem: organizational knowledge locked inside individual's minds, with no framework for extracting that knowledge and clarifying it in a way that can be transferred to others within the organization. It is this gap that the business terms component is intended to alleviate, which naturally follows from the identification of the business concepts described in Section 6.2.1.

Given a list of business concepts, the next step is to identify the different terms used in reference to each concept and create a mapping through which the subject matter experts can browse. The simplest approach is to develop a direct mapping between the concept and its various aliases. For example, for the concept of “customer,” the terms “customer” and “account” may have the same intention. However, this process should not be limited to developing a direct mapping between terms and concepts, but it should also include any business terms used in any type of reference. This will include the different terms used for the concept “customer” as well as the terms used for the characteristics of a customer (“customer type,” “relationship start date,” “contact mechanism,” etc.).

The metadata asset also supports a process for determining what each business term means in each of its contexts. Again, referring back to the business processes, evaluate the use of the business term, connect it to a business concept, and seek a clear definition for each business term in relation to a business concept drawn from an authoritative source. The range of authoritative sources includes internal documentation and external directives. For example, the concept of a “customer” may exist in one form in relation to the sales staff based on internal memos and corporate dictates. However, when reporting the count of customers in regulatory reports to government bodies, the definition of the “customer” concept may be taken from the regulatory guidelines. Both are valid uses of a concept, but it may turn out that ultimately, based on the different definitions, the business term “customer” actually is defined in two different ways, meaning that the term is used in reference to two different concepts! This dichotomy must be documented, and the semantics component (see Section 6.2.4) is the best place to capture that.

The definitions component harmonizes term usage and enables users to distinguish concepts based on classification by authoritative source. This includes the listing of authoritative sources and prioritization key for those sources so that if there is a conflict between two definitions, one can use the prioritization key to break the tie. Recognize though, that if one were to assess the different uses of the same business terms and find multiple definitions, it may turn out that despite the use of the same term, we really have multiple concepts that must be distinguished through some type of qualification. This is also documented as part of the semantics component.

The semantics components is intended to capture information about how business terms are mapped to concepts, whether business terms are mapped to multiple concepts, and whether concepts are mapped to multiple business terms, as well as to describe how the business concepts are related within the organization. To some extent, the semantics component captures the interconnectedness of the business concepts, business terms, and the definitions. Documenting business definitions using prioritized authoritative sources is a relatively formal practice that some people may find constraining, especially when attempting to shoehorn multiple concepts into one specific definition. The semantic component enables a more fluid practice, allowing the coexistence of concepts that are similarly named as long as their meanings are qualified.

This is the component in which practitioners collect similar concepts and terms together, qualify their meanings, and determine if there is any overlap among or between them. If there is, then the relationship is documented; if not, then the specific differences must be clearly specified and a means for distinguishing the meanings in application contexts determined.

An evaluation of the business concepts will reveal hierarchies associating specific logical notions or objects together. For example, there is a concept of a “U.S. state,” which represents a geopolitical subregion of a country (another concept) named “United States of America,” which in its own right is a concept. Whereas the “U.S. state” conceptual domain is composed of the concepts representing each of the states of the Unites States of America—Alabama, Alaska, and so forth through Wyoming—the conceptual domain does not direct the way the concepts are represented; this is done using value domains.

Despite the fact that they are “conceptual,” there will typically be some basic representation for the objects that compose the domain set. Continuing the “U.S. state” example, there must be some representation of each of the states that conveys the standard agreed-to meaning. Therefore, one of any number of value domains may be selected as that basic representation, as we see in the next section.

A value domain is a collection of representations of the values in a conceptual domain. To continue our example, we can define a collection of character strings that refer to each of the states of the United States of America: “Alabama,” “Alaska,” and so on.

As Figure 6.2 shows, different value domains may be associated with a single conceptual domain. In this case, U.S. states are represented by their full names, by their U.S. Postal Service two-character codes, by Federal Information Processing System (FIPS) two-digit codes, or even by graphical images showing each state's boundaries. Each of these data sets is a value domain, each has a unique representation for a concept (each individual state) that is included in the higher-level concept (U.S. states), and, in fact, each value in one value domain maps to a corresponding value in the other value domains. This means that we may have data sets that are intended to represent the same concept yet use different value sets in representation; capturing this information within the metadata repository enables making the necessary links when determining ways to integrate and consolidate data sets.

On the other hand, we may have a value domain that is used to represent different conceptual domains. Figure 6.3 shows a value domain consisting of the numerals 0, 1, 2, 3, 4, 5, and 6 used to represent four different conceptual domains. So even though the same values are used, their use across different data does not necessarily imply that the data sets represent the same business concepts. Again, capturing this in the metadata repository is critical when considering approaches to data integration and consolidation.

For any given conceptual domain, there must be a way to document the connection with a specific value domain and how the values within the value domain refer to the objects within the conceptual domain. Reference tables essentially capture this information by providing a direct one-to-one mapping between an enumeration of the basic representation of a value domain representing the conceptual domain and a value domain used in an application data context. These are often manifested in the organization as “code tables” or “lookup tables.”

For each of the conceptual domains shown in Figure 6.3 that use the same value domain, there is a direct reference table showing the relationship between concept to value. Consider the example of “Weekdays” presented in Table 6.1.

Another artifact of the historical variance of data representations is that different information architects may have selected different value domains to represent objects within the same conceptual domain. Yet before integrating records drawn from different data sets, one must know when different value domains are used to represent the same conceptual domain. Contrast the reference table in Table 6.2 for weekdays with the one in Table 6.1. The same conceptual domain is associated with two different value domains, and to appropriately establish that two records contain the same weekday value, one must know the mapping demonstrating equivalence between the two value domains.

Empirical data analysis (such as that performed using data analytics or data profiling tools) can reveal the mappings between value domains within the context of the conceptual domain. A mapping for our weekday example would show the relationship between the reference data sets (Table 6.3).

Of the thousands of data elements that could exist within an organization, how would one distinguish “critical” data elements from your everyday, run-of-the-mill data elements? There is a need to define what a critical data element means within the organization, and some examples were provided earlier in this chapter. For an MDM program, the definition of a critical data element should frame how all instances of each conceptual data element are used within the context of each business application use. For example, if the master data is used within a purely analytical/reporting scenario, the definition might consider the dependent data elements used for quality analytics and reporting (e.g., “A critical data element is one that is used by one or more external reports.”)

On the other hand, if the master data asset is driving operational applications, the definition might contain details regarding specific operational data use (e.g., “A critical data element is one that is used to support part of a published business policy or is used to support regulatory compliance.”). Some other examples define critical data elements as follows:

Critical data elements are used for establishing information policy and, consequently, business policy compliance, and they must be subjected to governance and oversight, especially in an MDM environment.

As data elements will ultimately reflect the instantiation of the concepts and business terms described in Section 6.2, it is important to capture precise definitions of the data elements that will contribute to the master model. In fact, one might consider the requirements to precisely define data elements to be as great, if not greater than that for business terms.

Section 4.1 and Section 4.2 of Part 4 of the ISO/IEC 11179 standard provide guidance for data definitions, and we can apply these to the definition of a data element. A definition should state what the data element is (not what it “does” or what is isn't), be stated with descriptive phrases, not rely on uncommon abbreviations, and be expressed without incorporating definitions of other data concepts or elements. In addition, a definition should state the essential meaning of the data element, be precise and unambiguous, be concise, and avoid circular reasoning. Avoid using a data element's functional use as its definition or including procedural information.

In modern data modeling environments, every data element is attributed by a data type (e.g., integer, varchar, timestamp) and a size or length, but there are still many data environments that are less structured. Older file-based systems were defined with specific data element sizes but without enforcing any rules regarding data type compliance. As part of the data format component of the metadata repository, each captured data element will also be attributed with the format of the valid value set.

This may be limited to data type, such as CHAR(2) for a “U.S. state” data element that uses a U.S. Postal Service state postal code for its value domain. Alternatively, the format may contain a more complex formatting information that reflects a greater degree of constraint, such as limiting North American Numbering Plan telephone numbers to data type of CHAR(12) and a format “999-999-9999,” where the 9's represent digits only; any value that does not conform to the format must be invalid. Lastly, specifying an enumerated data value domain to a data element provides a greater degree of format constraint, because the data element's value must be selected from that set of values.

Recognizing that different data elements ultimately contain data that represent the same underlying business concept allows the metadata analyst to establish a relationship between those data elements in that they are aliases or synonyms. Synonym data elements may or may not share the same value domains, data element formats, names, and other components, but through their connections within the metadata hierarchy one can determine that they represent the same notions and must be associated.

This observation underscores the value of the hierarchical layering of master metadata. This value is driven by the exposure of concept and type inheritance, because it provides the ability to determine that two data elements refer to the same concept, even if empirically they look completely different. Let us revisit our example of documenting mappings between value domains associated with the same conceptual domain from Section 6.3.4. Suppose the metadata analyst were presented with two data elements. The first is called “DayOfWeek” and details are presented in Table 6.4; the second is called “Weekday” and details also are presented in Table 6.4.

In isolation, other than slight similarity between the names, empirical analysis of the data elements' values would not suggest any similarity between them, yet the connectivity established in the metadata registry shows that both data elements employ data value domains that are mapped together. Documenting data element synonyms is valuable when determining data extract and transformation rules for master data integration.

Analysts can speculate on which master data object class types exist within the organization. Typical lists start with customer, product, employee, and supplier—the conceptual entities with which the organization does business. Yet there is a difference between the conceptual entities that the applications are expected to use and the actual instances in which these entities are documented and ultimately managed.

Analysis of the data sets from the bottom up and the business processes from the top down will help subject matter experts identify recurring concepts, manifested as information entities that share similar, if not identical structures. These similar entities reflect themes that are commonly used across the application fabric, and patterns will emerge to expose the conceptual master objects in the organization. Cataloging the details of agreed-to master data object types that are used (or will be used) helps in later processes for mapping existing instance models to the master model for each object type. In turn, a focused review of the data objects that are used and shared across applications will reveal a more comprehensive list of potential master data objects.

This component of the metadata model is used to maintain the set of master object types and will incorporate a list of the master object concepts with a logical enumeration of the high-level structure. More important, this layer should also document the relationships that are manifested through the business process and work flows.

Having articulated the types of each master object used in the organization, a resolved master model for each object type will be developed as the core representation. Each logical model reflects the attribution for the master object, referring to data elements defined at the data element layer. This will, by default, also provide the associated agreed-to definitions for the data elements.

The existence of defined models drives three management activities related to application migration and development. First, the model becomes the default for persistence of master data. Although the actual architecture will depend on many operational and functional requirements, the core model managed within the metadata repository is the logical starting point for any stored data systems. Second, within each business area, there must be object models suitable for application program manipulation, and these object models must correspond to the persistent view. Third, master data will be shared among applications, some of which may be legacy applications that require wrappers and facades along with transformations into and back out of the master model, suggesting the need for defined exchange model for information sharing.

To facilitate the development of a migration strategy for legacy applications, as well as to document entity use for planning and impact analysis, the metadata repository should also maintain a directory that maps the applications that use specific master objects. The mapping should describe whether the application uses the version of the master data object as presented by the MDM environment or whether the application uses an internal data structure that represents (and perhaps copies data in from and out to) a corresponding master data object structure. In other words, this component tracks how master object types are mapped to the applications that use them.

Lastly, the metadata repository will maintain the actual relational data models for both the master object types and the instance representations used by the applications.

Just as we maintain a mapping from the logical use of master data objects to their use by applications, a more general mapping from the data entities and their associated data elements to their application use can be maintained within the metadata repository. This mapping is critical for assessing the need for data transformations, data migrations, and the development of functional infrastructure supporting the movement of instance data from the application silo to the master data asset.

The business policies and their corresponding information policies provide the context for assessing the available data sources and how those data sources are used to populate the master data asset. More important, that process also will result in sets of data rules and directives that indicate quantifiable measures of information quality. These rules are managed as content and can even be linked to tools that automate data inspection, monitoring, event notification, and continuous reporting of master data quality.

A key component of establishing data governance is through the use of SLAs. That data quality SLA should delineate the location in the processing stream that it covers, the data elements covered by the agreement, and additional aspects of overseeing the quality, as listed in the sidebar. All of these will be documented as metadata within the repository, which will simplify the data stewards' ability to manage observance of the agreements.

Consolidating data instances into a master data view presents some potential issues regarding security and privacy, necessitating policies for access rights and observing access control according to those policies. Consider that capturing identifying information in a master registry does not only enable the consolidation of data—it also enables the segregation of data when needed. For business processes that must enforce policies restricting access to protected information, policy compliance can be automated in an auditable manner.

There must be defined roles associated with master data access, whether by automated process or by individuals. In turn, each of these roles will be granted certain rights of access, which can restrict access at a level as granular as the data element level within record sets limited by specified filters. Individuals and stakeholders within the organization are then assigned roles and corresponding access rights; these assignments are archived within the metadata repository as well.

There will be two collections of services. The first is an enumeration of the essential services employed by client business applications at a conceptual level, such as “create a customer” or “update a telephone number.” Master services can be segmented as well into core object services that address typical data life cycle events, such as “create or modify an object,” or business services applied as part of the business process workflow, such as “generate invoice” or “initiate product shipment.”

The second collection is a current view of the (possibly multiple) ways that each conceptual master service is actually deployed within the current environment. This is intended to assist in the development of an implementation road map by identifying the functional components to be ultimately replaced that will require an engineered wrapper during the migration process.

In addition to documenting the list of services and the way each is currently deployed, the services metadata layer will also list the clients of the services. This is a list of both automated clients and individuals that invoke the functionality that will ultimately be implemented within the set of enumerated services. This inverse mapping from service to client also is used in impact analysis and migration planning, both for the determination of risk during the transition from the legacy framework to the MDM environment and for ongoing management, maintenance, and improvement of master data services.

There are metadata representing the different types of services and the users of those services. What is left will comprise the third component of master data services metadata, which captures the interfaces used by the clients (both automated and human) to invoke those services. Consolidating functionality into services must ensure that the newly created services support the application's current functional requirements, and that includes details about the different ways the services must be invoked and, consequently, any necessary parameterization or customization for the service layer. Alternatively, as functions are evaluated and their invocation methods reviewed, it may become apparent that even though the functionality appears to be the same across a set of applications, the ways that the functionality is invoked may signal discrete differences in the effects intended to occur. Capturing this information interface layer will help analysts to make this assessment.

At a simplistic level, a business policy is a statement that guides or constrains a business process as a way of controlling the outcome as well as side effects of the business process. Business policies, which may be either documented or undocumented, reflect general practices to be observed by those within an organization along with those that do business with the organization.

Business policies are expressed in natural language, but requiring subject matter experts to capture these policies within the metadata repository is a way to encourage more precision in expressing policies. The objective is to specify business policies in a manner that can be linked to the ways that the applications enforce them. Presumably, it may be possible to meet this objective by using information already managed within the metadata repository: concepts, business terms, business definitions, and semantics that are associated with commonly used business language. Business policies are more likely to be well structured if they have been specified using terms with agreed-to definitions.

The difference between a business policy and an information policy is that a business policy guides the business process whereas an information policy guides information architecture and application design. An information policy specifies one (of possibly many) information management requirements to support the observance of business policies. In the data-sharing example in Section 6.8, one business policy translated into three information policies. In turn, an information policy may guide the specification of one or more business rules.

A business rule specifies one particular constraint or directive associated with a data element, a collection of data elements, one record, a set of records, and so forth. A rule may specify a constraint and be used to filter or distinguish compliant data instances from noncompliant ones, and it could also trigger one or more actions to be taken should some condition evaluate to true. One or more business rules can be derived from an information policy, and documenting these rules within the metadata repository enables the application of the rules to be automated via an associated rules engine. The metadata repository provides a centralized location for the subject matter to be reviewed by experts before it is deployed into the rules engine.