Introduction
About This Book
This book is about one of the top technology trends in the area of information management. This trend, known as Master Data Management, and its sister discipline—Master Data Governance—has evolved and become more pronounced over the last two to three years, and is considered by many analysts to be a high-impact, high-complexity, and broad-applicability technology that is focused on new ways of structuring, choosing, understanding, integrating, and disseminating information that is needed to run a business, service customers, and comply with numerous regulatory requirements.
To paraphrase Claude Shannon (the “father” of information theory and the concepts of information entropy), information is that which resolves uncertainty. Our entire existence is a process of gathering, analyzing, understanding, and acting on information. Progressive resolution of uncertainty is the key to the way we make business and personal decisions. The need to sustain new regulatory pressures and achieve competitive advantages by managing customer- or product-level profitability and risk-adjusted return on investment drives profound changes in the way business and government organizations operate. Traditional account-centric and application-specific silos of business processes restrict organizations’ ability to meet the aforementioned challenge. Therefore, in order to succeed in today’s highly competitive global and dynamic markets, businesses are making serious investments in the new business-entity-centric processes and technical capabilities. These new capabilities should allow organizations to effectively focus on selecting, acquiring, understanding, and managing accurate and relevant information about their primary business targets—customers, products, partners, patients, inventories, prices, services, and other areas of business concerns.
In doing so, enterprises are collecting and processing ever-increasing volumes of information, especially as business conditions change, markets shrink or expand, companies grow organically or by acquisitions, and customer retention and products’ timely market introduction and competitiveness become some of the key business metrics.
As we entered the digital age, this accumulation of data has been accelerating. Now we have access to the ocean of information that was created by or stored in computer systems and networks over the last several years. This information now includes not just traditional structured data, but also semi-structured (that is, images, graphics, full-motion video, sounds) and unstructured data. Indeed, we brought with us data that previously existed only in nondigital form, such as books and paper documents. We have learned to digitize that data quickly and efficiently and thus created even more computer files and databases, all the time hoping that all this “stuff” will be managed transparently and effectively by our reliable, trusted computer systems and applications. The reasons for engaging in this data collection are obvious: We live in the age of digital information, where the Internet and the World Wide Web have made enterprise boundaries porous or fuzzy in order to attract a large number of customers and to enhance their experience. In this new digital age, an agile enterprise can become competitive only when it has access to more relevant, accurate, timely, and complete data about business conditions and performance metrics, enterprise customers, prospects and partners, products and markets, and a myriad of other things. Having the right data at the right time is even more imperative if you consider the challenges and the revolutionary nature of transforming a traditional account-centric business into a customer- or product-centric, global, agile, and intelligent enterprise.
Given the ever-growing amount of data that is collected and managed by the business units of global enterprises today, we are facing the difficult challenge of creating, finding, selecting, and managing data that is complete, accurate, relevant, and secure, and that is uniformly available to all enterprises and users who need this data to run their business. This challenge of creating and managing a new authoritative system of record is the focus of Master Data Management (MDM) and its closely related management discipline—Data Governance. The issues, approaches, concerns, drivers, benefits, architecture, applications, and trends of Master Data Management and Data Governance are the subject of this book.
About the Second Edition
The first edition of the popular book titled Master Data Management and Customer Data Integration for the Global Enterprise, by Alex Berson and Larry Dubov (McGraw-Hill, 2007), proved to be a timely, useful, and influential work that was widely accepted as the authoritative text on the subject of Master Data Management.
Since the first edition was published, Master Data Management has matured and advanced significantly. In fact, analysts, practitioners, and industry observers agree that Master Data Management is one of the fastest growing and dynamic areas of information technology. The advancements in Master Data Management include the approaches to building multidomain MDM systems, advancements in master data modeling, maturity of the approaches used to define the business case for MDM, recognition of the value and the importance of the relationships between MDM and its “sibling” Data Governance, and evolving architectural patterns in MDM.
These advancements and changes in MDM are quite profound, and have given rise to a whole new family of applications and solutions in the MDM space. In particular, MDM has evolved from its mostly individual-based Customer Data Integration to a truly Master Data Management solution for a broad variety of business domains, including individuals and institutions, complex accounts, products, financial instruments, and many others. In addition, MDM has been moving toward business-process-driven specialization, specifically toward operational and analytical MDM. As the result of these changes, the MDM practitioners and business stakeholders are looking for deep, proven thought leadership that can help them on their journey to implement MDM solutions.
Thus, the need for a major revision of the book in the form of a second edition that addresses all new developments in the MDM space has become a clear necessity. This second edition is a significant revision and expansion of the first edition of the book: It adds several new chapters and covers a number of new or updated topics, including an introduction to MDM classification dimensions, discussions on Reference Master Data Management, hierarchy management, master data modeling, entity resolution for various domains, quantitative approaches to a justifiable business case, a discussion on the need for and composition of the MDM roadmap, use of information theory to measure and manage the quality of master data, an extensive discussion on the definition, approaches, and processes of Master Data Governance, and newly formulated MDM guiding principles. These new topics are explicitly mentioned in the table of contents for easy reference.
Who Should Read This Book
The topics of Master Data Management and Data Governance have very broad applicability across all industries. Indeed, the notion of transforming businesses from account-centric to customer- or product-centric enterprises applies equally well to any industry segment that deals with customers, products, and services, including financial services, healthcare, pharmaceutical, telecommunications, retail, and so on. MDM is beneficial and often necessary for any organization or industry that needs an authoritative source of customer, prospect, partner, patient, employee, information, product information, pricing and market, or reference data in general. The same argument applies to government entities that need to have a complete and accurate view of individuals for a variety of legitimate purposes, not the least of which are law enforcement and national security.
To discuss major issues related to Master Data Management and Data Governance, this book covers a broad set of topics, including the areas of business case definition, business transformations, data management, information security, regulatory compliance, Data Governance and data quality, and business process redesign. Therefore, this book is a must-read for a variety of business and technology professionals across all industry segments and the public sector. The audience for this book includes business unit managers; business process analysts and designers; technology project managers; infrastructure and operations staff; data analysts, data stewards, data quality managers, and database administrators; application developers; corporate strategists; information security specialists; corporate risk and regulatory compliance officers; and members of the offices of the CFO, CSO, CRO, and CIO.
Due to the complexity of the MDM problem space, many Master Data Management initiatives happen to be multiyear, multimillion-dollar projects that involve large teams of employees, external consultants, system integrators, and vendor-supplied professional services organizations. All these professionals will benefit from reading this book.
Finally, the topics of MDM and Data Governance are getting “hot” and attracting significant attention from the general and specialized industry analysts. All major industry research and analyst organizations, including Gartner Group and Forrester Research, have initiated appropriate coverage or created research services focusing on Master Data Management and Data Governance. Many vendors that have or plan to have MDM solutions in their portfolios are organizing user groups and vendor-sponsored conferences. Dedicated organizations such as The MDM Institute have leaped into existence and are aggressively organizing industry-wide forums and conferences. Technical and business professionals who plan to attend these types of conferences would find this book very useful.
The Style of This Book
This book is different from research and analysts’ reports on the subject of MDM and Data Governance, in that it does not base its discussions strictly on industry-wide surveys and published statistics. Rather, the book is based on our actual professional experience, and we continue to be involved in some of the more advanced and large-scale implementations of MDM in the commercial sector, especially in financial services and pharmaceuticals. The book has been structured as a self-teaching guide that includes an introduction to the business problem domain related to MDM and Data Governance, and a discussion on the core architecture principles and concerns that should be interesting to those readers looking to learn not just the “how” but also the “why” of the MDM and Data Governance approaches. We have included a significant number of references to the key printed and online materials on Master Data Management and Data Governance. This will help the reader who wants to use this book as a single point of entry into the fields of MDM and Data Governance.
The book includes a rather detailed discussion of the issues related to information security and data protection in MDM environments. We feel very strongly that MDM designers and implementers should address these topics at the inception of every MDM initiative, regardless of whether a chosen vendor solution provides these capabilities directly or indirectly.
In addition to being an architectural primer for MDM, this book is also a practical implementation guide that can help MDM practitioners to avoid costly technical, business process, and organizational mistakes. To that end, the book includes several chapters that provide a step-by-step discussion of the practical issues and concerns relating to the defining the MDM business case and the MDM roadmap, implementation approaches of MDM projects, and an in-depth discussion on the key issues and concerns of Data Governance. And for those readers who are looking to select a vendor solution, the book offers a brief overview of the state of the art in the vendor solution marketplace for MDM.
The book concludes with a few thoughts about the trends and directions in the area of Master Data Management.
This book includes a fair amount of diagrams, figures, formulas, examples, and illustrations in an attempt to present a lot of rather complicated material in as simple a form as possible. Due to the relatively high degree of complexity of MDM and certain aspects of Data Governance, such as data quality measurements and metrics, wherever possible, the book combines theoretical and architectural discussion of a specific subject with some practical examples of how these issues could be addressed in real-life implementations.
The book is about a “hot” and very dynamic subject. All material included in the book was current at the time the book was written. We realize that as Master Data Management and Data Governance continue to evolve, and as the MDM vendor market matures, changes to the material covered in the book will be necessary. We intend to revise the book if and when significant developments in the areas of Master Data Management and Data Governance warrant changes.
What This Book Includes
The book contains five parts and two appendixes. Each chapter of the book contains a list of references specific to the content of the chapter.
Part I of the book defines the business imperative, drivers, and benefits of Master Data Management as well as the need for and role of Data Governance. It also discusses the challenges and risks associated with enterprise business model transformation inspired and enabled by MDM.
Part II of the book continues the MDM discussion by taking a closer look at the architecture and design concerns of MDM solutions, with a strong emphasis on the design issues of MDM Data Hub platforms and master data modeling. Part II offers an architecture background that introduces readers to several key concepts, including the enterprise architecture framework and service-oriented architecture.
Part III deals with major regulations, compliance requirements, and risks associated with implementing MDM solutions. This part offers a detailed discussion on general information security goals, techniques, and approaches. It concentrates on several important themes, including general data protection, intellectual property, and content protection using Enterprise Rights Management. This part of the book also provides an in-depth look at authentication, authorization, access control, policies, entitlements, and data visibility issues that have to be addressed in practically every MDM implementation.
Part IV of the book discusses a broad set of issues, concerns, and practical approaches to implement an MDM solution. It begins with the techniques and methods of defining an MDM business case and the roadmap, and specifically talks about how to start a successful MDM project. Part IV also provides an in-depth discussion on the implementation aspects of master entity resolution and processes designed to discover and leverage the totality of entity relationships with other entities and the enterprise. This part of the book also discusses Data Governance, with the primary focus on Master Data Governance processes and metrics, and introduces the reader to the advanced approaches to data quality metrics based on key concepts of the Information Theory. Part IV includes topics on the MDM implementation concerns related to data synchronization, data quality, and data management standards.
Part V concludes the book with a brief discussion of the market landscape and an overview of the relevant vendor solutions available on the market at the time of this writing. It also provides a brief discussion on future trends and directions for Master Data Management.
The appendixes include a list of common abbreviations and a glossary of key terms.