Chapter 1 Introduction
I have worked with databases since they were first commercially available on a large scale. Along the road we tried a number of approaches, some of which were not exactly right the first time (seen in hindsight). For example: data modeling left the business realm to become an engineering activity. The real world ended up consisting of improper table designs, surrogate (internal) keys, and other pragmatic solutions to real problems.
Then I discovered concept mapping: a form of concept model which today has been successfully adopted by the business rules community. I also followed the graph world quite closely, but it was the advent of the “property graph” style of graphing that triggered me to write this book. My intent is to set a new standard for visualization of data models based on the property graph approach.
One of the major differences between relational and non-relational modeling is the absence of schemas (in the form of pre-defined metadata) existing alongside the data. Once upon a time, names and data types were determined once and rarely changed over time. The new world, with its general lack of schemas or self-describing data, changes this. Despite the fact that “No” in “NoSQL” stands for “not only,” most people associate it with “Not” SQL. In any case, the absence of schemas does not imply the absence of business requirements or the modeling of these requirements—a major theme of this book. We also will focus on the business requirements of good data modeling in schema-less contexts. Rather than modeling data architecture on complex mathematics, I believe we should focus on the psychology of the end user. If we do so, then engineering could be replaced with relevant business processes. In short, to achieve more logical and efficient results, we need to return to data modeling’s roots.
This book is written for a few overlapping audiences:
- The experienced relational data modeler/developer who thinks, “There must be a better way of doing this.” (And there is!)
- The people in analytics and data science who must prepare data for business and vice-versa, especially those in big data and advanced analytics.
- The developers who must develop data models as they go, whether they’re in relational or non-relational tables, and whether they’re on a traditional or new (like Graph databases) platform.
Most of the book is forward-looking and should appeal to many users, regardless of their level of previous engagement with traditional data modeling.
This book proposes a new approach to data modeling—one that “turns the inside out.” For years, relational modeling and normalization have been the name of the game. However, there’s something upside-down in that approach, as we will see in this book.
To me, data analysis (modeling) is much like exploration, an expedition into the unknown. The data modeler wanders around searching for structure and content. The modeler combines perception and cognitive skills with intuition (which is a psychological phenomenon), to determine how well the landscape of business semantics is mapped. Mapping is what we do.
Two compelling events have made a paradigm shift in data modeling possible, and also necessary:
- Advances in applied cognitive psychology have addressed the needs for proper contextual framework and for better communication in all settings, including data modeling.
- Non-relational technologies (big data and NoSQL) have exploded in popularity.
Chapter 2 covers the background of data modeling. We will consider the roles of data modeling in big data and NoSQL solutions. We also explore relational and other widespread paradigms. We’ll also highlight some concepts of human cognitive systems and our ways of dealing with our surroundings, including cognition and visualization.
Chapter 3 describes our understanding of the true requirements of data modeling. This chapter will outline the end user’s actual spatial and conceptual needs. We’ll end the chapter with a list of the real requirements of a data modeling solution.
Chapter 4 describes a data modeling solution that fulfills the requirements collected in chapter 3. We’ll describe how property graphs can be used with concept maps to collect meaning and structure. Solution modeling for specific project data models is finally followed by transformation, optimization, and deployment for a number of different physical data store targets.
Chapter 5 walks through a few detailed examples. Chapter 6 concludes and helps you pack a “mental backpack” for your journey into the “data modeling zone,” preparing you to explore business semantics.
The emphasis of the book is on solution data modeling (the “logical” level). Many of the new technologies are more or less proprietary, meaning that on the physical level we can only provide general directions in a book like this.
Many of the diagrams for this book have been made in Omnigraffle (Mac OS X) and a stencil for property graphs that I developed.
Most of the concept maps have been made in CmapTools. The publisher of the software is the Florida Institute of Human and Machine Cognition (IHMC). The tool, which is free, can be downloaded from http://cmap.ihmc.us. It is easy to use and is ideally suited for brainstorming sessions.
Many thanks to Ilya Katsov, blogger at The Highly Scalable Blog (http://bit.ly/2adlkAS), who gave us permission to reuse his excellent “Stop following me….” illustration of NoSQL and SQL.
Marc Rettig created a complimentary poster on normalization for Database Programming & Design Magazine, published by Miller Freeman in 1989. The intellectual rights belong to Marc Rettig, now of Fit Associates. It is included here with his kind permission.
The EU-Rent Car Rental fictitious business is used in the Object Management Group (OMG) documentation for the Semantics for Business Rules and Vocabulary standard (SBVR), (http://bit.ly/2abzjp7). The EU-Rent case study was developed by Model Systems, Ltd., along with several other organizations of the Business Rules Group (www.businessrulegroup.com). The body of descriptions and examples may be freely used, providing its source is clearly acknowledged.
Some concept maps and text refer to my first book: Design Thinking Business Analysis - Business Concept Mapping Applied, Thomas Frisendal, © Springer, 2012, [5]. That book introduces business concept modeling, which is mentioned here as a recommended first phase of a data modeling flow.
My former colleague, Gopi Nathan, contributed part of a classic Bachman Data Structure Diagram originating in the late seventies. Thanks, Gopi.
Steve Hoberman, the publisher at Technics Publications and fellow data modeler, was kind enough to contribute some illustrations from his book [9] Data Modeling for MongoDB.