Introduction – Speaking of Data (Big, Little, Dark …) in Anticipation of the Impending Tsunami
As this book was being completed (2013, Q1), the Gartner Group estimated that ‘big data’ was 2–5 years away from peak hype. Their widely watched, reported, and adopted forecast indicated that buzz around the concept of “big data” would be increasing the near future. If you have not heard the hype – you will.
We are not going to bore you by re-creating a bunch of statistics showing that data creation, data processing, and data understanding requirements are increasing at increasing rates. Just consider the widely-cited fact that AT&T mobile traffic increased 8,000% between the years 2007–2010 or Eric Schmidt’s often-repeated comment that, “Every 2 days we create as much information as we did up until 2003.” It is sufficient to say that the number of things that produces data is rapidly growing. For example, 1 billion smart phones were in use in 2012. It should take only three more years to double to 2 billion (Brown 2012). Increasingly, individuals make use of the thing’s (smart phone/tablet/sensor, etc.) data-producing capabilities. It all adds up to a number of challenges that are referred to as “big data.”1
While it has so far proven impossible to define data as ‘big,’ it is possible to objectively categorize big data techniques, which are what have really been responsible for the so-called big data successes2 (Figure 1.1).
When considering data issues, a minority of decision-makers possess the required KSAs/background – they are not data-knowledgeable. Poor understanding of how to successfully leverage data prevents an organization from obtaining a data advantage by making poor decisions about data, its management, and how it is shared.
We have crystalized the reasons that have prevented most organizations from obtaining a data advantage. We advocate a CDO, responsible solely for organizational data assets. Through the CDO’s perspective, the organization will understand what size and shape data challenge it is facing. Only then will the organization be able to make decisions about data using the appropriate context and understanding. We’ll describe how the CDO should get started on this daunting task.
This book’s contents can be visualized with the aid of Figure 1.2 – a word cloud of the text.3 Sequentially, the next chapter describes the Chief Information Officer (CIO) function as having too broad a technology management to focus enough resources on data issues. The third chapter details ‘exploiting a data advantage’ and how the CDO function provides much-needed relief for broadly focused CIOs, who are ill-prepared to devote requisite time and attention. Chapter 4 presents measurements showing how organizations are not ready to leverage their data assets and what must be done to correct this. The fifth chapter examines the causes for this poor DM practice maturity and suggests remediation. It also argues that the remedy starts with a properly supported CDO as a business capability reporting outside of IT. Chapter 6 sums up and provides some implementation guidance. Organizations can soon be in a position to take advantage of the promise of big data techniques as well as more mundane DM benefits such as more effective/efficient operations.
1Truth be told, there cannot be any such thing as big data. First – all other data would then have to be labeled “small” or “medium.” We are now left with the challenge of using vocabulary such as “dark” to refer to legacy data. Second, because no one has come up with an objective definition for “big data” – it is impossible to separate out any specific causes and effects to study. Any measurements, claims of success, quantifications, etc. attributable to big data must be viewed with appropriate skepticism.
2Appropriate processing trade-offs such as eliminating the von Neumann processing bottleneck, taking advantage of continuous availability; or eliminating human limitations – can be objectively evaluated along with other advances for potential organizational utility – as all proposed capabilities should be evaluated. If your organization suspects that it might be able to benefit from the application of big data techniques, or has already embarked on this journey, this material is a must read!
3That the word cloud isn’t on this book’s cover is evidence of still significant technology understanding gaps in the industry.