1. Costs to Store and Process Information Have Reduced

The cost of storing information has significantly dropped: $600 will buy a disk drive that can store the entire world's music.⁴ In 1990, it cost $11,000 to buy 1 GB of disk space; today, 1 GB disk space costs less than 1 cent.⁵ Data storage and data processing have grown tremendously. The smartphones we all carry have more storage and computer power than a mainframe did in the 1980s. And large amounts of complex and unorganized data, such as audio, video, graphic, e-mail, chat, and text data, are widely available across infinite networks of computing environments at very low cost. In parallel, the cost of censors, photovoltaic cells, and all kinds of high-technology data-driven products has also decreased, enabling more people to use them—and generating more data to store and process.

2. Interactive Devices and Censors Have Increased

There are countless interactive devices such as smartphones, tablets sensors, surveillance cameras, and wearable devices that can capture, store, and send information, including everything we see, touch, hear, feel, and smell. This information no longer resides on an offline siloed desktop. It is instantly sent online via Internet Protocol (IP) and shared, transformed, and enriched by millions of users. Data is pumping in from a growing universe of connected sensors and machines.

3. Data Analytics Infrastructures and Software Have Increased

Over the past few years, there has been a rapid increase of data analytics tools and software that process and analyze data with unprecedented speed. Technologies such as artificial intelligence, machine learning, and deep learning are advancing more quickly than anyone anticipated.

The recent enthusiasm was propelled by the emergence of technology that makes taming Big Data possible, including Hadoop, the highly used framework for data file systems processing. Nowadays data scientists turn to Spark (and other tools) as a more efficient data processing framework. Open source cloud computing and data visualization capabilities are commonplace. A self-driving car, for example, can collect nearly 1 GB of data every second—it needs that granularity to find patterns to make reliable decisions. It also needs the computing power to make those decisions in a timely manner. As far as the computing power goes, we have moved from the central processing unit (CPU) to the graphics processing unit (GPU), and more recently, Google has developed the Tensor Processing Unit (TPU). During Google's 2017 I/O Conference, Google chief executive officer (CEO) Sundar Pichai discussed how Google is rethinking its computational architecture and building artificial intelligence and machine learning first data centers. He introduced Google's TPU, which supports Google's new artificial intelligence computational architecture, because, according to Google, TPUs are not only 15 to 30 times faster than CPUs and GPUs but also 30 to 80 percent more powerful. TPUs are now used across all Google products. In 2016, the TPU powered DeepMind AlphaGo during its historic 4–1 victory against Lee Sedol, the Go grandmaster. Machines can diagnose diseases, navigate roads, and understand and respond in natural language.

4. User-Friendly and Invisible Data Analytics Tools Have Emerged

There is an emergence of easy-to-use software that enables the processing and creation of actionable business value from the raw data. These user-friendly tools do not necessarily require technical background from the end user. Data analytics is becoming pervasive because the volume of data generated by embedded systems increases, and vast pools of structured and unstructured data inside and outside the enterprise are now being analyzed. According to David Cearley, vice president and Gartner fellow, “Every app now needs to be an analytics app.”⁶ Organizations need to manage how best to filter the huge amounts of data coming from the Internet of Things, social media, and wearable devices, and then deliver exactly the right information to the right person at the right time. He continues, “Analytics will become deeply, but invisibly embedded everywhere.” Big Data is the enabler and catalyst for this trend, but the focus needs to shift from Big Data to Big Analytics, because the value is in the answers, not the data.

5. Data Analytics Is Becoming Mainstream, and It Means a Lot to Our Economy and World

There has been a significant proliferation of analytics benefits from a variety of sectors and industries. Cases demonstrating how organizations and businesses are winning with analytics are more and more available and accessible to everyone. Marketing has been extremely powerful in sharing those success stories everywhere, from business magazines to traditional media. Depending on their level of analytical maturity and their most pressing business objectives, companies and organizations across different industries and business functions have embraced analytics to create business value and develop competitive differentiators. Following are examples where advanced analytics are impacting today's world:

Politics

• Politicians have been leveraging data analytics to be more successful at winning elections.
• Politicians use social media analytics to determine where they must campaign the hardest to win states, provinces, and cities in the next election.
• During the 2012 presidential election, President Obama's team tactically leveraged Big Data analytics to target fundraising messaging to voters by location, age, group, occupation, and interest. For instance, according to their models, women from 35 to 49 years old were more likely to respond to messages such as “Dinner with the president and George Clooney.” Data-driven targeted fund activities helped the Obama team raise more than $1 billion (an all-time record in any presidential campaign in modern history). The team optimized their campaign ad spend and, more importantly, targeted undecided voters in swing states to register and persuaded them to vote for Obama.
• Following the unexpected results of the 2016 election between Clinton and Trump, analysts attributed the discrepancy to five factors that traditional polling data could not capture: (1) A nontraditional candidate (no previous political history that resonated with traditional nonvoters); (2) An unprecedented investigation by the FBI during the final few days of the election; (3) The alleged influence of a foreign state; (4) The miscalculation in the distribution of voters (turnout) among different demographic groups; and (5) The impact of the “digital disruption of media,” which allowed so many new channels of conflicting information to spring up. The 2016 election is likely going to change how political analytics is conducted in the future.
• It is important to note that the Trump campaign also leveraged Big Data insights from a U.K.-based Big Data analytics company called Cambridge Analytica, the same company that helped the Brexit leaders to win. Cambridge Analytica crunched data for Donald Trump, building psychological profiles of American voters. They leveraged more than 5,000 pieces of data about every adult American from sources such as personality tests on social media, voting history, personality type, shopping history, and television and cable viewing history. Insights from this analysis helped the Trump campaign tailor its message and strategy in battleground states and win.

Sport & Entertainment

• Forward-looking companies use machine learning to analyze video content and identify key information aspects from it. Machine-learning technique is also helping them to uncover actionable insights from video content. These insights are helping their digital marketers to align their advertisements with the right videos and drive customer engagement.
• Companies such as New York-based Dataminr, which provides a market monitoring platform for Brand, help businesses monitor and track their brand and immediately take appropriate actions. For instance, the platform can identify tweets related to stock trading and monitor discussion threads and communications to identify relevance and urgency. News organizations such as CNN and BBC use similar unstructured data analysis (UDA) technology to quickly identify news stories and have reporters deployed or engaged to cover them in a timely manner.
• Video analytics and sensor data of tennis, baseball, and football games are used to improve the performances of players and teams. You can now buy a baseball with more than 200 sensors in it that will give you detailed feedback on how to improve your game.
• Augmented reality technology is used to overlay supplemental information, such as scores and other statistics, over a live video field in sporting events to enhance the viewers' experience.
• The Oakland Athletics (an American professional baseball team based in Oakland, California) pioneered analytics in sports, and now all teams use analytics. Today information and statistics about players are a prerequisite for recruitment.
• Spotify uses unstructured data analytics to provide its popular Discover Weekly playlists, which are personalized to our musical tastes. The company leverages collaborative filtering, convolutional neural networks, and natural language processing to scan music blogs, build microgenres, analyze the contents of playlists, and detect and eliminate outliers to find songs that fit our profile, but that we haven't yet listened to.
• Artists such as Lady Gaga are using data about our listening preferences and sequences to determine the most popular playlists for live performances.

Business

• Amazon uses analytics to recommend what books to buy; more than 35 percent of their sales are generated from these recommendations: “People who bought this book also bought …”
• Netflix leverages analytics to recommend movies you are more likely to watch. More than 70 percent of Netflix movie choices arise from its online recommendations.
• Pinterest leverages UDA to provide a personalized experience on its home feed by surfacing content each user would be more interested in seeing.
• Companies use sentiment analysis from Facebook and Twitter posts to determine and predict sales volume and brand equity.
• Target (a large retailer) predicts when a pregnant woman is due based on products she purchases. The company simply combines her loyalty card data with social media information, hence detecting and leveraging changing buying patterns. As a result, the company can target pregnant women with promotions for baby-related products. The company increased revenue 15 to 20 percent by targeting direct mail with product choice models.
• Google's self-driving car analyzes a gigantic amount of data from sensors and cameras in real time to stay safe on the road.
• The global positioning system (GPS) in our phones provides location and speed information for live traffic updates. Location-based data has generated billions of dollars for companies and even more value for users.

Healthcare

• Pediatric hospitals apply data analytics to livestreamed heartbeats to identify patterns. Based on the analysis, the system can now detect infections 24 hours before the baby would normally begin to show any symptoms, which enables early intervention and treatment.
• After winning the TV show Jeopardy, Watson the IBM supercomputer now assists physicians to better diagnose and cure aggressive diseases such as cancers. Watson is being enlisted to help doctors predict cancer in patients. Watson has greatly speeded diagnosis of acute myeloid leukemia, a type of blood cancer, as well as glioblastoma multiform, an aggressive brain cancer.
• DNA sequencing: It took 10 years to sequence the DNA of one person, at a cost of more than $100 million. Today, DNA sequencing is done very quickly for less than $99. This availability leads to personalized medicine and drug prescriptions. Genomic precision medicine is becoming a game changer.
• Natera can predict Trisomy 21 disease without the risk of miscarriage from an amniocentesis test just by testing blood from the mother-to-be. The results of that blood test have a 99 percent accuracy rate, similar to those from the traditional amniocentesis test.

Government, Security, and Police

• The FBI combines data from social media, closed-circuit television (CCTV) cameras, phone calls, and texts to track down criminals and predict terrorist attacks.
• JPMorgan Chase & Co. invested $837 million in people analytics to anticipate rogue employee behavior that has cost the bank more $36 billion since the end of the 2008 financial crisis.

Human Capital Management

• HR analytics companies are now using people analytics to optimize talent management. Big Data analytics help to attract, acquire, develop, and retain talent. Google, Cisco, Microsoft, GE, Xerox, Bloomberg, Deloitte, and Pfizer have been accumulating success stories and benefits from people analytics to optimize their talent equation.

6. Major Leading Tech Companies Have Pioneered the Data Economy

Major tech companies such as Google, Facebook, eBay, Amazon, LinkedIn, and Yahoo began to monetize their data. As a result, their entire businesses reside on their ability to harness the data at their disposal. Most recently, some of these companies shared Big Data analytics algorithms they pioneered with the public: open source, Hadoop, and Spark. As a result, entire businesses that stem from data analytics and raw data have been developed. A new digital economy was born, in which data is the currency and analytics sets the trading rules. Data analytics has become pervasive and omnipresent. Social media, cloud, mobile, and Big Data provide disruptive ways to capture, process, analyze, and communicate data analytics findings and recommendations.

7. Big Data Analytics Has Become a Big Market Opportunity

Big Data analytics has a big market opportunity: The research firm IDC forecast that the market opportunity for Big Data and business analytics software, hardware, and services will grow by 50 percent from 2015 to 2019. This means that the market size will reach 187 billion by 2019. Services accounting for the largest portion of the revenue with manufacturing and banking poised to lead the spend.⁷

8. The Number of Data Science University Programs and MOOCs Has Intensified

Before Big Data, there were no university data science programs or degrees. The need to harness the complex volume of so-called “messy” data such as images, text, audio, video, chat, graphics, and pictures forced universities to adapt. The article “Big Data: The Next Frontier for Innovation, Competition, and Productivity” from McKinsey&Company⁸ has played a key role in raising the awareness of the eminent data scientist labor shortage. According to the article, in 2018, there will be 140,000 to 190,000 data scientists and 1.5 million data-savvy managers. This headline has pulled the alarm signal, creating awareness among organizations eager to join the new data economy.

A variety of business schools and universities now offer data mining and data science programs and degrees. In the United States alone, there are more than 20 universities offering data science programs,⁹ as well as a panoply of executive programs covering data analytics either locally from universities' classrooms or via the Internet and videos. This has led to the multiplication of massive open online courses (MOOCs). MOOC distance education was first introduced in 2008 and emerged as a popular mode of learning in 2012,¹⁰ and offers a lot of data science modules and programs.

The top of the list includes MOOCs in Exhibit 1.3, organized based upon the date of inception.¹¹

Other MOOCs include Future Learn Open Education Europe and The Open University Classroom.

At Harvard University, more people signed on in one year to the online courses than had graduated from the university in its 377 years of existence.

In addition, communities' and analytics groups' niche sites have been proliferating. The ever-growing data volume has put more pressure on companies to hire data scientists. This job occupation has been a hard-to-fill position, a situation that is not poised to change. How did we get here? What is the timeline that led to the Analytics Age?

Early Adopters: Insurance and Finance

To best navigate today's globally connected competitive marketplace, which has been propelled by the explosion of digital information, companies have been embracing analytics at different paces to derive strategic insights.

From an industry perspective, some early adopters pioneered the integration of analytics into their business processes, then the late adopters followed, and finally the laggards trailed far behind. While most businesses have heard about the benefits of analytics, some struggle to move beyond basic reporting. However, the majority of leaders are requesting their analytics team to deliver on Big Data promises, monetizing their data by creating new data products and services to truly compete on analytics—as one CEO told me, “to rethink our analytics journey.”

In Win With Advanced Business Analytics,¹³ I described how the insurance and financial service industries have been pioneers in collecting, managing, and leveraging predictive analytics to create actionable insight from their data. Given the mandatory reporting environment of these fields, not to mention the direct correlation between accurate data and revenue, these industries first began using analytics in the 1800s to price life insurance and underwrite marine insurance. They have since continued innovating, undergoing many waves of improvement, including the use of neural networks¹⁴ to optimize premiums, the introduction of credit scores in evaluating a customer's financial position, and the use of various behavioral, third-party, and social media datasets to supplement their forecasts and future predictions.

Over the years, the success stories have been numerous, and financial institutions that are not currently using predictive analytics in product pricing and underwriting are seen as obsolete and doomed to failure. As a result, these companies are now developing new applications and expanding their analytics into different business units. For instance, safety analytics is the science of using information about a company's current state and other geodemographic data to prevent accidents from happening and avoid the associated insurance costs by creating a safer workplace. Similarly, claims analytics is a discipline in which claim severity (amount of loss) is scrutinized for drivers of high-dollar claims by injury type. It aims at understanding and predicting probable losses (damages, litigation costs, etc.) based on frequency, severity, and other characteristics of the claims. Claims analysis is also used for early identification of litigious claims, efficient claims settlement, adequate loss reserving, and claims predicting, among other activities.

Fraud analytics is also a hot topic. It is estimated that fraud claims cost the insurance industry between $5 and $7 billion every year. Complex predictive models designed to find intricate patterns of unusual behavior to uncover fraudulent schemes are being used by various insurance organizations and state agencies throughout the United States. Models that use machine-learning techniques to adapt to new schemes are being built and put into production to pinpoint which incoming claims are likely to be fraudulent, prompting an investigation.

Today, major players in the data economy are monetizing their data to create new products and services, making their data the most powerful currency in this digital world.

The evolution of analytics can be summarized in three major eras:

1. Analytics before Big Data

   This analytics era was characterized by relational databases, data warehouses (DWHs), statistical analyses performed on samples, and numeric-based analytics. The advent of computers and the growth of personal computers have helped to increase adoption.

2. Analytics in the Big Data era

   The advent of Internet (World Wide Web) and the birth of Silicon Valley's online business behemoths such as Google, LinkedIn, Facebook, Amazon, and Yahoo! pioneered the digital economy and promoted the analysis of new types of complex and messy user-generated data. This enabled the development of infrastructures and tools powered by Hadoop and Spark cloud computing mobile, with computing power that made data processing, analysis, and distribution more efficient and effective than ever before. This analytics era is also defined by the emergence of computing power that enabled the realization of artificial intelligence (AI), machine learning, representation learning, and deep learning. The Big Data era fed those algorithms with endless data to learn by experience and to understand the world in terms of hierarchy of concepts. It also provided new solutions to previously unsolved problems, such as computer recognition of images and video.

3. Post–Big Data analytics era: The Analytics of Things and Apps, cognitive analytics, and the Augmented Age

   As demonstrated in Exhibit 1.4, we are just at the beginning of the Analytics of Things era, and disruptive changes are coming within the next decade. Billions of connected devices and intelligent machines interacting with human beings will generate a tsunami of data that organizations and companies will have to analyze to play in the data economy. The post–Big Data era will also be defined by cognitive analytics and the real-time integration of AI, machine learning, representation learning, and deep learning algorithms to address business and societal challenges coming from billions of connected objects. More importantly, these algorithms will build a dynamic intelligence based on the interaction between human censors and computers, using natural language to communicate with computers and devices for the most part.

   We are now heading into the augmented age, where Big Data, Internet of Things, intelligent apps, and AI will provide additional insights at our fingertips. Think about what we do today when somebody asks you a question and you don't know the answer. Your smartphones automatically come to your rescue, enabling you to Google the information needed. As Maurice Conti, chief innovation officer at Telefónica, eloquently put it:

Where does your organization sit on this scale?