TREND 4
Big Data and Augmented Analytics
In very simple terms, “big data” refers to the exponential explosion in the amount of data being generated in this increasingly digital age, while “augmented analytics” refers to the ability to automatically work with and generate insights from data. Let’s start with the data itself, because data is critical to so many of the trends in this book, including artificial intelligence (AI, Trend 1), the Internet of Things (IoT, Trend 2), natural language processing (Trend 10), and facial recognition (Trend 12). Without data, the massive leaps we’ve seen in these trends – and many other technology trends – wouldn’t be possible. At the heart of big data is the idea that the more data you have, the easier it is to gain new insights, and even predict what will happen in the future. By analyzing masses of data, it’s possible to spot patterns and relationships that were previously unknown. And when you can understand the relationships between data points, you can better predict future outcomes, and make smarter decisions on what to do next. It’s no exaggeration, then, to say that big data brings incredible opportunities to understand and change our world for the better. But what is it that makes data, well, “big”? After all, data isn’t exactly a new thing. What’s new is the unprecedented digitization of our lives, where almost everything we do leaves a digital footprint. This is largely thanks to the rise of computers, smart phones, the internet, the IoT, sensors, and so on. Think of everyday activities like shopping online, reading the news in an app, paying for the morning coffee by card, messaging friends and family, taking and sharing photos, watching the latest show on Netflix, asking Siri a question, swiping right on a potential love match…we’re all generating data all the time. The sheer volume of data that we’re creating, and the rate at which that volume is accelerating, is so vast that 90% of the data available in the world today was generated in the last two years.1 What’s more, every two years we’re doubling the amount of data we have available.2 How much data are we talking about? Well, we’re no longer talking about data in terms of gigabytes. These days, we’re talking about terabytes (just over 1,000 gigabytes), petabytes (a little over 1,000 terabytes), exabytes (roughly 1,000 petabytes), and zettabytes (approximately 1,000 exabytes). According to market intelligence company IDC, the amount of data in the world could grow from 33 zettabytes in 2018 to 175 zettabytes in 2025.3 To put that in perspective, if you stored 175 zettabytes on DVDs, you’d have a stack of DVDs so big it could encircle Earth 222 times! And the amount of data we’re generating is likely to accelerate further. In other words, big data is only going to get bigger. Our ever-increasing digital footprint has also given rise to another interesting aspect of big data: the fact that there are many new types of data that can be analyzed. We’re no longer just working with numbers in spreadsheets, or entries in a database; today, “data” includes photo data, video data, conversation data (i.e. asking Alexa to play a certain song), activity data (such as browsing online or swiping left or right), and text data (like social media updates). Increasingly, the data we have to work with is unstructured, which means it can’t be easily classified into neat rows and columns, like in a spreadsheet. This unstructured data is more challenging to analyze – which is a major problem when you consider that data is pretty much useless unless we can find a way to extract meaningful insights from it. This is where the augmented analytics part comes in. Handling masses of data can be an expensive, time-consuming, and highly specialized task. In other words, there are some serious barriers between the data itself and the ability to turn that data into actionable insights. Augmented analytics is about breaking down those barriers and making it easier to generate amazing insights from data. In a nutshell, augmented analytics involves using AI and machine learning (see Trend 1) to automate analytics processes, including gathering data from raw data sources, preparing and cleaning that data, building unbiased analytics models, and generating and communicating insights to those who need them. What’s really exciting about this is it makes it easier for people to interact with data and extract the information they need, without the involvement of data specialists. So, in theory, with an augmented analytics tool, a non-tech expert could simply ask the system a question – like “Which of our employees are most likely to leave in the next 12 months?” – and the system would automatically generate a response. Gartner predicts that by the end of this year, 40% of data science tasks will be automated,4 meaning augmented analytics is on track to become the leading analytics method of the future. As the trend really takes off, it’s likely we’ll see many more specialized augmented analytics apps and tools designed for specific industries in the future. This is good news for businesses, since augmented analytics provides a way for organizations of all shapes and sizes to handle the vast amounts of complex data they’re inundated with and give people in the organization easy access to analytics and insights from data. This wide access to data and insights is known as data democratization. Now might be a good time to mention that, personally, I prefer the term “data” to “big data.” The “big” implies it’s the sheer volume of data that’s really important. But equally important, if not more, is what we do with data. And, boy, can we do impressive things with data these days. Data, coupled with other trends like AI, is transforming our world – it’s helping to making our homes smarter (see IoT, Trend 2), physically augment humans (see Trend 3), and build the smart cities of the future (see Trend 5), and that’s just for starters. Data is also changing the way we do business. Let’s look at the main ways in which businesses can leverage data (big or otherwise) to their advantage. Making better business decisions is absolutely one of the top priorities for most of the clients I work with. From how to hire the right people and target the right customers, to how to boost revenue, success means making the best decisions for your business. With data, you can better understand what’s happening in the business and the wider market and predict what might happen in the future – information that’s critical to good decision-making. Therefore, across every business function, data can and should be used to make smarter business decisions. In one very simple example, US restaurant chain Arby’s discovered that its renovated restaurants made more money than its unrenovated restaurants. Based on this knowledge, the company decided to carry out five times more restaurant remodels over the course of a year.5 The better you understand your customers, the better you can serve them. Sales and marketing activity is often based on past sales history – effectively, which customers previously bought which products or services. But, thanks to big data and augmented analytics, this activity is increasingly becoming more predictive. In other words, companies are now confidently and accurately anticipating what customers will want in the future. Netflix predicting what you might want to watch next is one simple example of this. In another example, German retail company Otto discovered that customers are less likely to return items when they arrive within two days, and when they receive all their items at once, rather than in multiple shipments. Hardly earth shattering – keeping goods in stock and shipping efficiently makes good sense. However, Otto is like Amazon in that it sells products from many, many brands, which means stocking and shipping products all at once is a major challenge. So Otto analyzed the data from 3 billion past transactions, plus factors like weather data, to build a model that could predict what customers would want to buy in the next 30 days. Not only could the system do this, it could do so with 90% accuracy.6 Now, the company can order the right products ahead of time and, as a result, product returns have been reduced by over 2 million items a year. When you know more about your customers, you can give them exactly what they want: smarter products and services that respond intelligently to their needs. This has given rise to a wealth of smart products, such as smart speakers, smart watches, even smart lawnmowers. For plenty of examples of smart products and services in action, circle back to Trends 2 (IoT) and 3 (wearables), or turn to Trend 18 (digital platforms). Every business process and every aspect of business operations can be streamlined and enhanced, thanks to big data. Optimizing pricing, accurately forecasting demand, reducing employee turnover, boosting productivity, strengthening the supply chain – across all areas of the business, it’s easier than ever to make improvements, generate efficiencies, save money, automate processes, and more. Remember the Otto example of predicting demand in order to improve stock ordering? Thanks to data (and more than a bit of AI, see Trend 1), this impressive process happens automatically. The company’s system orders around 200,000 products a month without human intervention. In another example, Bank of America worked with Humanyze (formerly Sociometric Solutions) to implement smart employee name badges, fitted with sensors that can detect social dynamics in the workplace. From the data generated, the bank noticed that top-performing employees at call centers were those who took breaks together. As a result, it instituted new group break policies and performance improved 23%.7 You can find more examples of enhanced and automated business processes in Trend 13 (robots and cobots). Optimizing business processes, making better business decisions, and so on, will no doubt have a positive impact on the bottom line. But the link between data and the bottom line can be much more explicit, meaning data can be monetized to create new revenue streams. This may encompass bringing new data-driven products to market (such as the smart products outlined in Trends 2 and 3), or it could mean actively selling data through optimized services (such as Google’s data-driven advertising offering). Data can even increase the value of a company; at the time of writing, the world’s top three most valuable brands were Google, Apple, and Amazon – each of them data-driven businesses.8 You might think that some of the most obvious challenges around big data are the technology, infrastructure, and skills challenges. To put it another way, do you have to have the budget, infrastructure, and know-how of, say, Google or Amazon to benefit from big data? Thanks to augmented analytics and big-data-as-a-service (BDaaS), the answer is no. I’ve covered augmented analytics earlier in the chapter, so let’s briefly look at BDaaS. The term refers to the delivery of big data tools and technology – and potentially even data itself – through software-as-a-service platforms. These services allow companies to access big data tools without the need for expensive infrastructure investments (see also AI-as-a-service in Trend 1), thereby helping to make big data accessible to even small businesses. This also helps to overcome the massive skills gap in big data. Essentially, there aren’t enough data scientists to go around; the McKinsey Global Institute predicts that, by 2024, there’ll be a shortage of approximately 250,000 data scientists – and that’s just in the US.9 As analytics tools advance, my hope is that technology, infrastructure, and skills will become less daunting barriers to working with data. But that doesn’t mean there won’t be other barriers to contend with. I believe two of the biggest challenges around big data are data security and privacy. Ever-growing volumes of data – and the fact that data is becoming more of a critical business asset – brings huge challenges in terms of protecting that data. It’s therefore vital that organizations take steps to protect their data from attack, particularly when it comes to personal data (like customer or employee data). Advances like the IoT add an extra dimension to the threat, since many connected devices are totally unsecured, thereby providing a potential way in for hackers. (One study has found that 82% of organizations believe that unsecured IoT devices will cause a data breach in the next few years.10) But your employees are another significant threat to consider. So as well as having a robust data security policy in place, it’s vital you raise awareness of the potential threats and educate your teams on the need to protect data. Security is closely linked to data privacy, since so much of the data that organizations are working with contains personally identifiable information. Regulators are, to some extent, still playing catchup when it comes to data privacy laws, but that will change. Recent GDPR guidance in Europe is designed to promote the safe and ethical handling of personal data – and give individuals a greater say in how organizations use their data. Therefore, it’s not enough to protect your data securely – you also need to take an ethical approach to collecting and using that data. This means being completely transparent, making customers and other stakeholders aware of what data you’re gathering and why, and giving them the chance to opt out where possible. Those companies who don’t comply with tightening regulation, or who play fast and loose with people’s data, risk serious financial and reputational blowback in the future. Despite the challenges, most experts, myself included, believe the benefits of big data are huge. Data can bring enormous value to your organization, providing you prepare properly. For me, this means: Let’s look at each step in turn. The more data literate your organization is, the better your results will be. It’s as simple as that. But that doesn’t mean everyone has to be a data scientist. It simply means that everyone right across the business must be comfortable with data: talking about data, using data, thinking critically about data, pulling meaningful insights from data, and ultimately acting on what data tells them. Data literacy is about everyone putting data to use, essentially. Raising data literacy across the business is a case of establishing your current levels of data literacy, communicating why data literacy is important, identifying data advocates who can sing the praises of data, ensuring access to data, and educating those across the business on how to get the most out of data. It’s also vital you have a data strategy in place. A data strategy helps you remain focused on the data that matters most to your business – as opposed to collecting data on anything and everything, which is rather an expensive way to go about it! With so much data available these days, the trick is to focus on finding the exact, specific pieces of data that will best benefit your organization. A data strategy helps you do just that. With a robust data strategy you can set out how you want to use data in practice, clarify your top data priorities, and chart a clear course to achieving your goals. Your data strategy must be unique to your business, but, broadly speaking, I’d expect a good data strategy to cover the following points:The One-Sentence Definition
What Is Big Data and Augmented Analytics?
How Is Big Data and Augmented Analytics Used in Practice?
Informing Business Decisions
Better Understanding Customers and Trends
Delivering More Intelligent Products and Services
Improving Internal Operations
Creating Additional Revenue
Key Challenges
How to Prepare for This Trend
Improving Data Literacy Across the Organization
Creating a Data Strategy
Notes