Using data to improve operational processes

Businesses big and small are beginning to use data to refine their processes, reduce waste, increase efficiencies and increase revenue. Retail and sales companies are seeking to collect as much information about their customers’ lives as possible so as to fulfil their changing needs more effectively. Manufacturing companies are seeking to streamline operations. Equipment calibration settings can be recorded and refined, and product storage environments monitored to determine the optimum conditions that lead to minimum spoilage and waste.

Of course, in business, once a product has been grown or manufactured, it needs to be sold and distributed. The petabytes of customer data (including data on you and me) already gathered by big retailers tells them who will want to buy what, where and when.

Amazon uses its S3 system to keep track of millions of stock items across dozens of warehouses and distribution centres scattered around the globe. Operatives can track deliveries in real time to see what is where and where it should be going.

At the point of sale, retailers can use data to determine where stock should be displayed, which stores will sell most of which particular product and to track customer movements around stores.

In the US, Macy’s has reportedly been able to save 26 hours every time it optimises pricing for its 73 million products through big data, allowing them to change pricing more frequently to follow retail trends.

Loyalty cards are not new, but ever more sophisticated analysis of customer habits will lead to an increase in retailers predicting what customers will buy. This has advanced to the point where Amazon believes it will soon be able to predict what you will buy accurately enough to despatch it to you before you’ve even bought it!

The connectivity that’s now possible, particularly the Internet of Things (which I talk about in Chapter 5), is also changing the way you do business. For a business, the ability to have its production, stock control, distribution and security systems all connected and talking to each other will mean greater efficiency and less waste.

With so many possibilities, it can be difficult to know where to start. If you’ve already used data to improve your decision making (Chapter 11 talks about this), then you’ve already identified your strategic priorities and key business questions. That’s an excellent starting point as it gets you thinking about your key business goals and what you need in order to get there. Where to start differs from business to business but the operational priority areas for many business include:

• Manufacturing: Data uses include monitoring equipment wear and tear to reduce unexpected downtime.
• Sales and marketing: For example, monitoring social media and web search trends to predict customer interest, or using weather data to predict sales of a given product at certain times of the year.
• Warehousing and distribution: For example, automatic stock control, optimising delivery routes to avoid traffic jams and monitoring delivery vehicles and drivers.
• Business processes: For example, automatic diagnostic of tumours from scan images, predicting crime, fraud and cyber attacks and predicting and preventing accidents.

Reshaping your business model

Data has applications beyond your everyday operational processes. For some businesses, it has led to a change (even a complete shift) in their business model. John Deere, for example, is transitioning from its traditional agriculture manufacturing business towards more data-based services that meet modern customers’ needs (tractor over to the John Deere sidebar for more information).

If your company has the potential to generate quite a bit of data (through sensors in your products, for example), you, too, could develop services related to that data. Whether you manufacture huge equipment or tennis rackets, data that tells your customers how they’re using your product and how they can use it more efficiently is pretty valuable. This could lead to additional income streams or, for some businesses, a complete change in business model. There’s more on identifying new or additional business models in Chapter 13.

Finding external data

External data is all the information about the wide world outside your company that you can turn to your advantage. This could either be self-generated, for example customer surveys, or collected from an external source – either paid or free.

The data economy is booming and many companies exist purely to supply other companies with information. On the other hand, there’s a huge amount of information out there available for free – most governments these days make concerted efforts to make as much of their data as possible available to the public free of charge. This can be a great source for information on everything from population to weather and crime statistics. Depending on the human need or desire your company is catering for, you will almost certainly find something useful in external sources. I list my top ten free data sources in Chapter 15.

Using internal data

Internal data is information about your company’s performance, activities, wins and losses, which you can generate yourself. This includes sales records, customer databases, employee records and all the day-to-day data that can be captured in the course of your business activities.

In sophisticated industrial operations, internal data may include machine data that allows you to fine-tune the efficiency of your equipment. Data like this can be generated by dedicated devices such as sensors, radio-frequency identification (RFID) or cameras that collect the data in real time. I list my top ten data collection tools for businesses in Chapter 16.

Data has always played a role in how car insurance premiums are calculated: customer postcode, local crime rate, estimated annual mileage, where the car is parked overnight and so on. But insurance companies are now much smarter about this data collection process, using sensors and apps to gather data such as actual annual mileage, the time of day customers do most of their driving and driver speed. All this new data means insurance companies can price policies based on individual driving conditions. Thanks to data and algorithms, dynamic pricing is now a reality.

Making the business case

You wouldn’t invest in a new warehousing facility or renting new retail premises without first making a solid business case for doing so. The same is true of any big data project. It’s a bit like creating a business plan, but for data.

When making your business case, be sure to consider the following:

• The costs of integrating data into your business, including hiring new talent or getting specialist help as well as the hardware and software requirements for collecting, storing and analysing the data.
• The benefits this data will bring to the organisation – for example, increasing efficiency on the production line, saving time and increasing sales by introducing dynamic online pricing or reducing waste by managing stock more efficiently.

More information on making a solid business case can be found in Chapter 10.

Sourcing alternative data sets

Sometimes the costs just don’t stack up, and you can’t justify the data you originally wanted. But that doesn’t mean your data journey is over before you’ve even started.

Some tips for sourcing data elsewhere:

• Competition is becoming increasingly fierce between data companies, so be sure to shop around to see if you can get similar data elsewhere for less money.
• Look at your data requirements to see if you really need all the data you’re asking for. Many businesses fall into the trap of trying to obtain too much data. The trick is to get the minimum amount of data necessary to achieve your goals. While extra data may be nice to have, it’s not essential to what you’re trying to do.
• Consider whether you could generate similar data yourself, for example, by implementing a customer survey on your website.
• Instead of commercial data companies, look at other kinds of partnerships, such as partnering with a local organisation or university that might be looking to achieve similar things.

Big data as a business asset

The companies that are really going to succeed are those who consider big data as a core business asset. I believe those companies who don’t view data as an asset will lose serious competitive advantage.

Data as an operational or business asset means it’s essential to how your business operates and makes money, just like your employees, your inventory, your building premises and your intellectual property.

Amazon pioneered e-commerce in many ways, but possibly one of its greatest innovations is the personalised recommendation system – which, of course, is built on the big data it gathers from its millions of customer transactions. In fact, its ability to use data (and indeed, its whole attitude towards data as an asset) is one of the key building blocks of Amazon’s success. In the early days, when Amazon was primarily a book retailer, the company was the first to extensively use algorithms so that it could automatically provide recommendations for customers: ‘Customers who bought this item, also bought this one … .’ Today, it uses item-to-item collaborative filtering on many data points such as what users have bought before, what they have on their virtual wish list, the items they have rated and reviewed, as well as what other similar users have bought. This means Amazon can heavily customise the browsing experience using data and algorithms. And having worked out how to use data to get more money out of customers’ pockets, Amazon is now setting out on a mission to help other global corporations do the same – by making that data, as well as its own tools for analysing it, available to buy. In my opinion, Amazon is one of the best examples of a company that really values data. As a result, it’s even starting to reshape its business model, creating new revenue streams from its data.

Ensuring access rights and ownership

If you consider data as a key business asset, it’s very important you secure ownership of that data.

Ownership is becoming a big deal in big data, with many people becoming uneasy about the amount of data companies have on them. As an individual, you don’t own your data; the companies you interact with do. Sinister as this may seem (and it’s only going to become an even bigger issue in the next few years), it’s an important thing for companies to think about.

Make sure that, wherever possible, you own the data that’s crucial to your business operations. This is easy if it’s your own internal data, but it gets stickier with external data. If you can’t own the data, then you need to make sure that you at least aren’t going to lose access to it.

Finding the right data storage for you

Computer hard disks are still the storage medium of choice because, at higher storage capacities, they’re often very cheap. For many small businesses, this may be all that’s needed. Of course, you also need computers to house the hard disks, which in turn need a building with an electrical supply.

Distributed storage and cloud storage are two alternative ways for businesses to store their data without investing in expensive dedicated systems and data warehouses to put them in:

• Distributed storage is a method of using cheap, off-the-shelf components to rig up your own high-capacity storage solutions, which are then controlled by software that keeps track of where everything is and finds it for you when you need it.
• Cloud storage really just means that your data is stored remotely but is connected to the Internet and is accessible from anywhere you can get online. In current business usage it tends to mean that you’re paying a third party such as Google or Amazon, or one of thousands of smaller, dedicated cloud storage operations, to store it for you online somewhere. So, you don’t have to worry about physically holding onto the data yourself at all.

Ensuring data security

Security is a huge issue in big data (see Chapter 9 for more information on the data security layer). It’s also pretty specialist, so it’s a good idea to get some help from a data security advisor. Data needs to be secured just like any of your other business assets, like your property, your stock/merchandise, your hardware and so on.

Most people think cloud storage is less secure, but that’s not always the case. In-house storage systems are vulnerable too, sometimes more so. You need to weigh up the risks and the pros and cons of various storage methods. This is where expert help is invaluable.

Avoiding data breaches

Data breaches can lead to huge losses for businesses and there have been some very high profile breaches in recent years.

Hackers attacked American retailer Target’s systems during the 2013 holiday season. Initially Target said that credit card and debit card information connected to 40 million customers had been stolen. Later it emerged that other personal data connected to between 70 million and 110 million people had also been taken. Approval has now been given for over 100 million victims of the data theft to claim compensation, which potentially leaves the retail giant facing claims of over $10 million.

A hundred million victims seems like a lot, but it’s not even the biggest data breach in history. That dubious honour is currently held by Experian-owned data aggregation company, Court Ventures, who had 200 million records stolen! A slightly smaller but higher profile breach happened to eBay in 2014. Hackers accessed and stole millions of customer records by using employee details to log into eBay’s computer systems. All users were forced to change their passwords in what turned out to be a public relations nightmare for the company.

Incidents like this – and the many other large-scale data thefts that frequently take place – show that even the biggest companies often fail to keep the promises they make about protecting data. It’s something that businesses big and small absolutely must get right.

Big data analysis can do a lot of good – but it’s reliant on people (and your customers) trusting that safeguards are in place to keep their personal information secure. It’s therefore essential to protect your data against breaches.

Ways of securing your data and avoiding breaches include:

• Training your staff so they never give away secure information. This can be more of an issue in bigger companies where con artists can gain access to a system by pretending to be ‘Joe Bloggs from IT’ and asking unsuspecting employees for their login information. Regardless of the size of your company, your employees should never give their login information to anyone.
• Encrypting your data so that, even if it is breached, the hackers cannot decrypt the data.
• Have systems in place that detect and stop breaches while they’re happening.

Assessing infrastructure needs

Your first step is to establish what infrastructure you already have and what you need to beef up or create from scratch. There’s more detail on big data infrastructure needs in Chapter 9 and big data competencies in Chapter 8. Some companies build from the bottom up, their very existence made possible by big data.

When you’re looking at using data to improve operations, there are three key areas of infrastructure to consider:

• Creating processes and tools to generate internal data and/or access external data. (I talk about sourcing and generating data in ‘Sourcing the Required Data’ earlier in the chapter.)
• Ensuring that you have the facilities and systems to store and secure that data. As data becomes increasingly important to your business, it’s vital you protect that data – see ‘Ensuring data security’ and ‘Avoiding data breaches’ earlier in the chapter.
• Setting up the algorithms to automatically analyse that data and create the necessary outputs, whether that’s dynamic maintenance schedules, better targeting of customers, dynamic pricing systems or anti-fraud measures.

If you’ve already been using data in your decision making (as discussed in Chapter 11), chances are you already have some of the technology and skills needed. If this is your first dip into big data, then you may need to start from the ground up, but thankfully there are lots of products and services available that make this process easier.

Creating the infrastructure

For most businesses, creating a big data infrastructure doesn’t mean building a data warehouse and employing armies of analysts (unless you’re a Facebook or Google that is). There are plenty of services out there that are aimed at small and medium businesses.

For example, HP made its big data analytics platform, Haven, available entirely through the cloud. This means that everything – from storage to analytics and reporting – is handled on HP systems that are leased to the customer via a monthly subscription – entirely eliminating infrastructure baggage and costs. This removes many of the hurdles associated with implementing big data. Competition with other similar services such as Amazon’s Redshift and IBM’s DashDB should keep subscription prices low and lead to a big increase in the number of businesses employing analytics to improve efficiency.

Running it

This isn’t just a one-off project with a neat beginning, middle and end. Using data to improve your operations is an ongoing gig. As such, that means viewing data as just a part of your business from now on – as important as your product, your employees, your distribution channels, your customer service and so on. Like each of those areas, they need careful monitoring to check they’re working the way you want.

You may start this journey in one area of your business, for example, using data to automatically optimise your delivery routes. Chances are, you’ll soon identify other areas of your operations that could benefit from data. So, if you’re using data in your deliveries, a logical next step might be to use sensors to monitor vehicle wear and tear, thereby automating vehicle maintenance schedules (servicing vehicles as they need it, instead of according to an arbitrary time frame). Once you have the infrastructure in place, it’s relatively easy to extend the applications to other areas of the business.

Looking into the future

It’s a good idea to review progress regularly to see how things are progressing. If you’re not seeing the improvements you expected, you may need to tweak your algorithms or look at collecting additional data.

I’d say you should review progress at least every six months, although if your data is changing all the time, you’ll need to revisit things on a much more frequent basis. Unfortunately, when it comes to reviewing progress, there’s no exact timetable to follow – you’ll need to work out what’s right for your business, depending on the data you’re using and what you’re trying to achieve, through a process of trial and error.

The beauty (and I suppose the difficulty!) with big data is that things are changing all the time: new collection methods are being developed, new analytical platforms are hitting the market and storage options are increasing and getting cheaper all the time. Businesses are finding more and more ways to use data to improve how they do things and gain competitive advantage. As such, how you use data in five years’ time, or even one years’ time, may be different to how you start out using data now.

Knowing where to start is often the hardest part, which is why I place so much emphasis on building a big data strategy, which I talk about in Chapter 10. Once you have the foundations in place and you’re starting to integrate data into your operations and decision making, it’s important to keep that momentum going. In Chapter 13 I look at how to create a big data culture in your business. This means viewing data as an ongoing commitment to improving the way you do business, across all areas of the company and at all levels.