Chapter 16
Ten Key Big Data Collection Tools
In This Chapter
Understanding some of the most useful data collection tools
Being aware of the data available
Finding out how this data could be used in your business
The sheer volume of data available can be daunting – and so can the number of big data products, tools and platforms out there to help you collect and analyse data. What’s the best way to collect data and how are other businesses going about things? In this chapter I identify my top tools for gathering big data and give helpful examples of how other businesses and organisations use these tools.
Keep in mind that you may well need a combination of tools to answer your strategic questions (I talk about those in Chapter 11). For example, when working with clients, I often find the most useful insights come from combining internal data (like transaction data) with external data (such as Twitter posts). A similar approach may work for you.
Smartphone GPS Sensor
GPS sensors track where you are by using satellite information. Many people use their phones to find things of interest near them or to get directions to a particular location, but GPS can do so much more. Your smartphone is constantly collecting data on where you are, and this information can be incredibly valuable for businesses.
If local is a key part of your business, you could develop a customer app that uses GPS data. For example, if you run a food delivery service, you could develop a customer app and use the GPS data on where customers are located. This could be useful if you specifically wanted to target customers within a ten-mile radius or want to send out a promotional offer and skip customers out of the area or on holiday.
Smartphone Accelerometer Sensor
This is the sensor that measures how fast the phone is travelling, which is particularly helpful for companies in which vehicles or transport play a big role. Today, some insurance companies are using these sensors to tell how fast customers drive so they can offer dynamic premiums (essentially, cheaper premiums for better drivers). A taxi firm or delivery company could use these sensors to see how well drivers are doing, helping them to improve driver performance, make financial savings and improve customer experience.
You can either develop your own app or make use of the many existing apps available.
Accelerometer data has been cleverly used by the City of Boston, which created an app to detect potholes. Bostonians were asked to download the app to their smartphone which would then run in the background as residents went about their daily lives. As residents drove around the city, the accelerometer in their phone would measure how fast they were travelling and when they slowed down or braked. A little algorithm was then applied to this data to identify potential potholes – because when you drive along a road and see a pothole, you usually slow down or swerve to avoid it and then speed up again. (Even if you didn’t see it and slow down in time and end up driving right over the pothole, the sensor in your smartphone would detect the bump.) The algorithm identified potential potholes from the smartphone data, giving city officials a real-time picture of the condition of the city’s streets. Needless to say, this was a much more efficient and cost-effective system than the old one, which involved someone physically driving down all the roads in the city twice a year!
Telematics System
Telematics is a combination of telecommunications and informatics, and it involves the collection and transmission of data from vehicles. It’s a bit like a smartphone accelerometer sensor (see the preceding section) but is built into vehicles instead of smartphones. Telematics systems also tend to provide more detailed information than accelerometer systems.
Police forces are using telematics systems to monitor how officers drive their vehicles. Not only does the information collected help to improve driver education and performance, it can also help reduce emissions and tyre degradation and improve vehicle maintenance.
Businesses that might benefit from a system like this include bus companies, taxi firms, and haulage companies. Think of it as a more modern and accurate version of a ‘How’s my driving?’ banner on the back of a truck! The information you find out can improve staff training and reduce costs overall.
Wi-Fi Signals
Wi-Fi signals are an excellent source of data, particularly for retail businesses. Nowadays, almost everyone has a smartphone and smartphones give out signals when the Wi-Fi option is switched on (because the phone is constantly scanning for local Wi-Fi networks).
Wi-Fi signals can monitor how many people pass your store, how many people stop and look in your window and how many people then choose to come inside. This technology is now incredibly cheap and easy to implement and it’s an accessible option for retailers of all sizes.
Retailers are using these signals to varying levels of sophistication. Some track how customers physically move around a store and what they stop to look at. Some use personal information (for instance, if you’ve downloaded a store app or given your details to access the in-store Wi-Fi) to target promotions and make recommendations while you’re in the store. As an example, iBeacons provide a way of communicating with smartphones in the store, alerting customers to products nearby.
Increasingly, Wi-Fi data will be combined with video data and facial recognition software to identify a shopper’s age and gender, even her exact identity.
With 300 million registered users, LinkedIn is one of the most popular social media sites in the world (currently ranked third). It’s a great resource for professionals and businesses alike, from childminders to Fortune 500 companies.
Your business can use LinkedIn to find talent and make connections. If you operate in a niche field, it’s a great way to find people with very specific skills. Or, if you’re considering applications from candidates, you can check out their profiles, connections and recommendations before narrowing down the pile and selecting who gets invited for an interview.
Currently the biggest social media site in the world, Facebook is likely to be your first stop for social media big data. Behind the cat pictures and humblebrag status updates is a vast amount of data on customer behaviour. Facebook offers incredibly useful breakdowns of customer information and analyses of all the data it has. Some of this you need to purchase, but plenty is available for free.
Facebook is scarily accurate when it comes to insights into peoples’ behaviour. In fact, it can now predict very accurately when someone is about to change her status from ‘single’ to ‘in a relationship’ (presumably, the other way around too!).
Facebook data encompasses text data, photo data, video data, and user Likes. All this data can be analysed and used to your business’s advantage – whether you want to target a promotion or understand how many pregnant women live in a certain area.
Twitter is the second most popular social media site and, unlike Facebook, there are no privacy settings, which makes it very interesting to businesses.
Every time a Twitter user mentions a company or product, that information is visible to everyone, including the company. Even if a product isn’t mentioned explicitly in the text of the tweet, companies can detect when their product features in a photo. Examples of this might include a drinks company finding pictures of people drinking its product, restaurants finding pictures taken in their restaurant and fashion houses finding out who is wearing their clothes.
Carrying out sentiment text analysis (which I talk about in Chapter 5) is a good way to tell how your business or product is doing in the Twitterverse. You can gain insights into the popularity of a product or service, understand customer satisfaction and deal with any problems swiftly.
In another example of Twitter analysis, researchers were able to predict which women were most at risk of developing postnatal depression. They analysed Twitter posts, searching for verbal clues in the weeks leading up to the birth. They found that negative language and words hinting at unhappiness, as well as an increased use of the word I, indicated an increased chance of developing postnatal depression. Sentiment analysis can tell a lot about users’ feelings, opinions and experiences without having to trawl through individual tweets one at a time.
Machine Sensors
This includes any sensor on anything – whether it is a shop door or a tennis racquet. By incorporating sensors into your business or product, you can glean a ton of information about your customers. Thanks to the Internet of Things (discussed in Chapter 5), everything is becoming more intelligent, more data is being collected than ever before and it’s easier than ever to analyse this data.
Even very traditional industries, like farming, are getting in on the sensor act: tractors have sensors that dynamically adjust maintenance schedules and fields have sensors that monitor soil condition and temperature. Even yoga mats can include sensors that monitor your position and send information on how to improve your yoga practice.
Sensors are tiny, affordable and very easy to add to products. They are revolutionising the way businesses interact with their customers, enabling them to understand how customers actually use their product and to make personal recommendations.
Transaction Data
Transaction data is a great place for any business to start its big data journey, since it is internal and therefore relatively easy to access and analyse. In a nutshell, it shows you what your customers bought and when. Depending on what you measure, it can also show where the item was purchased, how the customer came across the product and whether she took advantage of a promotion.
Even basic transaction records can be very useful for measuring sales, monitoring stock levels and predicting what you need to order (or manufacture). You may already have all the transaction data you need to answer your strategic questions (I talk about them in Chapter 11), or you may find you need to implement new transaction systems to fully answer those questions.
Finance Data
This includes all your company’s financial data, not just the transactions. Finance data has many uses such as predicting cash flow and influencing investment and other long-term business decisions.
Often companies find that combining finance data with other kinds of data is particularly powerful. For example, you might look at your own internal financial data along with big data from open government sources about industry trends and the wider economy. Combined, this data could tell you whether now really is a good time to expand operations or invest in a new fleet of vehicles.