In This Chapter

Knowing where to look for free data

Understanding what sort of data you can get your hands on

Making the most of free data in your business

Data is everywhere – and a lot of it is freely available. These days companies don’t necessarily have to build their own massive data repositories before starting with big data analytics. Many big companies and governments have made large amounts of information available in the public domain, meaning there’s a wealth of data out there accessible to anyone and everyone.

Any company, from big blue chip corporations to the tiniest start-up, can now leverage more data than ever before. Many of my clients ask me for the top free sources they could use in their big data endeavour. Here’s my rundown of some of the best free big data sources available today.

Data.gov (US Government Data)

In 2013, the US government pledged to make all government data available freely online. This site is the result of that pledge, and it’s an absolute treasure trove of information. It acts as a portal to all sorts of government data on everything from climate to crime rates. You can use it for research purposes, or it can serve as a useful tool for developing web and mobile applications.

A security firm could use the portal to look up crime statistics, helping to narrow its focus on a particular area or type of service (linked to a rise in home invasions or burglaries, for instance). Or, if a company were looking to expand and set up a new factory or distribution centre, it could check out unemployment statistics to find a location where there’s likely to be a large number of available (and keen) candidates.

Head to www.data.gov to find out more.

US Census Bureau

This site provides a mountain of information on the lives of US citizens covering population data, geographic data and education. You can get statistics from multiple surveys and there are some cool gizmos like data visualisations and interactive maps to make the information more digestible.

Demographic data like this can be a useful indicator of trends, which is especially helpful if you’re developing a new product or service. It can also help you target products or services to particular local demographics. For example, a concentration of young families in a particular area would be of interest to someone looking for a place to set up a nursery or kindergarten.

To access the data, visit www.census.gov/data.html.

European Union Open Data Portal

The European version of the US census data, this site offers census data from European Union institutions. You can browse datasets by subject (such as industry, finance, and employment) or you can search for specific datasets using keywords.

Find out more at www.open-data.europa.eu/en/data.

Data.gov.uk

This is rather like the US government portal but specifically for the UK. Here you can find open data from the UK government, covering everything from census data to traffic information to energy consumption data. It also houses the British National Bibliography, which has metadata on all UK books and publications since 1950.

As well as published books, the British National Bibliography provides information on forthcoming books. This could be helpful to an independent bookstore looking to schedule events and promotions for the months ahead. If it were to combine that information with insights from local census data, it could predict which titles might be most valuable or appealing to the local clientele.

Visit www.data.gov.uk to access the data. You can submit data requests if you can’t find what you’re looking for.

HealthData.gov

Official site of the Health Data Initiative, here you can find US healthcare data stretching back 125 years. Information available includes claim-level Medicare data, population statistics, clinical care provider quality information, Food and Drug Administration recall data, databases of the latest medical and scientific knowledge, and much more.

This data could help identify trends in healthcare and would be extremely valuable for any company looking to develop health-related products or services in the US market. For example, if your company specialises in products related to quitting smoking, then the statistics and medical data will be a big help in product development.

You can find the data at www.healthdata.gov.

Google Trends

This is a very powerful and versatile tool, providing statistics on search volume (as a proportion of total searches) for any given term since 2004. You can see search popularity for certain phrases or words and how it has changed over time, either on a yearly basis or by drilling down to specific weeks (say, if you wanted to see how an event affected the popularity of a given search term). Given that there are 3.5 billion Google searches every day, the amount of data available is vast! You can narrow the results by geographic location, making sure you get the information most relevant to your business.

Google Trends can be used to understand trends in your industry, what is popular right now and what is becoming more popular (or less popular). It’s a great way of gauging consumer interest.

To get started with Google Trends, visit www.google.com/trends/explore.

Facebook Graph

Although some of the information on users’ Facebook profiles is private (depending on how savvy they are with their settings), a lot of information isn’t private. Facebook has developed the Graph API (Application Program Interface) as a way of querying the huge amount of information its users share with the world. Even if you have your privacy settings cranked up to the highest level (which many people don’t bother to do), Facebook can still provide information to companies about what you’re saying – just not that you’re the one saying it.

Facebook Graph provides lots of useful information for free, but you need to pay for access to the most detailed information. It’s worth checking out the free access in the first instance, particularly if you’re interested in monitoring consumer trends.

Find out more at https://developers.facebook.com/docs/graph-api.

Weather Data Sources

You can find a huge collection of environmental, meteorological and climate datasets from the US National Climatic Data Center, the world’s largest archive of weather data. UK weather data from the Met Office is available through the www.data.gov.uk portal.

Weather data can be used in a number of ways, from estimating customer numbers to deciding how many sausages and hot dog buns to stock on a given weekend.

A farm and petting zoo was having trouble estimating visitor numbers and managing staffing levels. It used standard weather forecasts but with limited success and was frequently caught out by having too many or too few staff on hand. By using historical attendance data it already had in-house combined with detailed local climatic data, it was able to predict visitor numbers and plan staffing levels much more accurately. It was also able to better manage stock and increase sales in the on-site farm shop by targeting the products that best matched the weather.

Climatic data for the US is available at www.ncdc.noaa.gov/data-access/quick-links#loc-clim; go to https://data.gov.uk for UK data.

Federal Reserve Economic Data

Federal Reserve Economic Data (FRED to its friends) is a huge repository of data from the Federal Reserve and a number of government agencies, such as the Bureau of Labor Statistics. Datasets include financial data, business data, consumer price indices and employment and population data. Information from FRED is widely used, widely reported and plays a big role in the financial markets around the world. You can view the datasets in text or graph format and even download it to import into spreadsheets.

FRED’s data can show where the overall business climate is going, which industries are booming, and which aren’t, helping you make better long-term business decisions. For example, if you’re considering expanding or investing in new infrastructure, then a strong economy may indicate it’s a good time to go ahead. However, if the economy is on the slide, then it might be a good time to reduce business costs.

Head to https://research.stlouisfed.org to access the data.

Google Maps

Encompassing its interactive maps, satellite images and Street View, Google Maps is a huge time-saver for many businesses. The range of organisations using this data is huge – from small businesses right up to government agencies.

For instance, a local government planning office could review planning applications using Google Maps rather than having to make a physical visit to the site. Even a local window cleaner could use Street View when quoting for services, creating more accurate quotes quickly and efficiently.

Visit www.maps.google.com to get started.