Self-Service Business Intelligence
If you are reading this book, it is most likely because you need to use data. More specifically, it may be that you need to take a journey from data to insight in which you have to take quantities of facts and figures, shape them into comprehensible information, and give them clear and visual meaning.
This book is all about that journey. It covers the many ways that you, an Excel user, can transform raw data into high-impact analyses delivered by Microsoft’s new self-service business intelligence (BI) paradigm. This fresh approach presumes presumes that you are not dependent on central IT nor do you need their help on a regular basis It is based on enabling the user to handle industrial-strength quantities of data using familiar tools and to share stunning output in the shortest possible timeframe.
The keywords in this universe are
- Fast
- Decentralized
- Intuitive
- Interactive
- Delivery
Using the tools and techniques described in this book, you can discover and load your data, create all the calculations you need, and then develop and share stylish interactive presentations.
It follows that this book is written from the perspective of the user. Essentially it is all about empowerment—letting users define their own requirements and satisfy their own needs simply and efficiently by building on their existing skills.
The Microsoft Self-Service Business Intelligence Solution
It is important to understand from the start that Microsoft’s self-service business intelligence solution is a constantly evolving process. It has been assembled from a series of parallel technologies and is in a continuous state of flux. Fortunately this perpetual motion is now at a peak of readiness, and although it is still undergoing some enhancements and revisions, it is already in a state in which you can use it with confidence.
The Microsoft self-service business intelligence solution has two parts
- The Excel BI Toolkit—Allows users to import and model data then create jaw-dropping visualizations.
- Power BI—Lets the creators share their insights and data with colleagues on a variety of devices.
By combining these technologies, Microsoft has made an amazingly powerful set of tools available that you can use to find and mash up data that you can then display in crisply interactive reports. Let’s take a more in-depth look at this solution.
At the core of Microsoft’s self-service BI is the Excel BI Toolkit. This consists of Excel (inevitably) and four add-ins that allow you to import, model, prepare, and display your analyses. These elements are
- Power Query—To import and transform data
- PowerPivot—To model data and carry out all necessary calculations
- Power View—To display your results interactively
- Power Map—To show your data from a geographical perspective
You may find that you do not need all these products all the time. Indeed, you may find that you use them independently or in certain combinations. This is because self-service business intelligence is designed to be flexible and respond to a variety of needs. Nonetheless, we will be exploring all of these tools in the course of this book so that you can handle most, if not all, of the challenges that you may meet.
Once you have developed reports (or presentations, if you prefer to call them that) using PowerPivot and Power View, you will probably want to share your insights with your colleagues. This is where Power BI enters the equation. Power BI, which technically is an aspect of SharePoint online, lets you load Excel workbooks into the cloud and share them with a chosen group of co-workers. Not only that, but your colleagues can interact with your reports to apply filters and slicers and to highlight data. Power BI also lets information workers share the queries and, possibly complex, data ingestion routines that they have created using Power Query. This way your organization can avoid the duplication of effort that can arise when staff work in “data silos.” In addition, you can validate certain data sources as being the key route to an approved data set. Power BI can also ensure that the Excel workbooks that have been shared are updated automatically and regularly so that users are always looking at the most recent data.
Note There is no Power BI for on-premises SharePoint sites at the time of writing.
Taken together, this combination of tools and technologies creates a unique solution to the challenges of creating and sharing analytical insights. However, let me say again that you may not need all that the solution can offer. If all you need to do is share workbooks, then you do not need to share queries. The advantage of self-service BI is that it is a smorgasbord of potential solutions, where each department or enterprise can choose to implement the tools and technologies that suit its specific requirements.
The Excel BI Toolkit and Power BI
To understand how all these elements fit together, it will probably help if I begin with a more detailed overview of the various technologies that are employed. This should help you see how they can let you discover and load your data and then calculate and shape your data model so that you can create and share presentations and insights.
Power Query is one of the most recent additions to the self-service BI toolkit. It allows you to discover, access, and consolidate information from varied sources. Once your data is selected, cleansed, and transformed into a coherent table, you can then place it in an Excel worksheet, or better still, load it directly into PowerPivot, which is a natural source for data when you are using Power View and Power Map.
Power Query allows you to do many things with source data, but the four main steps are likely to be
- Import data from a wide variety of sources. This covers corporate databases to files, and social media to big data.
- Merge data from multiple sources into a coherent structure.
- Shape data into the columns and records that suit your uses.
- Cleanse your data to make it reliable and easy to use.
There was a time when these processes required dedicated teams of IT specialists. Well, not any more. With Power Query, you can mash up your own data so that it is the way you want it and is ready to use as part of your self-service BI solution.
Power Query is discussed in more depth Chapters 12 and 13.
PowerPivot is essentially the data store for your information. Indeed, many people refer to the Excel Data Model when they talk about data in PowerPivot. Power Query lets you import data and make it useable; PowerPivot then takes over and lets you extend and formalize the cleansed data. More specifically, it allows you to
- Create a data model by joining tables to develop a coherent data structure from multiple separate sources of data. This data model will then be used by Power View, Power Map, and the Power BI natural language querying engine.
- Enrich the data model by applying coherent names and data types.
- Create calculations and prepare the core metrics that you want to use in your analyses and presentations.
- Add hierarchies to enhance the user experience and guide your users through complex data sets.
- Create KPIs (Key Performance Indicators) to allow benchmarking.
It is worth noting that you can load data into PowerPivot directly without using Power Query. As you will see in this book, you have the choice. Whether you want or need to use Power Query at all will depend on the complexity of the source data and whether or not you need to cleanse and shape the data first.
PowerPivot is discussed in Chapters 9 through 11.
I think of Power View as the “jewel in the crown” of self-service business intelligence. It is a dynamic analysis and presentation tool that lets you create professional-grade
- Tables
- Matrixes
- Charts
- Maps
Not only that, but it is incredibly fast and highly intuitive. It provides advanced interactivity through the use of
- Slicers
- Filters
- Highlighting
A Power View report is only a special type of Excel worksheet, and you can have many reports in an Excel file. In most cases, users tend to create Power View reports using a PowerPivot data model, but you can also create Power View reports using data tables in an Excel worksheet if you prefer. However (at the risk of laboring the point), a PowerPivot data set can be tweaked to make Power View reports much easier to create and modify than can a table in Excel.
Power View is discussed in Chapters 2 through 8.
Power Map is, as its name implies, a mapping tool. As long as your data contains some form of geographical data, and you can connect to Bing Maps, you can use Power Map to create geographical representations of the data.
The types of presentation that you can create with Power Map include
- Maps
- Automatic presentations of geographical data
- Time-based representations of geographical data
As is the case with Power View, Power Map is at its best when you use the data in a PowerPivot data set. However, you can use data in Excel if you prefer.
Power Map is discussed in Chapter 14.
Power BI is a cloud-based data sharing environment. Power BI leverages existing Excel 2013 PowerPivot, Power Query, and Power View functionality and adds new features that allow you to
- Share presentations and queries with your colleagues.
- Update your Excel file from data sources that can be on-site or in the cloud.
- Display the output on multiple devices. This includes PCs, tablets, and HTML 5-enabled mobile devices as well as Windows tablets that use the Power BI app.
- Query your data using natural language processing (or Q&A, as it is known).
Power BI is discussed in Chapter 15.
Preparing the Self-Service BI Environment
Before you can begin to use the Excel BI Toolkit you need to make sure that your PC is set up correctly and that everything is in place. This is not difficult, but it is probably less frustrating if you get everything set up correctly before you leap into the fray rather than get annoyed if things do not work flawlessly first time. If you are working in a corporate environment where these add-ins are the norm, then all your problems are probably solved already. If not, you might have a few tweaks to perform. So let’s see how to ensure that your version of Excel is ready to fly with self-service BI.
PowerPivot
To begin with, PowerPivot is only available in Microsoft Office Professional Plus, Office 365 Professional Plus, and in a standalone edition of Excel 2013. It is not available in Office on a Windows RT PC. PowerPivot does exist in Excel 2010, but it uses a different version of the Excel Data Model (which can be converted to the 2013 data model). So if you open an Excel 2013 workbook containing a data model created with Excel 2010, you will get a warning that you will have to convert the data model and that this step is irreversible. Note also that a data model created with the 2013 version of Excel is not backward compatible with the previous version
You will know if PowerPivot is enabled if you can see a PowerPivot menu and ribbon in Excel. If this ribbon is not available, you will have to enable it like this:
- In the File menu click Options.
- Click Add-Ins on the bottom of the menu on the left. The Excel Options dialog will look like Figure 1-1.
Figure 1-1. The Excel Options dialog
- In the Manage popup list, and select COM Add-ins.
- Click Go. The COM Add-ins dialog will appear as shown in Figure 1-2.
Figure 1-2. The COM Add-ins dialog
- Check the Microsoft Office PowerPivot For Excel 2013 check box.
- Click OK.
The PowerPivot menu and ribbon should now be available in Excel.
Note Depending on your exact configuration of Excel, you may see more or fewer add-ins displayed in the COM Add-ins dialog on your PC.
Power View
Power View is also currently only available in Office Professional Plus 2013 and Office 365 Professional Plus, as well as in the standalone edition of Excel 2013. Power View is not available in Office on a Windows RT PC. Not only that, but you will have to install Microsoft Silverlight 5 for Power View to work. Fortunately, however, Power View will detect if you have Silverlight installed and if it is not present, Power View will install it the first time that it is run.
Normally Power View is an integral part of Excel. Indeed, if you open Excel and activate the Insert ribbon, you will see the Power View button, as shown in Figure 1-3.
Figure 1-3. The Power View button in the Excel Insert ribbon
There may be times when the Power View button is grayed out. If this is the case, you will need to enable the Power View add-in. You can do this almost exactly as I described in the previous section for PowerPivot, except that at step 5, you need to check the Power View box, which you can see in Figure 1-2.
You should then see that the Power View button in the Insert ribbon is no longer grayed out.
Note Power View is also available in SharePoint and is virtually identical to the Excel version. If you need an introduction to this version of Power View, refer to Chapters 2–8, which will cover most of your requirements. I will not, however, be discussing SharePoint BI in this book.
Power Query
Power Query is currently an optional add-in for Excel—providing that you are using one of the following versions:
- Microsoft Office 2010 Professional Plus with Software Assurance
- Microsoft Office 2013 Professional Plus
- Office 365 Pro Plus
- Excel 2013 Standalone
Since it is optional, you may have to download and install the add-in. If it is already installed, you will see a Power Query ribbon available in Excel. If you do not, here is how you can install it:
- Close Microsoft Excel.
- Download the Power Query install file. At the time of writing this is available at the following URL: http://www.microsoft.com/en-gb/download/details.aspx?id=39379.
- On the download page click Download. You will see the Choose The Download That You Want page.
- On this page, ensure that you select the correct version for your version of Excel (32-bit or 64-bit).
- Click Next and select a directory to which you want to download the .msi file. The March 2014 file is named PowerQuery_2.10.3598.81 (64-bit) [en-US].msi.
- Go to the directory where you downloaded the .msi file in the previous step, and double-click the file. The security warning dialog will appear as in Figure 1-4.
Figure 1-4. The security warning dialog
- Click Run. The Power Query Setup dialog will appear as in Figure 1-5.
Figure 1-5. The Power Query Setup dialog
- Click Next. The License Terms dialog will be displayed. You can see this in Figure 1-6.
Figure 1-6. The License Terms dialog for Power Query
- Check the I Accept The Terms In The License Agreement box.
- Click Next. The Destination Folder dialog will appear as in Figure 1-7.
Figure 1-7. The Destination Folder dialog
- Leave the suggested destination folder unless you have a specific reason to select another and click Next. The final installation dialog will be displayed, as shown in Figure 1-8.
Figure 1-8. The final installation dialog for Power Query
- Click Install. The install process will run. You may see a User Account Control dialog requesting permission to run the install program. If you do, click Yes. Once the process has finished you will see the completion dialog, as in Figure 1-9.
Figure 1-9. The completion dialog once Power Query is installed
- Click Finish.
Power Query is now installed, and the Power Query menu and ribbon will be available in Excel.
Power Map
As of Microsoft Office 2013 Service Pack 1, Power Map is now an integral part of Excel. If you are running an older version of Excel, then you may have to download the add-in and install it separately. Since this process is virtually identical to the process I just described for Power Query, I will not reiterate all the details here. Suffice it to say that you follow all the steps you followed for Power Query, except that at step 2, you use the following URL instead (as at April 2014): http://www.microsoft.com/en-gb/download/details.aspx?id=38395. If you have Power Map already installed, you will have an active Map button in the Excel Insert ribbon, as shown in Figure 1-10.
Figure 1-10. The Power Map button in the Excel Insert ribbon
Power BI is the online solution that enables you to share the presentations and queries that you have created using the Excel BI Toolkit. It is an enhancement to SharePoint online and requires a subscription to use. As things stand, Power BI does not include Excel (unless you have a Power BI with Office 365 subscription), and at the time of writing there are three different pricing plans. As this state of affairs could evolve over time, I will not go into the details here, but I suggest that you see what is available on the Microsoft web site if you are considering a Power BI subscription.
One you have taken out a subscription to Power BI—and you have a valid Excel license—you can use the SharePoint online Power BI site application to add a robust, dynamic location where you can share Excel workbooks in the Microsoft cloud.
Note At the time of writing there is a free trial offer for Power BI. I can only recommend that you take advantage of this if you want to test out all that it can deliver.
To take full advantage of the enhanced functionality that Power BI can bring to SharePoint online, you will need to add a Power BI site to your cloud-based portal. This only takes a few clicks, but it will enable you to
- View workbooks up to 250 MB in a browser on Office 365 if you save and enable them on the Power BI site.
- Highlight certain spreadsheets as “Featured Reports.”
- Display thumbnail images of Power View reports in a spreadsheet in the Power BI site.
Assuming that you have a working subscription to Office 365 with Power BI and you also have a starter site, you now need to create the Power BI site To do this,
- In the navigation bar on the left of the portal window, click Site Contents. The Site Contents page will be displayed, as in Figure 1-11.
Figure 1-11. Adding the Power BI app
- Click the Power BI icon—or possibly click Add An App first to display all the available apps, including the Power BI app, and then click it—and it will be installed after a few seconds. You will then be taken into the Power BI app, which looks like Figure 1-12.
Figure 1-12. The Power BI app once it is initially installed onto a SharePoint Online team site
It really is that simple to enable the Power BI site on your portal. Once this is done, you should see Power BI listed in the navigation bar on the left of the portal window when you use your site.
If you are using a Windows 8.1 tablet, then you may well want to download and install the Power BI app for Windows. This app is available for free in the Windows App store; the current URL is http://apps.microsoft.com/windows/en-gb/app/b7e7c94d-2ea3-4fa6-a277-9d19a1f697ba.
This app will allow you to view and interact with Power View reports from multiple Power BI sites.
A version of this app for the iPad has been promised for mid 2014, and could be available by the time that you are reading this book. An Android version is rumoured to be in the works.
Corporate BI or Self-Service BI?
This book is all about self-service business intelligence. Although this concept stands in opposition to corporate business intelligence, the two interact and relate. However, the distinctions are not only blurred, they are evolving along continually changing lines.
In any case, I do not want to describe these two approaches as if they are mutually antagonistic. They are both in the service of the enterprise, and both exist to provide timely analysis. The two can, and should, work together as much as possible. After all, much self-service business intelligence needs corporate data, which is often the result of many months (or years) of careful thought and intricate data processing and cleansing. So it is really not worth rejecting all that a corporate IT department can provide for avid users of self-service BI. At the same time, the speed at which a purely self-service approach can deliver rapid discovery, analysis, and presentation can relieve hard-pressed IT departments from the kind of ad-hoc jobs that distract from larger projects. So it pays for central IT to see self-service BI as a friend, and for users to appreciate all the support and assistance that an IT department can provide.
Self-service business intelligence, then, is part of an equation. It is not a total solution—and neither is it a panacea. Anarchic implementation of self-service BI can lead to massive data duplication and so many versions of “the truth” that all facts become mere opinions. Consequently, I advise a measured response. When managers, users, or, heaven forbid, external consultants announce in tones of hyperactive excitement that Microsoft have produced a new miracle-working solution to replace all your existing BI solutions, I suggest you take a step backward and a deep calming breath. I would never imply that you use Power BI to replace “canned” corporate reports, for instance (to solve this requirement see Pro SQL Server 2012 Reporting Services [Apress 2012] by Rodney Landrum, Brian McDonald, and Shawn McGehee). Yet if you need interactive reports based on volatile and varied data sources, then the Excel BI Toolkit and Power BI could be a perfect solution.
The Excel Data Model
When introducing PowerPivot toward the start of this chapter I made a passing reference to the Excel Data Model. As this is fundamental to the practice of self-service BI using Excel and Power BI, you really need to understand what this data model is, and how it helps you to create valid analyses.
The data model is a collection of one or more tables of data that are loaded into PowerPivot and then joined together in a coherent fashion. The data can come via Power Query, be obtained from existing Excel tables or worksheets, or be imported from a variety of sources. There can only be a single data model for an Excel file.
Admittedly, you can place all your data in a single “flat” table in Excel and use that as the basis for Power View reports and Power Map output. However, it is highly likely that you will want to develop a data model using PowerPivot if you intend to use data sets of any complexity. There are occasions when building a good data model can take awhile to get right, but there are many valid justifications for spending the time required to build a coherent data model using PowerPivot. The reasons for this investment include
- You can go way beyond the million-row limit of an Excel worksheet if you are using the Excel Data Model in PowerPivot. Indeed, in PowerPivot tables of tens of millions of rows are not unknown.
- A coherent data model makes understanding and visualizing your data easier.
- A well thought out data model means less redundant information stored in a single table when it can be referenced from another table rather than repeated endlessly.
- PowerPivot saves space on disk and in memory because it uses a highly efficient data compression algorithm to store the data set. This means that a workbook using a data set will take up considerably less space than storing data in Excel worksheets.
- Since a data set is loaded entirely into the PC’s memory, calculations are faster.
- A data model can be prepared for data output. More specifically, you can apply formatting and define data types (such as geographical types, for instance) for specific columns so that Power View and Power Map will recognize them instantly and make the correct deductions as to the best ways to use them.
- A data model can contain certain calculations (some of which can get fairly complex) that are designed to ensure that the correct results are returned when slicing and filtering data in Power View and Power Map.
- A data model can contain hierarchies and KPIs.
- A data model can be used to create complex pivot tables in Excel if you not want to use Power View or Power Map.
- A data model can be the basis, or the proof of concept, for a fully-fledged SSAS (SQL Server Analysis Services) tabular data warehouse.
As an example of a data set, this book will use a simple model that uses the sales data for an imaginary company that sells classic and modern British sports cars throughout Europe and that is starting to expand into the United States. This fictitious corporation is called Brilliant British Cars, and it has been going for a couple of years. Their data is relatively simple, and the data model for the company can be seen in Figure 1-13.
Figure 1-13. The Excel Data Model used by Brilliant British Cars
The art and science of developing data models could easily be the subject of a separate tome. It is, in fact, not unrelated to basic relational database design, which has been described exhaustively in dozens (or hundreds) of books over the last couple of decades. As a reader, you can breathe a sigh of relief as I have no intention of attempting to cover this subject in this book. As far as our sample data model is concerned, I will just take it as is and suggest that you consult one of the many excellent resources already available should you need further guidance when developing your own specific data model.
Throughout this book I will be using the established best practice, which is to use the Excel Data Model as the basis for self-service BI. However, as I remarked earlier, you can use plain old Excel tables as a source of data for both Power View reports and Power Map deliverables if you wish.
How This Book Is Designed to Be Read
The suite of technologies that makes up the Microsoft self-service business intelligence offering are essentially independent products. It follows that you may need only to focus on one or two of them to solve a particular problem. Or it may be that you already know how to use part of the toolset but need to learn, or revise, other elements.
Because we are looking at a set of tools, each of which can be learned individually, this book is not designed to be read only in a linear fashion. Given that the primary focus of this text is on delivering output that has the “wow” factor, it begins with Power View to show what can be done with the new presentation tool that is now integrated into Excel.
The chapters on Power View, however, do not presume any knowledge of how to assemble or develop an underlying data set. Their aim is to get you up and running with interactive presentations as fast as possible. Nevertheless, it is likely that you will one day need a data model to use as the basis for your reports. So after the chapters on Power View, you learn how to use PowerPivot to create data sets and get them ready to be the bedrock of your Power View deliverables.
Frequently PowerPivot is all you need to connect to source data. Yet sometimes you need something more advanced to load and prepare data from multiple varied sources. If this is the case, you can learn how to perform these tasks using Power Query in the couple of chapters that follow the three on PowerPivot.
You can then see all that Power Map can do for you in the penultimate chapter and learn, in the final chapter, how to pull it all together by sharing your data and insights with Power BI.
There are, however, other possible reading paths, if you prefer. So, depending on your requirements, you may wish to try one of the following approaches.
If your primary focus is on discovering data and then preparing it for later use—that is, you need to load, mash up, rationalize, and cleanse data from multiple diverse sources—then Chapters 13 and 14, which introduce Power Query, should be your first port of call. Chapter 13 explains how to connect to many of the data sources that Power Query can read, and Chapter 14 gives the reader a thorough grounding in how to process and transform source data to make it coherent and usable by PowerPivot as part of a logical data set.
Conversely, if the source data that you are using is already clean and accessible, then you may be more interested in learning how to create a valid and efficient data model that is clean and comprehensible and contains all the calculations that you need for your presentations. In this case, you should start by reading Chapters 9 through 11.
Taking Data and Preparing It for Output
If you are faced with the task of finding, cleansing, and modeling data that is ready to be used for reporting, then you will probably need to use both Power Query and PowerPivot. If this is the case, you may be best served by reading Chapters 13 and 14 on Power Query (to import and shape the source data) and then Chapters 9, 10, and 11 on PowerPivot (to model the data).
Taking Existing Excel BI and Sharing It
You may well be a PowerPivot expert already and have possibly learned to use Power View in its initial incarnation as part of SharePoint. If this describes your situation, you may want to move straight to the part where you learn to share your reports in the cloud. This means that Chapter 15 on Power BI is for you. Here you will learn how best to load and share Excel BI workbooks and Power Query queries as well as how to update workbooks in the cloud with the latest data from on-premises data sources.
It is not just tables and charts that create the “Eureka!” moment. Sometimes an insight can come from seeing how data is dispersed geographically, or how geographic data evolves over time. If this is what you are looking for, then you need to look at Chapter 8 (which covers maps in Power View) and Chapter 14 (which covers Power Map) to learn how these two tools can create and deliver new insights into your data.
Delivering Excel BI to Mobile Devices
If you need to ensure that you and your colleagues can access their data on mobile devices, then Chapter 15 on Power BI is the one for you. Here you will see how to use the Power BI app on a Windows tablet, as well as how to use Power View on many other mobile devices.
To Learn the Product Suite Following a Real-World Path
If you are coming to self-service BI as a complete novice, then one way to learn it is by taking the path that you could need to follow in a real-world situation. If this suits you, then you could try reading the entire book, but in this order:
- Discover and prepare data—Start with Chapters 13 and 14 on Power Query.
- Create and enhance a data model—Next, read Chapters 9, 10, and 11 on PowerPivot.
- Create visualizations—Continue with Chapters 2–7 on Power View.
- Add geodata outputs—Move on to Chapter 8 and Chapter 12, which cover maps in Power View and Power Map.
- Share your insights—Finish with Chapter 15 on Power BI.
Anyway, these proposed reading paths are only suggestions. Each chapter is designed to cover a complete aspect of self-service BI in as thorough a fashion as is possible. Feel free to jump in and pick and choose the path that best suits you.
The Self-Service Business Intelligence Universe
The amalgam of products and technologies that make up the world of Microsoft self-service business intelligence can seem complex and even confusing at first glance. This is, to some extent, because some Excel add-ins seem to have overlapping aims, or that the interface between creating reports and sharing them is not always immediately clear.
Figure 1-14 attempts to provide a more comprehensible vision of the total toolset so that you can better see how all the pieces work together.
Figure 1-14. The self-service business intelligence universe
Conclusion
Microsoft self-service business intelligence, then, is not an application, but a suite of tools and technologies that allow you to find, import, join, and structure data that you then extend with any necessary calculations; you then use this data as the basis for interactive presentations that you can subsequently share in the cloud and access using a variety of devices.
More precisely, you will be using a set of Excel add-ins and a cloud-based subscription service to create and share data and high-impact analyses with your colleagues. The output can be viewed using a PC or a mobile device and can allow your public to select and filter the reports to discover their own insights.
In any case, that is enough of a preamble. The best way to learn any instrument is to practice using it. So it is time for you to move on to the chapters that interest you and start your journey into the wonderful world of self-service business intelligence.