This is the least used method of assessing BI success. Again, part of the problem lies in the ad-hoc nature of BI solutions. The other problem lies with the lack of a central point of control (e.g., the Information Center) where one might obtain statistics and usage results that could be rolled up into an ROI scenario.

If you need to use ROI as part of your justification, then you will need to provide several key elements and measures beginning with the associated costs.

You can add more questions of your own, I am sure. One question you have to ask is “Why are we doing this, and what if we didn't?” If the only reason to provide new BI functionality is because someone is fussing about the need to do more reporting independently, then satisfy his or her itch and be done with it.

If the BI project is departmental in scope, then justification is probably established enough to sell the idea to management. What if the department is going to implement a vendor's tool that others within the organization aren't thrilled about? I hope that the project and its associated usage will be defined as a localized process. Small projects can, however, come back to bite you later if a decision is made to move to an enterprise-wide standard. Are the short-term solution proponents prepared to do this? Getting rid of an entrenched BI tool is a little bit like trying to get rid of the family cat—it usually cannot be done.

This is the area where BI at the enterprise level will make the greatest sense within all levels of the enterprise. If we look back at the benefits that we assigned to our BI solution, we should be able to clearly state what this will provide the business.

As part of this approach, you really have an opportunity to look at how a BI solution spans the enterprise. Let's look at a simple sales and marketing example. Our theoretical current need is to provide rapid feedback and analysis for a campaign that we are going to run in a new geography. As a part of this effort, we will have a special promotion for new customers. At a minimum, we should have a logical tie among the sales, marketing, and advertising departments.

We should be able to quantify current sales in similar geographies and demographics. If we have run campaigns in the past, can we provide any predictive information about how useful we believe the results will be? Suppose that we cannot provide any of these items. Why would we not be able to provide them? Are we better served by putting the proper database in place before we begin, so that we are not always launching efforts and campaigns in the dark?

I am currently part of a global sales team. It would be invaluable to know when we are making specific, concentrated efforts to promote sales. Just as some other firms do, my company sometimes warns me in advance, but sometimes it does not. I am often unaware until a call comes in for assistance. Just like the theoretical example above, there is a need to provide pre-effort information so that we can maximize our investment.

If we are taking the business justification approach, there are several things we need to know. Write these things down:

Sometimes, an answer of “Why? Because we want to!” is the only honest one. There is nothing wrong with this. The problem is that when the rougher times come (and they will come), you look a bit silly complaining about not getting the results you wanted or the difficulty the users are having with the ease of use of the tools recently purchased.

Table 8-1 lists some topics and parts of an evaluation form once used by a customer to document and justify a BI solution. I have added a few bullets of my own. I submit that one should be able to complete this form in its entirety or pause for a moment and ask “Why not?”

It may provide a basis or template for your own justification and analysis of the impact of a BI solution in your organization.

Obviously, you would modify the document to suit your internal processes, and it's helpful to discuss with others why you are doing this. IT and the end users should spend a serious day or two discussing what they are trying to accomplish.

One project I worked on involved a unique internal approach. The end users and IT spent a day off-site developing a presentation that they would make to their key executives. The ultimate purpose was to convene a meeting and have both sides present the scope and benefits of the project to executives.

I was not involved in the final presentation, but I did hear back from the sponsor. He said the results surprised both the users and IT. Executives wanted to know the following things (if my memory serves me correctly):

One of the key aspects of this meeting was that the scope of the “good news” about the project was far too narrow. The executives asked for a broader range of thinking and for the inclusion of some other key areas in the project. The final item, cost, is one that is usually misrepresented in most BI planning scenarios. Why is this?

The real cost of any of this end-user computing “stuff” lies in the many hours that no one will quantify or track. When you set out to engage in the typical query and reporting activities, you will hit the same brick walls every time, regardless of the tool you are using. Those brick walls are as follows:

So how much do you make per hour? How many hours are you willing to spend or can you spend learning a tool? What if your requirements are heavy in calculations outside the scope of your data as delivered, but you have minimal time to spend producing the results?

Every moment that you spend working with a tool without obtaining the results you need costs you and your company money. If you never obtain the correct results, then every moment that you spent is a waste and is pure cost with no return to the business. That goes for every user who invests time in “playing” BI without getting to the end of the job. If you have not identified the specific users and expected usage in advance, then it is guaranteed that you will end up with a select few who can do the math and many hangers-on and significant dropouts.

After you have identified the population that intends to use the BI tools, it is important to track them and see how they are faring. Let's assume that a single user copy of a tool that you have selected has cost you $500 after a discount. There are 15 users in Department XYZ who are slated to work with this tool. Your software outlay would be $7,500. You have no expectation or ROI numbers at this time. There are five users in particular that you believe would receive the greatest benefit from this tool.

After 30 days, you go back to see how everyone is doing and discover that three of the 15 really like the tool, and seven others use some of the items created by the three new “mavens.” There are five people who gave it a decent run and simply couldn't get the job done. The seven who get ancillary benefit also are rendered pretty well useless for doing anything with the tool on their own.

So is the cost of the 15 users greater than the overall benefit to the three heavy users and the seven casual users? How do we know? There are some costs that we can associate with this effort, regardless of the benefit. Let's assume the five unhappy and unproductive users make the equivalent of $15 per hour and that they tried to work with the tool for 20 hours each. These are some costs that will net a return of zero.

So in our hypothetical scenario, the net cost of this experiment for the failed participants was not bad. However, there are some nuances regarding cost factors, and I suggest that you consider them as well. The cost of the seven semi-productive users should be factored in because they mostly sit around and wait for the three creators to deliver. (Remember the chapter on end-user types?)

If we assume that the seven also spent 20 hours each (at the same hourly rate), we have an additional cost.

We now have a non-productive outlay of $4,000 + $2,100, which equals $6,100. We haven't factored in any of the IT staff's work, the setup, and any other costs associated with the effort. We are just looking at some basic costs. Most of the cost/benefit analyses I've seen for BI stick to those costs that can be seen. In most cases, only software, hardware, and IT-related costs are used.

What if you had performed a reasonable user survey and knew that the five disgruntled users were the key to most of the expected return that we had targeted? In such a case, we haven't realized much on our return at all. It all boils down to knowing as much in advance as you can and having a plan to begin with.

These numbers we've tossed around present a very simplistic set of metrics, and the actual dollar amounts are irrelevant. By now, you should be wondering about the loss in productivity and whatever business impact the non-productive might have made had they not been diverted by their involvement in the BI “experiment.” Or what if a different tool had been selected? Might they have been rendered more productive with a different set of products?

You can also fall into a trap of purchasing an enterprise license at such a tremendous discount that all sense of justification flies out the window. There simply is no justification for this if there is no deployment plan.

The thought in most cases is that the per-seat charge is so small compared to what might have been spent that a company can afford to carry the “shelf ware” of all the extra licenses. Eventually, people begin to question what has been done to deploy and use the software, and seldom will the answers be to anyone's liking.

Many BI vendors will discount every potential sale even when there is no volume involved. Why do vendors provide such huge discounts if their products are so superior to their competition? Heck, they told you this in the demo and presentation, didn't they? I have some unverified reasons why this is a common tactic. One of the greatest equalizers is competition.

There are several large and highly competitive vendors in the BI space. Unlike a diverse software company like IBM, their smaller portfolio is all they have to sell. They often feel that they have one crack at you on this go-round, so they are going to do all they can the win the business. Because BI is a bit more mainstream, many industry pundits pontificate as to who and what is #1, and customers and vendors use this news as well.

However, the #1 vendor will still gleefully engage in “discount wars” because you may not have read about their elevated status—and you may not care anyway. So we go to battle over price.

Another tactic I have seen from BI vendors is to throw in everything but a butter churn and new spats when they are bidding on a large deal. I recently was involved in a deal with a large IBM customer looking at some BI software from another vendor. It was a product that we did not compete against, but one about which I know a fair amount. The total package was valued in the millions of dollars. The discount was extremely high.

I had an opportunity to chat with some individuals on the customer side about the proposed outlay for the BI tools. I was fortunate to find a customer who actually knew the vendor's product and how it might be used within his organization. When we discussed the potential usage for some of the myriad products being thrown into the pot, as well as who would use the package, it became very clear that the customer had nowhere near the requirement for this volume of product, but the deal was certainly impressive. Was the vendor being dishonest, or was this a typical ploy to cement the deal? I do not know and do not care to speculate. It did, however, lend a bit of credence to my insistence that BI be considered at the enterprise level. Why?

Vendor discounts are a reality. The larger the deal you are looking at, the more they will want to sweeten the pot to get your business. In many cases, you are looking at server-based software, not a zillion individually shrink-wrapped boxes of user product. The material cost to the vendor is often quite small. So let's assume the following:

If you have such ammunition, you can walk into your vendor meetings with confidence and a plan to utilize the large wads of software that they are probably going to throw in the box. I would suggest that you keep your plans to yourself and do not disclose your justification scheme. Why?

One area that is not factored into the equation in many cases is the annual maintenance fee that you will have to pay. If I were selling you a large box of stuff and had to support it knowing full well that you will attempt to use it all, I would not be very generous in my discounting. This would be especially true if I had been known to discount maintenance.

Maintenance fees vary, but 20 percent of the list price is quite common. Look at a situation in which a customer is going to buy $4 million in BI software discounted 75 percent, which would result in a net sale of $1 million. The vendor charges 20 percent of the list price for annual maintenance. The calculations look like this:

In my business case, I have seemingly saved the company $3 million in software. But what if we deployed only 15 percent of the software in the first 24 months of our operation, and we were already getting a little disgruntled with this vendor and their products? What would 15 percent of the original product package have cost us?

The only factors in this equation are software costs and maintenance. If we went with plan B, does this mean we save $3.8 million by not going with the larger deal? If you had no tight plan to begin with, the answer is probably “yes.” If, however, you had a large scale, well-oiled BI scheme in mind and knew that large volumes would be needed, utilized, and put into action, then you would want to “go long.” Take advantage of the market and competitive environment to get the largest deployment, lowest cost BI solution that you can muster.

I believe that BI solutions have been so nebulous in providing proven value metrics and so difficult to quantify that low cost is about the only way that many customers can be enticed into jumping over the stick and taking a chance.

Can you imagine someone purchasing $1,000,000 in BI tools on an enterprise basis but not having a plan to deploy and track the usage? I can because I have seen these situations many times. I know that many of these firms can afford to throw away this much money and more without feeling the loss. I do have to ask, however, what was the point? Can you have a decent justification model and still fail or come up short? Oh, yes, most assuredly, yes.

I recall very vividly a customer company that had spent a fortune on BI implementation based on recommendations from a major consulting firm. The customer wanted a solid BI plan and seemed incapable of producing one of its own. Internal politics and in-fighting were killing the customer.

The customer had a mainframe-centric IT organization and a significant investment in older but very productive BI tools. The consultant's model included some workstation-based query and reporting tools. These tools had been touted as a sure-fire replacement for the existing host-based suite. There was no validation that the proposed tools could replace and enhance what was already in place, but the consultant said the new software could do the job. Mind you, this was not a firm that provided significant work fulfillment; their job was to pontificate and counsel, not to get their hands dirty, except at astronomical prices.

The customer stepped into an environment of perpetual churn and confusion. The company couldn't attain what it had already developed; it couldn't begin to explore new solutions and functions. The worst part of this was the executives developed a mantra whereby they could not let go of the money they had already spent. There was an attitude of “We paid over $5 million for this advise, and we are going to follow it!”

The BI consulting firm I helped manage at the time had several people on-site and was directly involved in supporting and delivering a quality BI environment. We sat in seemingly endless meetings trying to develop a credible BI plan with the customer. Our consultants had been involved with this customer for literally thousands of hours over several years. We knew, and tried to articulate, that a totally new data warehouse was the only possible means of implementing the changes that the customer wanted to put into effect. We also tried to point out that the customer and the impressive pundit firm had never validated the ability of the proposed new tools to replace what the customer already had in-house.

We actually proved beyond a reasonable doubt that our ideas were the best for the customer. We developed an extranet solution at no cost to help the company stop some significant customer abandonment problems at no expense to the customer—just to prove a point. However, we were a small firm with nowhere near the presence and prestige of the large firm that had already drained the customer of significant funds.

The ultimate outcome for this customer was not pretty. I do not wish to share any more specifics, lest the customer be identified. The customer rejected our extranet solution because it required external customers to access their mainframe. Security and data-related issues were all totally addressed, but the customer's technical committee had rejected the plan strictly based on the committee's preconceived notions and some heavy biases. The real solution was attempted later and was a dismal failure, and a few heads rolled in the process. I have heard less and less about this customer over time, and although it is still a major player in its industry, I suspect this customer's annual revenue is a shadow of what it once was.

The reality is that the successful deployment of BI solutions has been well documented and written about for quite some time. The failures or half-hearted efforts seldom make the press. I submit to you that all the justification scenarios and documentation will make sense only if you successfully implement the plan you set out to accomplish.

Sit down and ask yourself “Why are we doing this?” If your scope of BI spans the enterprise, then you will have answers at your fingertips. You will have your answers because such an approach can be determined only in a committee setting. It takes a more global agreement to satisfy the enterprise.

Create an outline similar to the BI Project Outline shown earlier in Table 8-1 and discuss it with executives in the organization. Part of the preparation for this should have been the meeting with end users and IT in which both sides identified what they believe they will accomplish and how.

If you are of a mind that defies such justification and are within an organization that does not want to put in the time and effort to develop a plan for BI, you can opt to purchase a set of tools and provide access to data for a set of users and wish them the best. In many cases, this is the norm for BI.

If you have concerns about your enterprise plans for BI and you lack a mission or plans, then this may be a good thing. If you are not ready for the enterprise view and approach, try identifying a set of users who have some potential interest and who you believe could provide significant new business value.

Assuming that you decide to do this, what kind of support structure and processes should you consider when implementing a BI solution? If you can develop a solid end-to-end plan for one part of the enterprise, you will have a model for expanding this success and creating an enterprise BI environment that can grow and permit all facets of the enterprise to work cooperatively.