Sink or swim:
evaluation as the
key to IT outsourcing

6.1  Introduction

From our research base, we have identified five main occasions when organizations are pushed into improving their IT evaluation practice. The first is when a new senior executive feels the need to get to grips with what is going on in the organization, and institutes new management and evaluation practices. The second is when senior executives are exposed to a new evaluation tool that seems to address their concerns, such as the balanced business scorecard described in Chapter 5, and decide to adopt the approach across the organization, or at least in several sub-units. The third is when the organization is experiencing a crisis and it is not clear what is wrong and why. The fourth is when the IT function comes under strong pressure to justify its, invariably, rising expenditure. The fifth and more enduring influence tends to be where an organization is considering, then enters an outsourcing agreement of any size, consuming say 15% or more of the IT budget. Very often, however, organizations do not anticipate this, and are dragged into many of the evaluation improvements needed for outsourcing, with experience being a hard learning route.

This is a particularly important finding because IT outsourcing – the handing over of IT assets, services and activities to thirdparty management – has been a fast-growing phenomenon. On Yankee Group estimates the global market revenues were US$50 billion in 1994. According to IDC reports, this was predicted to rise to US$121 billion in 2000. Lacity and Willcocks see the global market exceeding US$150 billion by 2004¹. From the mid–1990s, the UK has seen a 15–20% revenue growth rate, with 2000 revenues probably exceeding £6 billion. By 2004 most large organizations will be outsourcing on average at least 30% of their IT budgets.

If IT outsourcing is to be a central plank in an organization's business and IT/e-business strategy, how can business value from outsourcing be assessed, managed and captured? Referring back to the cost/contribution framework shown in Chapter 2, it is clear that IT outsourcing, in theory, and often according to vendor claims, can contribute to all five value drivers. Historically, the most obvious contribution has been to the ‘cost efficiency’ area, particularly where organizations have outsourced IT infrastructure and operations. This also has been one of the easier areas to monitor. IT vendors can also make contributions as a ’service to the business’, although many organizations realize only late in the game that there is usually a cost-service trade-off in an outsourcing arrangement. Moreover, often ’service’ is measured against technical criteria rather than impact on the business, and so the value of the service is not leveraged as much as it could be. The years 1998–2001 have seen a rising trend towards vendors claiming they can also add value by leveraging business improvement, direct profit generation and making a competitive edge contribution. All three contributions require much closer partnering between client and vendor than has traditionally been the case.

The levels of success being achieved from these arrangements is regularly under scrutiny, and Lacity and Willcocks in their most recent work on global IT outsourcing sound many notes of caution². However, it is clear that these different sorts of outsourcing arrangements require different but connected evaluation regimes if there is to be any chance of exploiting their potential business value.

Before looking at the establishment of an evaluation system for an outsourcing arrangement, we will review organizational patterns and contract economics that contribute to and influence the evaluation system.

6.2  Outsourcing: patterns of assessment

In practice, the pre-existing pattern of IT evaluation has a large part to play in the effort required to assess a vendor bid. Several patterns of pre-outsourcing performance measurement dominate. We will call these ‘traditionalist’, ’service to business’ and ‘trading agency’.

(1)  Traditionalist – these organizations tend to focus their evaluation around the feasibility, development and routine operations stages of IT investment. For feasibility, the traditionalists use predominantly financed-based cost-benefit criteria. For IT development the major criteria can be summarized as ‘within time and budget to acceptable technical quality’. For routine operations the technical efficiency of IT performance combined with some end-user service measures dominated.

In the move to outsource, traditionalists tend to either spend a long period of time thrashing out detailed parameters of service requirements from the vendor or they established a general rule that service will not deteriorate from pre outsourcing levels.

(2)  Service to business – prior to outsourcing, these organizations have moved the evaluation focus from IT efficiency towards the IT function's level of user service and business contribution. Such objectives are sought through chargeback systems, IT as a profit centre and/or the introduction of service level agreements with or without penalty clauses. These organizations find it a less difficult transition to evaluate a vendor's performance.

(3)  Trading agency – here organizations allow the IT function to market its services internally as well as to the external marketplace. In these trading agency arrangements, the business divisions frequently also have the right to buy IT services either internally or externally (or both), although the internal IT provider is regarded as the ‘preferred supplier’. In some examples, the in-house IT department is made a part-owned separate company.

From these assessment patterns, there are several evaluation routes into outsourcing. Organizations jumping straight from a pre-existing ‘traditionalist’ approach to outsourcing evaluation find the most difficulty in assessing vendor bids, drawing up contracts and assessing subsequent vendor performance. If such organizations attempt to stick to pre-outsourcing levels of service, they will invariably experience unanticipated costs, conflicts over service quality and disaffected business users. Organizations closer to the ’service to business’ type evaluation still find a number of problem areas when carrying out an in-house versus out-of-house assessment, and when setting up performance measures for an outsourced aspect of IT. ‘Trading agencies’ offer the clearest comparisons between bids by the in-house team and those by external vendors. The experience of setting up a trading agency can also feed into developing measures to assess performance of an alternative vendor.

The move from a ‘traditionalist’, through a ’service to business’ to a ‘trading agency’ assessment pattern suggests a maturing of the ability to assess in-house IT costs and performance against a vendor bid. As always, it is difficult to jump from the earliest state to the most advanced state overnight – most organizations need to evolve through the stages. Evaluation problems will always exist relative to the outsourcing contract, regardless of the pre-outsourcing assessment pattern. The ’service to business’ or ‘trading agency’ patterns, however, are much better suited to the assessment of a vendor bid and ease the transition to outsourcing.

6.3  The centrality of the contract

If an organization outsources IT, the outsourcing contract is the only certain way to ensure that expectations are realized. In practice, weak contracting, based on inadequate assessment of a vendor bid and backed up by poor monitoring systems, results not only in unanticipated, higher costs but also in creation of major problems for client organizations. It is easy for all parties involved in the contract to agree broadly on what is required from a vendor. Those same parties often rely on notions of ‘partnership’ to offset any difficulties arising from loose contracting. These assumptions rarely proved a sufficient base from which to run effective outsourcing arrangements.

When it comes to drawing up effective outsourcing contracts, we found that the devil is indeed in the detail. Kevin Tomlinson, a vendor manager, stated: ‘Outsourcing contracts are agreed in concept and delivered in detail, and that's why they break down.’ Figures 6.1 and 6.2 provide summaries of the typical benefits that organizations look for and vendors promise.

Figure 6.1 Assessing the vendor bid: benefits and issues – 1

Figure 6.2 Assessing the vendor bid: benefits and issues – 2

It becomes clear that unless, as a minimum, the questions listed here are addressed before IT outsourcing contracts are signed, then unanticipated costs and some significant problems tend to fall upon client organizations.

Bid economics: ten lessons

By way of summary, a number of cautionary points can be raised for those assessing the economics of vendor bids.

(1)  You may be able to achieve similar savings internally as those offered by a vendor.

(2)  Some savings may be less real than others (and arise from creative accounting).

(3)  Your IT costs may already be falling.

(4)  You may be comparing total in-house costs against a vendor's selective bid.

(5)  Vendors do not necessarily get better deals on hardware and software.

(6)  Examine carefully the assumptions behind the vendor bid.

(7)  The economics can change quickly even during three-year contracts.

(8)  Establish if and where the vendor makes a profit (to avoid opportunistic behaviour).

(9)  Outsourcing can carry hidden costs (e.g. through incomplete contracting, post-contract management time and effort, and ambiguities in the contract exploited by the supplier).

(10) The vendor bid can reveal ways of improving in-house performance.

6.4  Setting up a measurement system

Once an organization has decided to outsource any aspect of its IT function, it will need to monitor vendor performance. In this section we extend the discussion on the importance of the contract as the fundamental building block for a measurement system, and look at different types of outsourcing contracts. We then focus on issues relating to measurement systems and service level agreements, and provide guidelines on these topics.

Tight and loose contracting

Where organizations are setting up and running measurement systems for vendor performance, the most common potential ‘bad practices’ observed are:

(1)  over-reliance on the pre-existing standards and measurement systems; and

(2)  failure to define comprehensively in the initial contract the detailed expectations of both sides on standards and how measurement will proceed.

These usually occur as a result of either time pressures or a belief in the ‘good offices’ espoused by the vendor. Much depends on the quality of the relationship between vendor and client. Many organizations are still actually refining measures during the first six months of the contract. This works reasonably well where, in the early stages of a contract, the vendor is anxious to demonstrate flexibility and good partnering.

However, in other contracts, vendors are more concerned about maintaining the letter of the original contract. This creates contract management issues that become exacerbated where the contract is vague or does not cover issues that are arising. The problem arises from different perceptions by the vendor and client of the meaning and role of the contract. The client may believe in the rhetoric of partnership much more than the vendor. The emerging lesson is that participants need to be clear as to what the relationship amounts to and how both sides understand it.

Moreover, while clients have a tendency to believe that the quality of the vendor-client relationship will see them through the limitations of performance measurement arrangements in the contract, in fact the latter can affect the former adversely. In one contract at the high street retailer WH Smith it took 18 months of vigorous contract management finally to get the performance measures right and end disputes on service. The contract manager described the situation as follows:

There was a contract, a legal one with our signatures on it, with various sections in it, but really it did not define what the service was going to be. And worse still when we invoked penalty clauses, the section in the contract that talked about the penalty clauses was so ambiguously written that we had a bun fight for nearly six months over it . . . I think that all of that proved to me at that stage anyway, that we needed to get out of this partnership issue and back into a proper business contractual relationship and that is what really set us off, I think, down the right road.

The learning point is to ensure that in the period just before signing the contract what is actually said is written into the contract, and be prepared to spend time chronicling the agreement in monotonous detail. One reason that a vendor might want to adhere rigidly to the contract deal is because of very slim profit margins.

More recently, as Lacity and Willcocks note, there have been creative attempts to improve on contracting practice. These have included:

–  longer evaluations, for example Ameritech studied outsourcing for 15 months before awarding a contract to IBM;

–  customer written detailed contract included with the request for proposal/ITT (invitation to tender);

–  provision for competitive bidding for services beyond the contract, as in the British Army's Logistics Information Systems Agency-EDS five-year deal signed in 1996 and extended beyond 2001;

–  flexible pricing, for example a share in the vendor's savings, ‘open book’ accounting and reduced fees based on the vendor's other customers; and

–  beginning a long-term contract with a short-term one.

Measurement: systems and service levels

We have seen that it is possible to operate on a more flexible partnership basis and also that some areas to be outsourced may be difficult to specify precisely in terms of service and performance required. However, in outsourcing, discretion may be the better part of valour. In reviewing 18 organizations contemplating or undertaking outsourcing, Lacity and Hirschheim³ suggested the following ’safety first’ guidelines on creating a measurement system.

(1)  Measure everything during the baseline period.

(2)  Develop service level measures.

(3)  Develop service level reports.

(4)  Specify escalation procedures.

(5)  Include cash penalties for non-performance.

(6)  Determine growth rates.

(7)  Adjust charges to changes in business, technology and volumes.

Our own research base reflects the finding that it is advisable not to start a contract until current information systems services have been measured in a baseline period. There are a number of points here. Some organizations leave the contract incomplete with a view to carrying out measurement in a baseline period after the contract has started, or trusting in a good relationship with the vendor to deal with problems as they arise. Both approaches are hardly low risk and can leave an organization as a hostage to fortune. See the following case study of a UK retail and distribution company for an example.

–  A contract between this company and a vendor of telecommunications services was producing major cost savings, but the measurement system still needed tidying up during the course of the contract because of lack of specific targets when it was drawn up. The Contract Manager stated:

(The vendor) largely wrote it and we signed it. All we were looking for was escalation procedures, what the nap connection charges were, and there was some discussion as to how they would respond and that they wouldn't do this that and the other. Even those comments were ambiguous in the way that they had been written. In undertaking the service level agreement procedure and the definition of our service level agreements I think two things were sorted out. First of all, we then developed a service level agreement that was ours .. . Once we'd done that, we then realized that we had no telecommunications targets and strategy relative to the vendor, so we developed them. It was from that point onwards that we rested control from . . . (the vendor) . . . of our telecommunications strategy, back to where it should have been, which was here all along.

There may be time pressure to get a contract started. This may cut down on the time in which the baseline period for measurement is allowed to run. Lacity and Willcocks recommend a six-month period to allow for fluctuations in service levels due to factors such as tax season, seasonal business oscillations and end-of-year data processing. If service levels are measured monthly and compared, this allows the setting up of a target variance for each service performance within which to fluctuate before ‘excess’ charges to the vendor will become payable, or underperformance by the vendor can be penalized.

Refining the measurement system after the contract has started can work, provided this was part of the contractual agreement in the first place. Thus, a major US-based bank signed an outsourcing contract for running its mainframe service. A notable point here is that the pre-existing measurement system was already fairly rigorous.

Broad inferences were given based upon the service agreements that . . . (the bank's) . . . data centre already had on its businesses. They were not really formal service level agreements. They were merely statements of what the online day would be from when to when by service, of all the many different services we have across all the different hardware platforms . . . All that was made available to . . . (the vendor) . . ., and established as the initial service level agreement guidelines. The bank then tasked itself with the vendor's agreement to getting much more crisp service level agreements and we spent a lot of time with piloting and doing various other things to try to come up with standard, across-the-region support agreements and service level agreements which everyone could sign up to. We are on to reiteration three and close to signing off what will be the final service level agreements.

Contract Manager

A further stipulation about the baseline period is to measure everything, not just what is easiest to measure. The problem is less prevalent where what is being outsourced is an area that was heavily measured before and is on the more traditional ‘factory’ or production side of IT operations. However, there can be a learning curve even here. Thus, WH Smith outsourced telecommunications to DEC, and from the experience was able to formulate much tighter measurement for a contract between Our Price Records, a company in the group, and the vendor Racal.

A more difficult area to measure, whether outsourced or not, is systems development. Our own view is that these areas are probably best managed on an ‘insourcing’ (hiring resources from the market to operate under in-house management) rather than on a strict ‘outsourcing’ basis. The problem is that the end goal for systems’ development, and how to get there, can rarely be well-defined. Some organizations, Pilkington for example in their on-going deal with EDS, handled this by stipulating that the vendor will make defined resources available for a defined period. Measures of vendor staff productivity must also be in place. However, these measures should not be based merely on the speed with which development proceeds through a defined methodology and timetable. The outputs of development also need to be measured in terms of business impact, for example improvements in cost, quality and service, systems reliability, ease of use and ease of user learning. The elapsed time to business use is also a useful measure, with financial penalties built in for under-performance.

These are difficult types of measures to formulate unless there are in-house systems people on the team, who already have key targets and measures in place before outsourcing. (See the following case study of a UK manufacturing company for an example).

–  The company outsourced software development of warehouse and carousel control systems in the factory. There was a lack of in-house expertise in this area. It emerged that there were complex interfaces with the company's mainframe systems and the mainframe development work was outsourced to a further vendor. According to the IT manager, the company had no complaints about the work of these vendors:

It was lack of appreciation of the complexity of the interface early on because of not high enough calibre staff internally to recognize that. Also nearly every department was affected in some way by these systems. Yet despite the fact that at the outset it was recognized as a business objective to enhance the capability of the site to deal with orders, we found ourselves in a situation where as software was developed and then went to user acceptance testing it was only then that users realized what the system was doing and then raised very real problems . . . it has enabled us to rationalize production in Europe but it was longer than it should have been and cost more and it took us longer to become self-sufficient. We were too dependent on a third party for a core, essential business application.

In later development projects this company successfully operated an ‘insourcing’ approach, bringing external expertise to in-house teams to ensure that transfer of learning took place. They have also operated with clearer objectives and measures on development performance, not merely on hours worked and effort expended.

In addition to the above areas, there may be a series of services commonly provided by the IT department, but not documented. Before outsourcing, it is important that these are analysed and included in the service agreement with the vendor along with measures of their delivery. What is not included in baseline period measurement will not be covered by the fixed price offered by the vendor and subsequently may be open to ‘excess charges’. Examples of such services may be consultancy, PC support, installation and training services.

Clearly, specifying service level measures is critical. While this is regularly prepared, a common mistake is to then not stipulate 100% service accountability from the vendor. At one help desk support contract we researched in a major oil company, 80% of service requests had to be responded to within 20 seconds and 90% within 30 seconds. Financial penalties were attached to failure to meet these criteria over a specified period of time. However, the vendor was also made responsible for reporting in detail on this performance and providing explanations when 100% was not achieved. Essentially, a measure was also needed to ensure that the service requests not handled within the stipulated criteria were subsequently dealt with in an agreed reasonable time. Note the importance of detailed service level reports in this example, as well as the importance of agreeing in advance what happens when problems require escalation. This must lead to financial penalties for non-performance as shown in this statement of an IT manager of a major bank:

It was agreed at contract time there would be financial penalties . . . for failure to perform of two kinds. One, if a direct operational loss is caused because of negligence on (the vendor's) part in any one instance and the bank has to pay its clients money in terms of interest charges or penalties, then (the vendor) will indemnify . . . (the company) . . . up to certain limits, and that's on a one time basis . . . Secondly, if on an overall average service evaluation, taking its ability to provide a service over the course of a month, it fell below a specified level. So we have two specific financial implications of failure to deliver the service, one over time and one for a particular occasion, and that's well in place.

It is important to look to the future when negotiating service levels and their price. In particular, the rate at which service needs will grow is often underestimated. Organizations need to include realistic growth rates in the contract agreement's fixed price. Specific clauses may also be needed to cover large service volume fluctuations due to merger, acquisitions or sale of parts of the business.

Overall, client organizations need to make measurement work for rather than against their interests. Thus, in one US manufacturing company, the contract specified a two-second response time for key applications such as order entry and customer service. It also specified vendor support for up to 20 users at a given time using a 4GL. However, during the first week the 11 4GL users were taking up more than 30% of the machine cycles and making response times for critical applications unachievable. The vendor could have been forced to upgrade the technology provided, but the client, in this case, felt the demand would have been unreasonable. A tight contract reasonably handled by both sides was felt to improve the vendor–client relationship and levels of satisfaction all round.

6.5  Outsourced: anticipating evaluation issues

In this section we signpost some further problem areas encountered. Some of the following issues are quite widely known, and yet are still experienced by organizations undertaking outsourcing. Other problem areas identified below are more difficult to predict. All are very real possibilities which any organization contemplating outsourcing should consider.

Significant effort may be required to develop adequate measuring systems

This was stressed earlier in the context of assessing vendor bids. Here we look at the issue for organizations that have already outsourced. As an example, a UK County Council signed a five-year contract with a major vendor. The contract involved mainframe operations and applications service. This was the first of three major contracts for the council. Even though the IT department had been operating largely as an internal trading agency, there was still an immense amount of work needed to get the measurement system in place. The IT Director stated:

There was a lot of information available to the outsourcing companies about the current equipment, current costs of running equipment and staffing costs. A lot of information from us to them. But although we had been operating as a trading organization, there weren't any service level agreements in place to build up a business relationship between the departments buying the service and the operations area itself. That was why it took five months to get from preferred supplier to contract stage. We actually wanted to have the whole of the contract underpinned by agreed and signed service level agreements before we entered into the contract.

It is interesting to note here the desire to get the detailed service level agreements in place before the contract is entered into. From a review of our research base, it emerged that user involvement in the establishment of service level agreements was particularly important if user needs were going to be identified properly and if there was to be user buy-in to the measurement activity. However, more detailed service measures and costing procedures can have unanticipated effects on user behaviour, as will be discussed below. The sheer detail of the service level agreements may also create a monitoring problem. Reflecting on their experiences, outsourcing organizations stressed the importance of focusing on the key measures in any service level agreement. As Robert White of outsourcing consultants Lucidus stressed:

You can actually reduce the number of measures if you focus on what is valuable. One US manufacturer we advised had over 150 measures and a unit of 50 collecting the stuff . . . no one there acted on the results, the information was reported up. Now they are working with 17 measures and the unit of 50 people has been dispersed to do something more useful.

Outsourcing can require a culture change in measurement

It is all too easy to underestimate the gap between the pre- and post-outsourcing approaches to measurement and control. Many organizations do not, at least in their early contracts, move that far away from their existing measures and standards for IT performance. This can leave latent problems and conflicts that will emerge across the life of the outsourcing contract, as one IT director of a UK public sector organization explains:

The performance measures were not strong enough. This was because of the culture we operated in. I think they took us and I don't blame them, we weren't very professional. I think there have been some very good attempts at tightening up performance measurement. But once you've got a contract you've got a contract and if you are dealing with sharp guys like them, then it's very difficult.

The fact that this IT director is getting a better deal on service towards the end of the contract suggests one reason why many organizations might choose to go for contracts of five years or less in length. The prospect of contract renewal reinvigorates vendor motivation to deliver service.

The possibility of vendor opportunism

This issue was hinted at in the previous point. A number of organizations operate a multivendor strategy explicitly to limit vendor opportunism. Other organizations learn over time about vendor opportunism and how to deal with it. As one example, a manufacturing company outsourced IT to a major bureau in order to cut costs. A year into the contract it still had no real in- house expertise to manage the contract and no real IT strategy, despite the fact that IT was beginning to impact adversely the conduct of business. The vendor was providing trainee contractors for systems’ development, but charging top rates.

Moreover, the company was having to train vendor staff about the company and the industry in which it was operating. Subsequently, the company gained contract expertise, built up an in-house technical staff base and renegotiated the vendor role to a much smaller one.

One cautionary case presented by Lacity and Hirschheim⁴ reveals another facet of vendor opportunism. One ten-year contract was the product of senior management looking at ways to contain or reduce rising IT costs. A six-month baseline period was measured with the vendor contractually bound to deliver the average service level of this period. However, the contract was signed before the baseline measurement was completed. Therefore, the services covered in the contract were not completely defined. In data centre operations the contract specified a fixed number of resources, for example tape mounts, CPU minutes and print lines for a fixed price. On applications’ development and maintenance, the company received a fixed number of man-hours of service. Other utility services were ill-defined in the contract. While the promised 20% reduction of projected in-house IT budgets may be delivered by the vendor on the fixed price, in fact the ‘excess’ charges as a result of an incomplete contract may well cancel out any benefits.

In fact, responses from the operating companies suggest that the vendor reduced its own costs by degradation of service, lack of responsiveness, ‘excess’ charges and transferring skilled staff from the client to other contracts. Additionally, despite promises, the vendor failed to develop service levels and performance measures after the contract went into effect. The vendor also took a strict view of the contract's ‘change of character’ clause, continually interpreting it in its own favour. In a second case there was again failure to negotiate a sound detailed contract: ‘The threat of opportunism was readily apparent in the excess charges, degradation of services, loss of IS expertise, antiquated technology and overworked (vendor) staff.’

Internal charging systems may create problems

Even in a well-managed outsourcing contract, there can be latent problems between users in business units, the IT people managing the contract and the vendor. In many examples, the problems built around the charging system as it affected users. In this respect, the experience of a major brewing company in the first year of the contract is quite a common one, as a senior manager explained:

Complexity of recharging is what's causing those communications challenges . . . They (the business users) are very wary of it because they now realize that it's an outsider who's charging them and maybe there is a risk they will get charged more and people are going to get more commercial.

However, the recharging issue can actually become such a bone of contention that it undermines the whole basis of the contract. For example, consider the five-year contract between a major vendor and a multinational manufacturing company. The strategic thrust in this multinational was to get rid of national data centres and devolve IT to business units. The company also wished to rationalize head office computing and save on IT costs. The vendor took over all existing IT work for head office and European companies in the multinational. However, the inadequate initial analysis of IT costs, together with incomplete specification of performance measures and insufficient detail on what services were required fed into creating large-scale problems in recharging users for services. The IT director stated:

They came to us and said what do you spend on computing at the moment and I'll tell you it was difficult to actually arrive at that figure. There were bits of computing going on all over the place, so that was a stab in the dark. The best educated guess we could make, we think all of that costs us that, and they said right we will give you 15% more.

The initial difficulties were unbundling exactly what the client was paying for, and then determining when excess charges became payable. There were also problems over ‘change of character’, where the hardware base was changing along with the services required:

Immediately problems began to occur. How much computing power were we entitled to before we paid any excess charges? ... Of course, there was a billing cost mismatch because they said we will do it for in total about £8 million. So we were going to give them £8 million and they were going to provide services. That then gave us the problem of actually charging out the £8 million, so we had to create an arbitrary invoicing system to recover the £8 million . . . We wondered well the £8 million has got to come down hasn't it. They were saying ‘But we are having to take a beating on this, it's all swings and roundabouts.’ Well that's all right, but when you're the guy in the department with a PC, you are thinking its madness. They want to cross-charge me £10,000 for a PC, I can go out and buy it for less.

Clearly these problems could have been less contentious if the vendor was making more than a slim margin on the original deal. Here the lack of clarity in measurement at the contract stage led to opportunities for a pressured vendor to argue legitimately that many of the services were in addition to those covered by the fixed price agreement. This also brought additional cost pressure on the hastily erected recharging system. At the end of three years the vendor cancelled the contract and each user negotiated separately directly with it. Most of those companies stayed with the vendor, initially, but some drifted away.

Users may become more wary of the IT service

In the cases of charging systems described above, users clearly became much more anxious about the service they were getting for the money they were being charged. In some respects, this is often a healthy development, and may lead to users focusing on necessary rather than ‘nice-to-have’ services. It may also induce a much greater commercial awareness about computer use in their department. On the other hand, as one systems’ developments manager of a manufacturing company said, there can be some less attractive outcomes in outsourcing situations:

I think when you talk about service level agreements and the sorts of terminology you start surrounding a lot of the services with then they tend to get in the way of true fast response, and a question of how much does it cost tends to be an issue. We are already finding that by charging internally for services then the idea that you have to locate a customer who has a budget that can pay, gets in the way of getting a fast response. It shouldn't do, but it still does. It's magnified beyond reasonable proportions with an external agency.

IT costs may become too transparent

This would seem to be a contradiction, in the sense that the transparency of costs as a result of outsourcing is usually proclaimed as a desirable outcome. However where there is still a large in-house IT capability, there may arise inflexibilities in the ways in which funds can be utilized and additional IT work achieved. This statement of the IT manager of a US bank confirms this:

One of the things that some of us were concerned about as part of the deal, but which was overlooked by senior management who signed up to the deal, was the fact that we know the way we operated here in IT. We could always fudge costs . . . There's always a little bit of fat in any budget that allows us to take on something unexpected. I am talking about the ability to bring on a new software package which might enhance processing in an area, a systems software package, a new tool, which might cost you a license fee plus £5000 a year in maintenance costs. We could always do these simple things in-house.

The irony here is that the in-house IT staff felt that the vendor had done too good a job of analysing costs and establishing what the price for different services would be. The problems are very much for operational IT staff rather than more senior management who in fact, as this contract manager of a financial services company explains, continue to see the non-degradation of contracted service plus large cost savings as very much a good deal:

When we transferred across it was on the basis of our (existing IT) budget . . . They (the vendor staff) had up front clearly what we were using at the time of the out-sourcing deal. They were very thorough and did their homework. But it's a loss of advantage to us really. We no longer have the ability to demand flexibility of the data centre in the way it manages its cost. We can't force the vendor to find ways to do things on the cheap which is something we would have sat down and found a way of doing, and they feel under no obligation to do this.

6.6  Staffing: a vital resource

For sound evaluation practice it is vital to have the capabilities and skills in place to monitor and manage vendor performance. In their work on this subject, Feeny and Willcocks⁵ show that four core capabilities are needed in-house to set up and run an effective IT market sourcing strategy. Here we will illustrate these capabilities with examples from our research base. The first two capabilities are more directly about evaluation practice, while the latter two have evaluation tasks imbedded in the roles assigned:

Informed buying: ‘Managing the IT sourcing strategy which meets the interests of the business’

This involves analysis of the external market for IT services; selection of a sourcing strategy to meet business needs and technology issues; leadership of the tendering, contracting and service management processes. One informed buyer describes the role:

If you are a senior manager in the company and you want something done, you come to me and I will go outside, select the vendor and draw up the contract with the outsourcer, and if anything goes wrong it's my butt that gets kicked by you.

Informed buying means:

(1)  monitoring available services of external suppliers;

(2)  analysing the nature of the service requirement for the short and long term;

(3)  structuring the tendering process; and

(4)  overseeing contact negotiations.

Contract monitoring: ‘protecting the business's contractual position, current and future’

A major consequence of IT outsourcing complexity and the commercialization of the IT operation is the need for contract monitoring. The contract monitor ensures that the business position is protected at all times. The role involves holding suppliers to account against both existing service contracts and the developing performance standards of the services market. The capabilities and number of people needed can be considerable. Thus, in the on-going UK Inland Revenue-EDS total outsourcing contract, according to 2001 IT Director John Yard, the contract management team covered 14 disciplines and ran highly detailed monitoring mechanisms. In the on-going BAe-CSC deal in the defence industry, an IT manager in the company commented:

We need a significant number of people in-house to monitor vendor service performance. In one business unit alone we have 16 people working on contracts, six exclusively on the monitoring side.

The main tasks of contract monitoring are:

(1)  monitoring results against goals;

(2)  benchmarking existing contracts against developing market capability;

(3)  negotiating detailed amendments; and

(4)  identifying and protecting against potential precedents.

Contract facilitation: ‘ensuring the success of existing contracts for IT services’

Most IT outsourcing situations involve considerable complexity. Typically a large population of users within the business are receiving a variety of services from multiple suppliers (or supply points) under a set of detailed and lengthy contracts. The contract facilitator ensures that problems and conflicts are resolved fairly within what are usually long-term relationships.

In our experience, both users and vendors place high value on effective contract facilitators. The role arises for a variety of reasons:

(1)  to provide one stop shopping for the business user;

(2)  the vendor or user demands it;

(3)  multiple vendors need coordinating;

(4)  enables easier monitoring of usage and service; or

(5)  users may demand too much and incur excessive charges.

One contract facilitator noted: ‘They (users) have been bitten a few times when they have dealt directly with suppliers, and it's a service we can provide, so now we do.’

Vendor development: ‘identifying the potential added value of IT service suppliers’

The single most threatening aspect of IT outsourcing is the presence of substantial switching costs. To outsource successfully in the first place requires considerable organizational effort over an extended period of time. Subsequently to change suppliers may well require an equivalent effort. Hence, it is in the business interest to maximize the contribution of existing suppliers. The vendor developer is concerned with the long- term potential for suppliers to add value, creating the ‘win-win’ situations in which the supplier increases its revenues by providing services which increase business benefits. A major retail multinational has a number of ways of achieving this, including an annual formal meeting:

It's in both our interests to keep these things going and we formally, with our biggest suppliers, have a meeting once a year and these are done at very senior levels in both organizations, and that works very well.

Human resource challenges

In practice, the development of these capabilities presents a number of strong challenges for contemporary organizations. Often in our research these capabilities were found to be missing. A common tendency when outsourcing is initially to appoint a contract manager, whose responsibilities are conceived as some mix of the informed buying and contract monitoring roles. In one major bank, however:

I am not physically managing anyone in the data centre environment . . . but a lot of my time is being taken up as being not contract management but service relationship management . . . dealing with senior managers in the bank who are coming to me to explain service issues on a day-today basis. We are having to do lots of work we thought we had outsourced.

In fact the contract manager was being stretched across many needed capabilities, and the vendor development capability did not exist.

In these situations we found organizations developing and staffing the capabilities outlined above, but over the contract span in response to problems as they arose, rather than pro- actively. In many cases this was a response to the difficulties inherent in developing what in fact needs to be an in-house, high-performance vendor management group. According to Feeny and Willcocks, the people required to staff such a group need to be high performers with strong business orientation, interpersonal skills, as well as different degrees and types of technical know-how. These are not easy to find in the current skills market. Moreover, organizational human resource policies are often not set up to pay such people the (volatile) going rate, offer them the challenges they need and provide suitable career paths. However, these challenges cannot be shirked, because sound evaluation practice will depend on the staff, the processes they put in place and the staff's capability to improve continuously evaluation practices and, therefore, vendor performance.

6.7  Summary

The economics of IT outsourcing uncovered by our research suggests that organizations need to pursue in-house improvements first, identify full IT costs and establish performance benchmarks, pursue further in-house improvements, and only then make in-house versus outsourcing comparisons. If the outsourcing option is rejected initially, it needs to be revisited at regular intervals, not least because the reassessment can act as an external benchmark on in-house IT performance. However, in practice, we found a number of other objectives and interests often impacting this economic logic.

Organizations found it difficult to assess vendor against in- house bids on a comparable basis, especially where prior evaluation practice exhibited the kinds of weaknesses discussed in this and earlier chapters. However, research base experiences suggested that the time and effort spent on fully assessing in- house performance, and revamping measurement systems proved vital in feeding into more effective contracting. All too often outsourcing deals can be based on varying degrees of ‘voodoo economics’. Another finding was that in order to enable comparison with and assessment of vendor bids, this evaluation work is best done before any contracts are signed, even where a specific vendor has been chosen. Organizations cannot assume safely that vendor opportunism will not occur. From this perspective, we found the contract to have the central role in determining whether outsourcing expectations would be realized. Hidden costs in outsourcing arrangements were identified, and these were found to be most frequently the outcome of weak contracting.

It also became clear that even good contracting, based on detailed IT evaluation and supported by comprehensive service measures and reporting systems, did not stop problems arising during the course of the contract. This stresses again what emerged strongly from our research: the importance of active monitoring and management of the vendor. As one IT director remarked ruefully about a particularly difficult outsourcing experience, ‘the one definite thing I have learned is that it's not like ringing for room service’.

6.8  Key learning points

–  IT outsourcing offers a major opportunity to improve evaluation practices. Organizations not accepting the opportunity will be forced into making the changes or will have significantly disappointing outsourcing experiences.

–  Prior patterns of IT evaluation greatly influence the degree of difficulty experienced in entering and running an outsourcing arrangement. A number of significant concerns emerged, namely evaluating total IT contribution, identifying full costs, benchmarking and external comparisons, the role of charging systems, and the adoption of service levels agreements by the in-house operation.

–  Assessing the economics of a vendor bid is fraught with difficulty. A client organization needs to understand the likely sources of hidden costs and the centrality of the contract in monitoring and conditioning vendor performance. The ten lessons in bid economics need to be understood.

–  Measurement systems need to be carefully designed and staffed, ideally before the contract is finally signed.

–  Once outsourced, certain evaluation issues can be anticipated. Much effort will be needed to set up a measurement system. A cultural change in measurement will usually be required.

–  Guard against the possibility of vendor opportunism. Internal charging systems may create problems. Users may become more wary of the IT service. IT costs may become too transparent.

–  Staffing for improved evaluation practice provides often unanticipated challenges. In-house informed buying, contract monitoring, contract facilitation and vendor development capabilities are required. Attracting and retaining the high performers with the necessary distinctive skills, attitudes and behaviours presents traditional human resource policies with considerable problems in a fast-moving labour market.

6.9  Practical action guidelines

–  Understand where your organization is on the scale of assessment patterns in order to best prepare for a transition to outsourcing evaluation.

–  Follow the ten lessons of bid economics.

–  Ensure that your outsourcing contract is specific enough to document what was said in negotiations and flexible enough to deal with arising situations.

–  Limit the length of the contract (3–5 years is most typical).

–  Measure everything during the baseline period.

–  Commit the resources required to develop an adequate measuring system.

–  Develop service level measures.

–  Develop service level reports.

–  Specify escalation procedures.

–  Include cash penalties for non-performance.

–  Determine growth rates.

–  Adjust charges to changes in business, technology and volumes.

–  Ensure that the staff/roles required to support the contract exist, including in informed buying, contract monitoring, contract facilitation and vendor development.