Chapter 8. IT Vendor Disasters

 

‘What you spend years building, someone or something could destroy overnight. Build anyway.’

 
 --Mother Theresa

What is an IT vendor?

All organizations depend on a number of external and third-party agencies for hardware, software, telecom, support, consumables, spares, and other IT equipment. It is not possible to run any organization without having one or more IT vendors supporting some critical equipment or function. Selecting the right vendor is therefore of utmost importance to get timely support and assistance during all problems. For example, if an organization is heavily dependent on e-mail for its business, the vendor who supplies and supports the email software will be very critical to the organization’s business. If the e-mail vendor goes out of business then the organizations that have implemented those e-mail systems will not get any more upgrades or support. Suddenly a vendor’s disaster now becomes your disaster, as you were dependent on their services.

What is an IT vendor-related disaster?

A disaster occurring to a critical IT vendor is indirectly a disaster for any organization using the vendor’s products. For example, assume your organization has purchased a database application from a vendor, XYZ Databases Corp, to load all your critical financial and other information. If XYZ Databases Corp goes bankrupt, gets hit by some internal disaster or goes out of business, then your organization will be affected. Suddenly, there could be nobody to support, upgrade or troubleshoot the application. Such situations can be classified as IT vendor-related disasters.

How can organizations protect themselves against IT vendor-related disasters?

Organizations usually have no control over disasters relating to IT vendors, but they can minimize the effect by having more than one vendor for similar functions wherever possible. For example, organizations can buy hardware and networking equipment from different manufacturers and vendors or they can have equivalent software from multiple manufacturers for similar functions. In the event that an IT vendor does go out of business, organizations should be in a position to speedily switch over to equivalent alternative systems from other sources. Some of the key factors to be taken into account when choosing vendors are:

  • Reputation of the vendor and manufacturer

  • Availability of competitive products

  • Disaster recovery competency of the vendor

  • Availability of support from third party sources.

How does one prevent IT-vendor support disasters?

Most IT vendors just supply hardware and software. Some of them also provide basic support like first-time installation, troubleshooting, etc. A few of them provide detailed support and consultancy options. However, your organization should not depend entirely on one IT vendor for all advice and support. You should also have qualified and knowledgeable internal staff to verify and understand the business pros and cons of vendors’ recommendations. A vendor may not appreciate the business implications of his recommendations (see example). Organizations must understand that not all vendors may be qualified to give accurate business protection advice. For example, a vendor may view a hard disk crash on your critical server as a simple hard disk replacement issue, whereas you may see it as a bankruptcy and chaos signal for your business if that disk was not being backed up regularly.

Example . 

In another potential disaster case, there was a certain Novell Netware file server that had developed some freak freezing problems, and would often lock up. The server was the heart of the organization, and important files were stored on it. The IT staff within the organization did not know how to solve the problem without losing data, so a vendor was called in to have a look. To everyone’s shock, the vendor actually suggested reformatting and re-loading the Novell operating system to see if that would solve the problem. He did not understand the business implications of suggesting a reformatting or reloading on a live server containing valuable data without a bunch of prior precautions. Fortunately, reloading was not done and applying some minor upgrade files from the Novell website later solved the problem.

Should IT staff be outsourced?

Yes and no. It depends on the business’ management and how they view IT. Nowadays thousands of small to large organizations outsource their IT functions. Many view IT as a burden they can avoid and try to outsource it. Sometimes outsourcing is actually done with a herd mentality attitude – everybody is doing it, so we should too. Other organizations view IT as a core, essential function that cannot be outsourced for a variety of reasons. Either way, there are risks. Actually, for best results, IT should be a combination of internal employed staff and some outsourced staff. This is because many organizations become over-zealous and decide to outsource every IT function. This is when trouble starts. There will be nobody within the organization with the required technical expertise to verify or certify whether the outsourcing company is actually delivering what they have committed or promised. Although there are some cost advantages with outsourcing, a balanced approach has to be taken after considering various factors, eg:

  • Outsourcing decisions are usually based only on cost factors. Hence, the cheapest quote from a bunch of vendors will get the order. Six months down the line, or during renewal, if another vendor quotes US$50 less he will get the order.

  • Vendors may or may not have the expected loyalty, dedication and commitment to a serviced organization’s business functions.

  • Information security could become a serious issue.

  • Inadequate service level agreements (SLAs) – or none – can cause painful legal problems when there are more important IT service issues to be worrying about.

  • Outsourcing vendors usually rotate staff between different companies and smooth transitions are rare. For example, there could be a disconnection between the outsourced IT staff that were providing support between January to March, versus the new outsourced staff who will provide support between April to June.

    Example . 

    A certain Company A had outsourced all its computer and network services from a reputable networking Company B. The outsourcing contract was for one year. About half a dozen of Company B’s tech staff were fully involved in implementation of the computer and network equipment for Company A, so only those IT staff knew everything related to the IT infrastructure. After six months, Company B wanted to recall its staff to use them for another big client and offered to provide a different set of staff to Company A. The new staff had to start everything from scratch. They were not familiar with the setup, staff names, priorities, etc, and started learning afresh on the job, resulting in total chaos to Company A. Later, Company A finally ended up hiring the previous IT staff of Company B at much higher salaries than they were being paid by Company B.

  • The best staff of the outsourcing company will usually be placed in the best-paying client’s premises.

  • Information security, confidentiality, etc, will become a serious issue as non-company staff will have access to internal information.

  • During a change-over of outsourcing vendors, the handover of responsibilities from one vendor to another will always be a serious and troublesome issue. The outgoing vendor will usually not do a proper handover to the next vendor, as the account does not matter to them anymore.

  • Outsourced IT staff usually go strictly by the book or scope of contract, and will rarely be flexible without additional costs. For example, if there is a need for the outsourced IT staff to be present in the company after office hours or weekends for some urgent work, it will usually involve additional hourly costs.

  • Many companies think that by outsourcing work they are ridding themselves of the internal responsibility. That’s incorrect. Your organization must also monitor the work to ensure that the work is progressing as expected – even though you outsource the work, you can’t completely outsource your obligation to make sure everything is progressing smoothly. If all goes well with the outsourcer, you don’t have much work to do. But very often the outsourcer does not perform to your expectations, and then you have a bigger problem on your hands. Remember that vendor problems eventually become client problems.

  • Other factors, like location of the outsourcing company, travel distances, holidays, internal problems, etc, all affect a client organization.

What can be outsourced?

It depends on the nature of the organization and the availability or non-availability of certain skills in-house. Other factors like costs, logistics, security, etc, also play an important role. Usually defence and military establishments do not like to outsource anything and expect to have qualified in-house personnel to handle everything, except for certain functions that have no security impact. Some organizations may like to outsource everything as they feel they don’t have, or can’t afford to have, expensive, qualified IT staff on their payrolls. The following areas of work are outsourced in many companies today:

  • Desktop and server hardware support: An organization’s internal tech staff may not have the necessary skills to repair or replace various types of failed or new hardware. This can be outsourced and will mainly involve repairing or replacing failed hardware, setting up new hardware, etc. Depending on the speed necessary, spares and IT staff can be external or housed on-site.

  • Networking: When organizations grow or set up new offices, factories, etc, the entire place has to be wired with data and voice cables necessary for local and wide area networking. It does not make sense to have qualified in-house staff with those skills and the job can easily be outsourced to reputable vendors to wire up a building.

  • Turnkey projects: Assume that an organization wants to establish a branch office in a different location or city. The entire project of cabling, networking, installation of new equipment, power, etc, can be outsourced to a reputable organization that can complete and hand over the project for a fixed fee within a fixed timeframe.

Whatever the reason for outsourcing, or not, organizations must consider the availability of critical support, spares, talent, etc, required for ensuring disaster recovery and business continuity. Clear service level agreements outlining a detailed scope of work, expectations, roles, responsibilities, etc, must be enforced to cover all preventable risks.

Some questions to ask vendors

Here are some questions to ask, and get satisfactory answers to, when selecting vendors for critical equipment or services. When one or more vendors go bust, you must be able to quickly locate another vendor to maintain the service to your end-users as quickly as possible. It is always better to have more than one vendor for any product or service.

  • Does the vendor have enough trained support personnel to handle technical support?

  • Does every support person carry a mobile or a pager for contact during emergencies or otherwise?

  • Does the vendor have adequate stock of critical spares?

  • Does the vendor have a 24x7 support option?

  • Does the vendor have a DR or BC plan?

  • Can the vendor give some good references to verify and/or any other testimonials or certification?

Is it necessary to have contracts with vendors?

Absolutely. If you are using external vendors to support or maintain critical equipment and services it is absolutely necessary to have a proper contract or Service Level Agreement signed and agreed by both parties. The contract should be prepared in detail, covering the following:

  • Scope of work

  • Exclusions

  • Roles and responsibilities

  • Service hours

  • Duration of contract

  • Spares support

  • Reports to be provided

  • Payment terms

  • Penalties for non-adherence.

Contracts should be prepared with the support of staff from technical, financial and legal departments so that all aspects are properly covered and worded accurately. A contract must withstand scrutiny by lawyers or the courts, if necessary. In addition, a detailed technical service level agreement (SLA) is also necessary to ensure proper support. Periodic audits should be conducted to see that SLAs are being met.

What are the key elements of a maintenance contract or an SLA?

As mentioned before, it is necessary to have proper, written agreements with appropriate vendors, service providers, consultants, etc, responsible for maintaining critical services for an organization. Without a clear, signed agreement it is not possible to ensure or expect that the required assistance will be provided by external parties for essential activities in various situations.

A general purpose SLA will normally cover the points listed below.[2] Each point needs to be elaborated in clear and definitive terms for the area of coverage. Additional items can be added depending on the specific nature of work or industry.

  • Name of the project or area of support.

  • Contract number or reference number with date.

  • Start date and end date for contract.

  • Description of the project or work expected.

  • Parties to the agreement, including authorized persons, departments and workplace addresses.

  • Detailed scope of work.

  • Common obligations of both parties.

  • Out of scope (both parties).

  • Assumptions, constraints, risks and limitations.

  • Hardware, software, spares, other requirements.

  • Legal aspects, jurisdiction, non-disclosure clauses.

  • Financials, budgets, payment terms, penalties, additional costs, extra charges, taxes, billing methods, etc.

  • Standard working hours or service windows covering number of hours per day, holidays, etc.

  • Number of staff required on-site or on call.

  • Training requirements.

  • After-hours work, eg, weekend work, if any.

  • Help desk or support procedures, turn-around times for response, resolutions, work-arounds, etc.

  • Incident and problem management procedures.

  • Escalation procedures.

  • Change management procedures.

  • Reports and metrics (what standard reports will be exchanged).

  • Project termination clauses, notice periods for closure.

  • Signatures of authorized representatives from both parties.

Example . An IT service without a maintenance contract

IT Support: ‘Hello ABC Computer Company? We are calling from RockSolid Corp. One of our main server’s power supplies has failed. Can you replace it immediately?’

ABC Company: ‘Can you tell me the serial number of the server?’

IT Support: ‘It is QW1246.’

ABC Company: ‘Sorry, that server is out of warranty, and also not under any support maintenance agreement, so we will not be able to replace the power supply.’

Example . An IT service with a maintenance contract

IT Support: ‘Hello ABC Computer Company? We are calling from RockSolid Corp. One of our main server’s power supplies has failed. Can you replace it immediately?’

ABC Company: ‘Can you tell me the serial number of the server?’

IT Support: ‘It is QW1246.’

ABC Company: ‘Thank you for the details. That server is under our maintenance contract. We will replace the power supply within the next four hours.’



[2] There is also substantial support material, including SLA templates, available on www.itgovernance.co.uk/sla.aspx

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
52.15.231.106