The 12-Question Software Vendor Evaluation Test

Selecting the best software vendor for your customer relationship management automation project is no easy task. Software vendors come and go, many are backed by venture capitalists who can opt to pull the plug if financial returns are not met, technological changes often shake the very base on which vendors build their application, and so forth. With these potential difficulties in mind, ISM recommends that you ask each vendor that ends up on your vendor “short-list” the 12 questions that are detailed in this chapter.

Here are a couple of observations prior to the 12 questions. First, if you find that a short-listed vendor is evasive in responding to any of these 12 questions, continue to demand answers now rather than later. Second, do not rely on the answer to any one question as the basis for your vendor selection decision. Rather, get answers to all 12 questions, and then apply business judgement as to how well the vendor's 12 responses hold together in view of your specific needs.

Question #1: How long has the vendor been in business, and what is the history of their business?

The Customer Relationship Management automation industry celebrated its 22nd birthday in 2001. To the best of our knowledge, there are no CRM software vendors older than 18 years and most CRM software vendors are between 5–10 years old. Perhaps this makes the number of years in business less relevant, though we think not.

A software vendor like Firstwave, which began in 1984 as Brock Control Systems, has been through the good times and the bad times, through an initial public offering, as well as a number of organizational acquisitions and structural reorganizations. Over the years, Firstwave has proven itself a resilient CRM player, whose products consistently score high in independent user reviews such as ISM's The Guide To Customer Relationship Management (CRM) Automation. While we do not want to infer that being around for a long time is an essential characteristic of a solid CRM vendor, we do want to make the point that longevity has a value.

There are, nonetheless, dozens of outstanding CRM vendors, several of which are today's market leaders, who have been around 5–10 years, and whose CRM software offerings are excellent today and are likely to remain excellent in the future. For example, McGraw-Hill may have been seen as taking a risk when it became the first customer of YOUcentric (recently purchased by J.D. Edwards), a CRM vendor that was reasonably new to the CRM industry. McGraw-Hill was willing to take that risk with YOUcentric's Java-based Web CRM offering, and has not looked backwards ever since. So, despite some vendors having venture capital behind them or despite some vendors being brand new to the CRM industry (e.g., the emerging Web-based CRM vendors), there are no hard and fast rules concerning their long-term viability.

In fact CRM software vendors come and go whether they are established players (e.g., Sales Technologies decided to pull out of the CRM marketplace in the early 1990s) or new players (e.g., Corepoint, a CRM vendor backed by IBM in the late 1990s, never really got off the ground). Our conclusion is that years in business, while important, are not sufficient to determine whether the CRM vendor in question is appropriate for you.

Question #2: Do they have experience and customers in your particular industry?

Experience in your particular industry can be quite important. A software vendor who demonstrates to you their understanding of how your industry works, including, for example, your industry's best demonstrated practices within their software offering, may be a real plus. You should not, however, be overly swayed by a vendor's demonstration of industry-specific software, since creating impressive demonstrations using today's software tools is reasonably easy. More importantly, therefore, you will want to ask questions about their active customers in your industry, and whether they are willing to provide you with the names/contacts of these customers. Answers to these types of questions will allow you to conduct your own due diligence concerning how well the CRM vendor in question has met the needs of a live, industry-specific customer.

You may find yourself in a dilemma in that new CRM software vendors may not have specific industry experience, yet have software whose technology and flexibility seem ideal based on your specific needs (e.g., Talisma for customer contact center). Increasingly the industry is seeing vertical focus from CRM software vendors (e.g., Siebel in the pharmaceutical industry) and thus the need for experience and customers within your specific industry may soon become a moot point.

Question #3: What is their technological direction (e.g., Web strategy, CRM module expansion intentions)?

It is imperative that your CRM vendor's technological direction is known, and that this direction “fits” within your own company's technological direction. The best way to determine this is to hold a half-day technology session with the vendor's chief technical officer. The objective of this session is to determine where the vendor is headed (i.e., new business modules, new development tools, and the likely timeframes in question), to reveal as best as possible what your specific business functional and technical feature needs are, both today and in the future, and to come to an understanding as to whether you and the vendor see eye-to-eye. Don't be surprised if you are asked to sign a nondisclosure agreement, which is standard for these types of meetings.

Question #4: Who are the members of their management team, and what are their backgrounds?

The cumulative background and experience of a vendor's management team can provide some insight into the stability and credibility of the vendor within the CRM market. Ideally, you would want to see a mix of business discipline backgrounds (e.g., accounting, finance, information systems, operations) as well as industry background (e.g., healthcare, CPG, etc.) that is representative of the vendor's product focus and offerings. Also, educate yourself about the companies where the management team members were previously employed. Are those companies thriving? Are those companies no longer in existence?

Question #5: How are they financed?

It is very important to know the financial backing of a prospective CRM vendor. Venture capital groups finance many new companies in the high-tech area, particularly software companies. In many cases, the payback periods for the venture capital funds are very aggressive. In other words, the venture capitalists want to be paid back in a short period of time. Therefore, there is additional pressure on the software vendor to sell as many seats or licenses as it can, in as short a period of time as possible. In addition, if the venture capital organization has dictated an exit strategy by acquisition, there is increased pressure on the software vendor to do whatever it deems necessary to make the financials look attractive. In other words, in the process of reviewing the financial stability of the vendor, go beyond the financial statements.

Question #6: Is source code included with the product?

Increasingly, vendors are providing toolkits that allow the customer to make changes to the software without using third party implementation consultants. Much of the underlying code is programmed into reusable objects. Nevertheless, the ability to customize the CRM system without the help of vendor technicians or third-party consultants may be limited by the lack of provided source code. If the vendor does not provide the source code, you need to know why this is so. What happens if the vendor goes out of business? Do you get source code to ensure the continued functioning of your system? Pay close attention to the fine print about what is and what is not provided.

Question #7: What training do they offer (e.g., end-user, train-the-trainer, systems administrator training)?

Remember that over the life of the CRM system, the cost of training will probably exceed by 1 to 5 times the cost of the software itself. Does the vendor provide any training? Is the training provided in-house, on-site or online? Does the vendor pass the training responsibility on to the resellers or implementation partners? Regardless of whether the vendor offers some or all training programs, it is extremely important to ask how the vendor measures the performance of their own trainers and how they will measure the effectiveness of their own training methods, for resellers or the final end-users. Do they have structured plans for ongoing training, refresher training, and training for new hires? Is the vendor using the latest technologies to provide training services (i.e., Web-based training)?

Question #8: How do they support their software (e.g., do they have a guaranteed response time)?

Many vendors provide support services as part of some type of maintenance agreement. It is imperative that you ask what types of support are offered with a maintenance agreement (e.g., phone support, Web-based self service, on-site support). Some vendors are offering à-la-carte support programs like pay-per-incident or per-support interaction. Are the support services passed on from the vendor to the third-party implementation partner? If the support services are “outsourced” to a third-party company, what certification programs does the vendor have to teach, train, and certify third-party partner personnel?

Question #9: What is included in their maintenance agreement (e.g., fixed number of upgrades per year)?

The maintenance programs provided by CRM vendors vary greatly in cost and complexity. Be prepared to get specifics on what exactly is provided by the agreement, the timeframe of the agreement, and who will provide the service and support (the vendor and/or the vendor's implementation/service partner.) Ask the vendor how upgrades and updates will be made available (i.e., via download from a Web site, via a reseller). Will you have access to a dedicated technician or group of technicians, whether it is by phone, e-mail or Web site? Can the vendor guarantee a particular turnaround time for problem resolution? What kind of resources has the vendor devoted to the particular support/service options? These are all questions that should be asked.

Question #10: What is their warranty period, and what is their bug-fix policy during this period?

It is important to know what type of warranty policy a CRM vendor has for its product(s). Many vendors offer a standard 90-day warranty period. Be careful to ask the vendor what is included in this warranty and if it is exclusive of an annual maintenance agreement. It is absolutely appropriate to question the vendor as to how, when, and where any software “fixes,” updates and upgrades will be made available. In addition, you should question the vendor about the extension of the warranty and any costs associated with the extension.

Question #11: How do they implement their software (e.g., themselves, via a third-party implementer)?

Most of the CRM vendors that provide comprehensive suites of products employ the services of third party CRM implementation companies. If the vendor does the implementation using their own technical staff, how many consultants are dedicated to implementation efforts? You might want to know if the vendor provides any type of “rapid implementation” option and if so, is this really appropriate to your own business situation. Ask questions about the qualifications of the vendor's implementation staff and in the case of third-party implementation partners, the type of certification, and length of training required in becoming a vendor implementation partner. In the event that the vendor uses third-party implementation companies, ask how the vendor ensures the quality of the implementation company. Also, make sure that the vendor and/or the implementation partners are actually using the CRM software for their own front-office operations.

Question #12: How important to them is your piece of business?

Will you be an important customer to the vendor? Will the vendor commit the necessary resources to ensure the successful implementation of your CRM system? To ensure that the software vendor provides appropriate attention, some customers insist that they sit on the software vendor's management board of advisors! Regardless of how you do it, secure guarantees from the software vendor that it is prepared to commit the necessary resources to ensure the successful realization of your CRM project.