2007 Apr 01
Software Selection Methods
David M. Raab
DM Review
April – May 2007
When most businesspeople think about buying software, sale they imagine a structured process: define requirements, identify qualified vendors, send them a Request for Proposal (RFP), research the most promising respondents in more detail, and make a choice. It’s somewhat bureaucratic, perhaps even ritualized. But it does promise a thorough, objective process that should yield the sound result.

Vendors hate it.

It’s not precisely that vendors dislike competition, although none would turn down a non-competitive sale. It’s more that they feel the standard process doesn’t let them learn enough about prospects to propose a solution that really meets their needs. They also argue that structured interactions prevent clients from understanding what truly makes each vendor special—which often has more to do with people and processes than technology.

But most of vendor ire is directed at the RFP itself. A serious RFP can run to scores or even hundreds of pages, requiring detailed answers to questions with little obvious bearing on a customer’s true needs. Often these needs themselves are not stated clearly enough in the document to allow relevant answers. And the RFP is frequently sent with little preliminary contact, leaving the vendor unable to judge whether it is being considered seriously or is just “column fodder” included to make the process appear comprehensive.

Yet RFPs do serve a purpose. In fact, they serve several. One is simply to meet the bureaucratic requirements of an organization. In any company bigger than about five people, the product research will be carried out by someone other than the person who signs the check. That person needs to know the selection process was run properly. The RFP and resulting responses show what questions were asked and how vendors replied. Decision makers can review this easily to assure themselves that the recommended solution makes sense.

A second value of RFPs is that they really do force vendors to provide complete information. It’s pitifully easy for a good salesperson to focus prospects’ attention on strengths and distract them from weaknesses. The RFP gives a thorough list of questions and answers, which buyers can review at their leisure to identify topics that merit further exploration. The proposal resulting from an RFP is also often the only place to get specific pricing from a vendor—although most experienced buyers consider this just a starting point for negotiation.

But the greatest value of an RFP lies in its preparation. A good RFP includes a detailed description of the company’s business situation, technical environment, project goals, and specific requirements. Assuming the company or its consultant is already familiar with the available vendors and functions of the system being purchased, most of the work in preparing an RFP lies in gathering the company information and defining an appropriate solution. Done properly and in enough detail, this provides vendors with the information they need to bid intelligently and greatly shortens the discovery process at the start of implementation.

How much detail is enough? Your best guide is to think like a vendor. Try to provide all the information that someone unfamiliar with your company would need to understand your situation and propose a suitable solution. This begins with a general overview of your business, going into more detail in the areas that are relevant to the project at hand. Describe the major components of your technology infrastructure and any relevant standards or constraints. Give additional details about the systems that relate to the project at hand: data volumes, update cycles, file structures, and so on.

It may seem odd not to start the RFP with the project objectives. But it really makes more sense to first tell vendors enough about your company that they can understand why the project is needed, where it fits into the larger scheme of things, and how best they can help you. You want their best thinking on your situation, not just responses to questions about their product. You also want the vendor to judge whether they are really a good fit for the project. If not, having them bow out sooner rather than later is in everyone’s interests.

After project objectives are defined, you can jump into specific requirements: product functionality, processing windows, numbers and types of users, target response times, and so on. Again, you should be as specific as possible. Include an appendix with sample files, process flows and reports if these are available and relevant. Define any specific constraints such as particular operating systems, security features or on-premise deployment. You also need to describe the project schedule and how you want to divide work between the vendor’s resources and your own during implementation. A complete RFP also asks about vendor background, professional services available, support policies, contract terms, and of course pricing.

In other words, an RFP is much more than list of questions for a vendor. It’s really a tool to organize your own information and present it in a way that lets the vendor help you. This means preparing a proper RFP is a lot of work. Fortunately, you don’t always need to do it. I’ll discuss some alternative approaches to system selection next month.

* * *

Last month’s column described the information needed to make a Request for Proposal (RFP) useful to both the company issuing it and the vendors who must respond. But RFPs and the formalized process they represent are not needed in all purchasing situations. Several other approaches are available. Which approach is chosen depends on mostly on two factors: how well the company understands its requirements and how familiar it is with the potential vendors. These are summarized in figure 1.


little known

well known


well known


informal review of key questions

little known


trial or prototype

figure 1

– when requirements are well known but vendors are not, it’s sometimes possible to replace a formal RFP with a “bake off” during which several potential vendors demonstrate how they would solve an actual problem. The trick here is selecting the right problem: it must include the critical issues facing the buyer but be contained enough to address in a reasonable amount of time. This approach most often makes sense when buyers are replacing an existing system, since they will know exactly what to look for and suitable test data will probably be available. Of course, some preliminary due diligence is still required to identify the most appropriate vendors, and some additional research will be needed after the bake-off to ensure that apparent winner is suitable in other ways as well.

One additional caveat is that this approach can favor vendors with quick deployment times: sometimes most suitable solution will do poorly in a quick test because it takes more initial tuning to reach its potential. If the tuning is just done once, it should not be overweighted in the final decision.

– when requirements are poorly defined but the vendors are known, it may be worth postponing the final decision and instead setting up a trial or prototype to gain some additional experience. Many analytical and business intelligence vendors in particular offer these sorts of programs. The goal is to get a clearer picture of how you might actually work with the system, what it can and can’t do for you, and what non-system issues you’ll face such as data quality, staff skills, and end-user expectations. It may be necessary to invest in a little staff training on the trial product, but this itself adds knowledge that can later be used in defining general requirements.

There’s some danger that you’ll fall in love with the trial system and never get around to considering other alternatives—but if you do find something that works well on the first try, what’s so bad about that? A more likely outcome is you’ll learn what’s hard, what’s easy and what’s important, and use that to guide your assessment of the competitive products, possibly using the bake-off approach described above.

– when requirements and vendors are both well known, you may be able to skip much of the structured process and make a choice after limited interaction with the leading candidates. This may seem like such a risky strategy that you would never employ it. But consider several situations:

– you have such simple needs that any major vendor is certain to meet them. This might apply to near-commodity products such as a small business accounting package or email broadcast system.

– the risk is low enough that you can afford to make a mistake. Risk includes the investment in the product and the cost of having it not work. Open source technical utilities often fall into this category: they’re nearly free and if they don’t work you’ll find out right away and try another. It can be cheaper to work this way than conduct a lengthy, formal evaluation.

– you have access to detailed information based on a previous experience. You may have hired a consultant who specializes in this area or know someone who has recently evaluated similar systems and is willing to share the information they gathered. You have to make sure they were looking at the same requirements, but assuming they were, their information should let you to form detailed judgments of candidate vendors without gathering it again yourself.

In each of these cases, you’ll still need some research to be sure the selected vendor can really do what you want. But with enough pre-existing knowledge about vendors’ capabilities and your own needs, you can move directly to very specific questions to resolve your uncertainties and then make a sound decision.

– when both requirements and vendors are not known, you might still be able to use a trial or prototype to improve your understanding of your requirements. But for projects that exceed a certain threshold of complexity, a trial won’t teach you enough to be useful. In these cases, your best bet is the traditional structured process and a formal RFP.

When deciding which method to choose, bear in mind that running a great selection project is not an end in itself. Your object is to find a system that works. Do that as quickly and efficiently as possible, and then you can move on to the real goal, which is deploying a system that benefits your organization.

* * *
David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.

Leave a Reply

You must be logged in to post a comment.