Identifying the best value proposal: The challenge of vendor selection

Identifying the best value proposal: The challenge of vendor selection

William B. Reid
6110 Bayou Grande Blvd., NE
St. Petersburg, FL 33703-1804

Abstract
The evaluation of proposals for geospatial projects is a challenging proposition. The objective is to select the proposal that yields the best value for the project. Persuasive low pricing can result in a failure to meet project expectations and goals. This paper will introduce and explain successful techniques that can be used in the selection process to level the field of competitors and to identify the best value proposal based on solid, objective, and time-proven criteria.

Introduction
The selection of a vendor to provide a service for a GIS project is a significant event in the implementation of a project and can have a major impact on the success of the project. A structured approach to procurement is crucial to the orderly selection of the vendor who offers the best value to the project. The process begins with a Request For Information (RFI), followed by the receipt of Statements of Qualifications (SOQ), a series of reference checks, the selection of a group of vendors who are considered to be qualified (usually 3 to 6), and then the issuance of a Request For Proposals (RFP). Our focus will now be on the identification of the proposal (vendor) that represents the best value to your project.

Evaluation of Proposals

Evaluation Team
The evaluation of the proposals should involve a team of people (generally 4 to 6). This team will typically consist of representatives from the user and purchasing organizations as well as members of the project implementation team. Each member of the team should read and score the proposals independently.

Content
The Request for Proposals should be checked carefully so that information is not requested in more than one location in the proposals. Whenever possible, questions should be phrased to require a statement answer that explains how the task will be accomplished rather than to list the solution and ask the respondent to say that they will comply. However, the RFP should ask the respondent to give specific answers to crucial questions. In a recent example, the RFP said that a Performance Bond would be required from the selected vendor. This statement was in a lengthy legal “Terms and Conditions” section of the RFP that asked the respondents to list those items that they could not comply with. Only one of four respondents took exception to any part of this section. However, when the selected vendor was asked to provide the Performance Bond, he stated that he had never intended to provide one and essentially he was unable to comply with this item. But this did not become evident until all of the other bidders had been notified if the selection.

Scoring
The RFP should also clearly define the format that is to be followed in the preparation of the proposals to be submitted and should tell responders that they will loose points in the evaluation if they do not follow the requested format. Adherence to a consistent format simplifies the use of a spreadsheet for scoring the proposals. Each proposal evaluator should be given a copy of the spreadsheet to use in scoring each proposal as it is read. The evaluators should decide on the weight to be given to each section and subsection of the proposal. The team should also agree on the point range (i.e., 1-5, or 1-10) that is to be used in scoring each item and the point penalty that will be assessed to any section or subsection that is not answered in the correct place. When all evaluators have scored each of the proposals the individual ratings should be combined into one composite spreadsheet. The scores can either be averaged for each section and subsection or the team can meet and discuss each section and subsection to come up with a consensus score for each item. Once the scoring is complete the weighting of the various sections can be changed in the spreadsheet so that you can see if a heavier weight on one section such as “Technical” or “Company and Financial Background” will change the ranking of the vendors.

Reference Checks
The next major activity is the conducting of interviews to check each vendor’s references. These reference checks are usually best conducted face to face if you have the time and resources to travel to visit the references. Remember that most people do like to publicly admit that they made a bad decision or selection. During this reference check phase you are trying to learn the good as well as the “not so good” about every vendor. Sometimes it is helpful to extend these interviews into an informal environment such as lunch or dinner.

You should ask detailed specific questions about the personnel assigned to their project by the vendor and how easy or difficult the vendor was to do business with. You should get specific information about the quality of the deliveries made by the vendor. This information should include the accuracy specified in the contract, the accuracy of each delivery, the turn around time for each rejected delivery, the number of deliveries on a one for one basis, the number of deliveries expected (allowing for some rejections), and the actual number of deliveries. Similarly, you should ask about the adherence to schedule. Was the pilot and each subsequent delivery submitted on schedule. What was the time interval between when deliveries were due to be accepted and when they were actually accepted (in days or weeks)? You should also question the degree of difficulty associated with coming to agreement with the vendor on the contract. Some companies tend to ignore difficulties with the “Ts & Cs” portion of RFPs and then expect to request a number of changes when they are selected to negotiate a contract.

You should also ask whether a Performance Bond was requested and what the outcome of that request was. Another good question is how the final amount amount of money paid to the vendor compares with the original quoted price and whether or not there were significant changes made to the work scope. If the work scope was changed, it is usually good to try to explore what the changes were and why they were made.

Vendor Presentations
The vendors who have submitted proposals should be asked to make presentations about how they plan to accomplish your project. The vendors should be required to have their proposed project manager make the presentation on their proposed technical plan. It is desirable to have other key members of the proposed project team at this presentation. This will help you to see how well the proposed project manager and his or her team understand your project. It is desirable to provide some specific questions and topics that your team would like to have addressed. In addition to gathering additional information, you will also get a chance to see if the vendor will methodically address your concerns and follow your instructions. This will also give your project team a chance to interact with the vendor’s project team so that you can get some idea as to how the personalities will interact.

Site Visits
At some point in the evaluation process the evaluation team should visit the vendor production locations. This will provide an opportunity to see how the production organization is organized and how accessible it is. When making a site visit, individual team members should be assigned “roles” in order to help you to get a consistent evaluation. For example, you might assign one person to ask questions on one general topic while another lags behind to see what happens as the group moves on.

Project Justification
Another activity that should take place during the period of time when the respondents are preparing their proposals is a review of how the project was justified and the ramifications that this has on scheduling.

CEOs of organizations don’t like projects (large one time expenditures) unless they will significantly improve the processes (the on-going day-to-day operations of the organization). Thus projects which can be justified as either “strategic” (“We need it to run the organization the way that we want to and to stay competitive”) or “tactical” (“The cost is justified by the reduction in on-going expenditures that it will bring”) are almost always justified as “tactical.”

Thus there are specific benefits that our project is supposed to provide and there are expected to be cost reductions or savings that will begin to accrue as each project delivery is accepted and brought online. Therefore, there are specific “lost benefit dollars” that occur if your project does not stay on schedule.

For this discussion, let’s assume that you have budgeted $ 4 million dollars for the work covered by this RFP and that you are expected to deliver a completed project with all data on line and up to date within 2 years. Furthermore, let’s assume that the payback period for the project is 4 years and that you expect to divide the work so that there will be 18 production delivery batches.

Let’s also assume that you have estimated your cost for checking and accepting each delivery batch at $15,000. for each delivery and that you have allowed for 4 “re-deliveries” as you prepared your project justification.

Price Quotes
Up to this point we have not talked about price or the price quotes. The evaluation process should have sorted out any vendors who do not appear to be appropriate for the current project leaving a group who are generally capable of accomplishing the desired results. It is important that the project justification be reviewed and that the reference checks are both completed before the prices (bids) are revealed so that objectivity is maintained. These “Actual Quoted Prices” may not represent the final cost to the project which we will refer to as the “Probable Real Cost.”

Let us suppose that vendor “A” provides a quotation of $ 3,400,000, while vendor “B” quotes $ 3,600,000, and vendor “C” quotes $ 3,800,000. With the high and low quotations being within 6% on either side of the median bid we can feel reasonably comfortable that our RFP and it’s specifications have not left a significant number of unknowns that resulted in a big price range.

If the range of the prices had been over 10% from the median, or in one price had been extremely high or low, there would be cause to eliminate the out of range prices as being suspicious. Now we turn to the information that we gained during the review of our project justification and during our reference checks on each vendor. We learned that vendor “A” averaged 6 months late on all deliveries and that the quality was such that they averaged an extra 1.5 deliveries for each batch, thus it would be reasonable for us to conclude that they might need to make a total of 27 extra deliveries on our project. The references on vendor “B” indicated that required an extra delivery on about half of the batches on their previous projects and that they averaged 3 months behind the original schedule on all of their deliveries.

Vendor “C” was said to be on schedule or within a week of the schedule across their deliveries and seemed to need to make redeliveries on about 20% of their batches. The following table summarizes the implication of this information.

The Actual Quoted Prices are adjusted by adding a probable cost for “lost savings” due to late deliveries and an additional amount for the cost of checking re-deliveries. The Probable Real Costs as shown in the bold numbers reverse the low to high order of the three vendors. You may also want to adjust the Actual Quoted Price for other items such as expected additional legal expense to negotiate the contract, extra cost if a Performance Bond was requested but cannot be provided, or probable scope change requests.

In the example given above we have a clear case of the comparison of “high quality versus low cost” and we see in this example that the high quality of vendor “C,” as evidenced by the information gained from the reference checks, has a value that out weighs the lower quotes from the other two respondents. This information needs to be combined with, and factored into, the rest of the information in the evaluation process. Careful use of a structured process as described here combined with careful, complete reference checks can go a long way toward insuring that you make the best value selection for your project.