Chapter 15
Proposal Reviews
Proposal reviews are conducted during the proposal development phase to assess progress and make any necessary course corrections. Progress in this sense is not about how much you have accomplished but what you have accomplished. Is your proposal responsive to RFP requirements? Is your bid strategy being implemented effectively? Are your themes and discriminators clearly articulated and integrated into the proposal narrative? Is your proposal well written and organized?
Proposal reviews are intended to answer questions like these. They should also provide clear directions about how to remedy identified shortcomings. No one would expect to hike across Antarctica or through the Amazon without periodically checking to ensure that they were on course. Likewise, no one should expect to prepare a winning proposal without performing comparable course checks. So, the issue is not whether you will conduct proposal reviews, but how and when.
Gain competitive advantage by integrating timely, effective proposal reviews into your proposal process. Use review information to correct deficiencies and improve proposal quality. Those who stumble through the review process risk slipping behind the competition.
DIFFERENT TYPES OF REVIEWS
Proposal reviews are typically referred to by color. Different “color” reviews occur at different stages of proposal development. The best-known review is the red team review, which seems to get its name from those glory days when proposals were prepared on typewriters and evaluated in hard copy format. Proposal reviewers marked up the draft proposal with red pens—hence the term “red team.” (Some argue that the term “red” comes from the spilt blood of the proposal team.) Since then we have added other reviews, each with its own color designation. These various reviews are summarized below, although not everyone is consistent in their use of the color labels or in their review focus.
Storyboard Review
Storyboard, or what I call author guide, reviews are not assigned a color and are applicable only when storyboards are part of the proposal development process. Normally conducted by the proposal manager or technical volume manager, they consist of a check of completed author guides. These checks focus on verifying that the author has a solid outline to work from and one that clearly addresses all the requirements in the proposal requirements matrix.
Pink Team Review
Logic says that if there is a red team review, then a review before it should probably be called a pink team review. Pink team reviews generally are early course-correction reviews. They identify areas inconsistent with the basic proposal architecture and organization established by approved proposal outlines and author guides.
Pink team reviews also identify cases where proposal narrative is not responsive to RFP requirements. For those cases, they identify the deficiency and define the necessary corrective action. Pink teams usually avoid minor issues of proposal format, page count, or things likely to be corrected during the normal revision process.
Red Team Review
The red team review is a formal evaluation of your proposal. It should be conducted using the same evaluation methods and criteria that your customer will use to select the winner.
Red team reviewers evaluate your proposal for compliance with RFP instructions, requirements, and evaluation criteria. They also assess the extent to which you have effectively conveyed your proposal sales messages and bid strategies, and they check for the presence and substantiation of themes, clarity of text and artwork, and overall proposal persuasiveness.
Red team members identify proposal deficiencies and define corrective actions. They also list proposal strengths and weaknesses to help guide subsequent revision efforts.
Gold Team Review
The gold team (sometimes called the blue team) conducts a final review of the proposal. This is the last opportunity to identify areas for improvement and to make those improvements before the proposal is submitted. Typically, gold team membership is limited to just the proposal manager and a few hand-picked persons who have extensive proposal experience. They provide the final “polishing” to ensure that the proposal communicates the bid strategy clearly and effectively and is well written. In some cases, gold team members perform final proposal proofreading and quality assurance checks as well.
Black Hat Review
Black hat teams perform a detailed assessment of your major competitors. The purpose of a black hat review is to determine the most likely bid strategies of your competitors. In some cases this review might be in the form of a “straw man” proposal, intended to represent what your competitor is expected to submit. This information is then used to develop offsetting strategies. To be effective, black hat reviews must be conducted during the pre-proposal phase to allow enough time to use the results.
Conducting an effective black hat review is extremely difficult and expensive. Therefore, such reviews are restricted to strategically important contracts. They are typically conducted by outside consultants who possess in-depth knowledge of competitors or a consulting group that can investigate the competition without involving your company directly.
FORMAL VERSUS INFORMAL REVIEWS
Proposal reviews may be formal or informal. Formal proposal reviews are conducted by people who have not worked on the proposal. These people review the entire proposal at one time, formally document their findings, and debrief the whole proposal team and other interested parties on the results.
An informal review, or in-process review, is one conducted by a small team, sometimes just the proposal manager. It is conducted as individual proposal sections become available. The team gives feedback concerning deficiencies and corrective actions to individual authors on a section-by-section basis. For an in-process gold team review, the final changes may be made by the proposal manager or a small group appointed specifically for that purpose.
Determining which proposal reviews to conduct, as well as whether they should be formal reviews, depends on a host of variables. These include:
Time available to prepare your proposal
Number of draft proposals planned
Experience and capability of proposal capture team members to perform reviews
Proposal budget (if consultants will be used)
Importance or value of the contract
Availability of people to serve on formal review teams.
In some cases, pink, red, and gold team reviews are all conducted as formal reviews. However, this is usually the case only when contracts have a value in the multi-hundred-million- or billion-dollar range and when you have 60 to 90 days to prepare your proposal. More routine are cases where a formal pink and red team review are conducted, or only the red team review is a formal review.
For contracts with a 30- to 45-day response time, consider limiting your formal review process to the red team. Add a formal pink team review only when it can be completed prior to final RFP release or if the program is extremely important. Trying to cram too many reviews into the proposal development process will likely undermine proposal quality, not improve it. Proposal authors must have adequate time to develop and edit their sections.
Here is a typical scenario. After losing a string of proposals, or one very significant bid, management becomes convinced that the problem lies in failing to detect problems early in the development cycle. To avoid this, they schedule earlier reviews. Authors faced with insufficient time to prepare their sections then submit nonresponsive material for review. Management’s suspicions are confirmed. So, they schedule even earlier and more frequent reviews—with predictable results.
Proposal success, like most things in life, is a matter of balance. For proposal reviews, the balance is between giving authors enough time to complete their writing assignment effectively and conducting a timely assessment of proposal progress. As a baseline, I recommend that you plan to conduct informal author guide and pink team reviews, a formal red team review, and an informal gold team review. Ideally, this will include pink team reviews of a draft proposal written in response to a draft RFP and of the first draft proposal after receiving the final RFP; a red team review after the second draft has been edited; and a gold team review after red team corrections have been made. You can adjust the suggested baseline to accommodate individual circumstances.
INFORMAL AUTHOR GUIDE REVIEW
The purpose of an author guide review is to ensure that authors have a good foundation before they start writing the first draft. Either the proposal manager or the technical volume manager should check Sheet 1 of the author guide to verify that it is thorough and appears to address the required topics adequately. He or she should also check to ensure that the paragraph numbering and headings are consistent with the proposal outline. Once Sheet 2 is complete, it should be checked to ensure that themes are consistent with the bid strategy and that any suggested features, benefits, and risk mitigation strategies meet basic criteria.
IN-PROCESS PINK TEAM REVIEW
The purpose of initial or pink team reviews is to check the proposal to ensure that it complies with and is responsive to all RFP requirements. You are just trying to “bake the cake” at this stage. Therefore, do not worry about themes or your sales message yet.
The proposal manager or other capture team leaders can conduct initial in-process reviews. To check compliance, use the proposal requirements matrix discussed in Chapter 10. Recall that this matrix lists Section L preparation instructions and shows corresponding evaluation criteria and other relevant RFP requirements. The matrix conveniently provides all the information required to perform a compliance check. It saves reviewers the hassle of looking up all the relevant RFP sections. Moreover, it is the foundation for your proposal outline and author guides and therefore provides the perfect reference for assessing your first draft narrative.
The proposal requirements matrix is easily converted into a review guide by adding a column to each section of the matrix as shown in Figure 15-1.
Figure 15-1. Review Guide
Read each proposal section and compare it against the requirements contained in the proposal requirements matrix. Use the added columns in the review guide to record the results of your evaluation. I normally score requirements using a plus (+), checkmark, (√), or minus (–) to denote cases where the proposal narrative (1) exceeded requirements, (2) met requirements, or (3) was either nonresponsive to instructions or noncompliant with RFP requirements. Once you have read and scored a proposal section, document your results in an evaluation form like the one shown in Figure 15-2, or one of your own design or choosing. Another alternative I often use is to make an electronic copy of the proposal section. Then I embed my comments and recommendations within the proposal section using a different-colored font. This is efficient, and it works well for me.
Figure 15-2. Sample Red/Pink Team Evaluation Form
Once you have completed your review, provide the author with a copy of the filled-out review guide and copies of the evaluation forms, along with a marked-up copy of the proposal section, if applicable.
RED TEAM REVIEW
The red team review should always be conducted as a formal review. A properly conducted red team review can be an invaluable asset and an important ingredient of a winning proposal. A poorly conducted red team review can be worse than no red team review at all.
Conduct the red team review using the same methods, procedures, and evaluation criteria the government will use to evaluate your proposal. Evaluate your proposal at the subfactor level as defined in Section M of the RFP. If your customer assigns color scores as an evaluation method, use colors to score your proposal. If the customer uses a point system for evaluation, use a point system. If the customer will evaluate proposal risk, you also must assess proposal risk. Whatever the customer does is exactly what you should do. The value of the red team review results will be directly proportional to the extent to which you mirror your customer’s source selection process.
Of equal importance are the people assigned as members of the red team. In the mid-1980s, I was hired as the proposal director to establish and manage a proposal process for a company that was not winning much new business. Red teams consisted primarily of company vice presidents and directors, who were using the red team reviews to familiarize themselves with proposals they would eventually have to approve before they were submitted to the government. Unfortunately, they practically never read the RFP, did not understand government source selection, were frequently interrupted during the review, and generally gave terrible advice to the proposal team. To help solve this problem, I scheduled red team reviews on weekends and set as prerequisites for all red team members a thorough reading of the RFP and red team training. This solved the problem, and I have used this strategy ever since.
With few exceptions, the people who wrote the proposal should not be red team members. They know too much about the proposal and thus are not representative of government evaluators. In addition, to the extent that it is within your power, keep senior management off the red team. In general, they make terrible reviewers. They normally do not have enough time to prepare for the review properly or give it their full attention. Perhaps worse, because they are senior management, few people on the proposal team will be willing to oppose their advice, even if it is wrong.
Of course, there are exceptions to this general rule. Furthermore, if the president or vice president of your company wants to be a red team member, it could be career-limiting to refuse his or her request.
Organize the red team according to the organization of evaluation criteria—areas and factors—contained in Section M. This is how the government’s source selection will be organized. Have a separate group of red team evaluators for each evaluation factor and subfactor. If possible, have at least three people read and score each proposal section.
Some reviewers should be knowledgeable about the proposal area they will review, and a few should be experts. Whenever possible, use people with past red team experience who understand government source selection. An effective reviewer is someone who can read proposal material, compare it with the requirements, and give clear directions about how to fix shortcomings. If major subcontractors helped prepare the proposal, consider asking them to contribute a member to the red team.
The leader of the red team is responsible for training and organizing the team, structuring the review process, managing the review, assimilating review results, and preparing and presenting the red team debriefing to the proposal team. For small proposals, the proposal manager can serve as the red team leader. If it is within your budget, and justified by the value of the contract, I highly recommend you use an outside red team consultant as the red team leader.
For major proposals, consider having outside consultants lead the evaluation of key proposal factors. If the normal scope of your proposals does not justify an outside consultant, consider having one come in and evaluate your proposal review process. This can be a one-time event, which you can use to build an effective review process or modify the one you already have in place.
Whenever schedule constraints permit, I prefer to schedule red team reviews for weekends. Allow one full day to prepare and format the proposal for review. Red teams can be conducted in two days (three days for larger, complex proposals if you have the time). Schedule a debriefing session for the proposal team at either the end of the second day or the beginning of the third day.
Red team reviewers must be familiar with the RFP. Before the red team review begins, make sure all team members have read at least RFP Sections L and M, plus the statement of work and any other pertinent RFP attachments. Conduct a brief training session for red team members before they start their review, preferably the day before. Minimum topics for the training session include:
An overview of your customer’s source selection process—do not assume reviewers already know this
A schedule of red team events (review, assimilation of results, proposal team debriefing)
An overview of the red team review process
Instructions to reviewers about the purpose of the review and what you want the review to accomplish
A review of red team documents and evaluation forms and how they should be used.
Plan on about an hour to complete the training session.
Good planning and organization will contribute mightily to the effectiveness of your red team review. Provide each reviewer with a document package that includes the following:
Written instructions about how to conduct the review with definitions of review terms such as technical strengths and weaknesses
List of proposal sections to be reviewed, by reviewer assignment
Proposal requirements matrix
RFP Sections L and M (and access to a complete RFP)
Relevant author guides (Sheet 2)
Proposal evaluation forms.
Red Team Instructions
Provide reviewers a secure, quiet, comfortable work environment. Have an administrative or clerical person available to help the review team organize and collate evaluation forms and prepare the final proposal debriefing. If you are under a schedule crunch or reviewers expect to work long hours, plan to bring in nutritious meals for them. If your customer plans to evaluate your proposal electronically, I highly recommend that you conduct the red team review electronically.
Written instructions provide reviewers with specific direction about how to conduct the review. They state the review’s purpose and set some boundaries concerning appropriate ways to evaluate the proposal and provide feedback to authors. Reviewers need to be mindful of the hard work of the proposal team, regardless of how bad the proposal may appear. Derogatory, rude, or distasteful reviewer comments are uncalled for and should not be permitted. They only discourage and inflame those tasked to fix the problem.
The reviewer’s task is to identify deficiencies and areas for improvement and then clearly communicate that information to the author. No one likes a critic, but nearly everyone appreciates definitive instructions that will enable them to improve the quality of their product. That should be the focus of red team reviewers. Accordingly, corrective actions captured on evaluation forms should be specific and detailed. Otherwise, the author will be unable to determine the recommended solution to an identified deficiency. Comments like “This section is vague,” “What are you trying to say?” or “Fix it” will be of little help to the author. Ask reviewers to be specific in identifying proposal deficiencies and explicit in defining the necessary corrective actions.
Consider collecting past evaluation forms, both good and bad, to use as examples for red team training. Finally, ask reviewers to refrain from marking typographical, spelling, or grammatical errors. They need to focus on more substantive issues, and these types of errors are easily fixed during the next rewrite and edit cycle.
Conducting the Red Team Review
Equipped with the proposal requirements matrix and relevant author guides, the reviewers read the proposal section to determine if each requirement is adequately addressed. (The same review guide shown in Figure 15-1 can be used for the red team review.)
Each evaluator reads his or her assigned proposal section and compares it to the review guide and any associated author guides. The results of the review should be captured in an evaluation form like the one shown in Figure 15-2. Deficiencies are noted on the proposal evaluation form, along with the recommended corrective actions. In addition, the reviewer also checks to see if the themes, features/benefits, and risk mitigation strategies contained in the author guide made their way into the proposal. If not, or if they are ineffective, a deficiency should be noted.
Use a separate evaluation form for each issue or deficiency. This will help track the subsequent resolution of deficiencies. The evaluation form includes space to list proposal strengths and weaknesses and a space to identify and score proposal risk. As appropriate, evaluators also recommend changes to enhance the proposal with the purpose of transforming an acceptable section into a superior one.
For identified weaknesses, reviewers should provide their recommended corrections. Strengths are listed to ensure that they are preserved during the next revision cycle and to potentially enhance their effectiveness. The “control number” on Sheet 1 is used to keep track of deficiencies. Assign each reviewer a unique alpha identifier—A, B, C, etc. The reviewer assigns each deficiency a control number, using his or her alpha code as a prefix, and sequentially numbers each deficiency (e.g., A-1, A-2, A-3).
To complete their evaluation task, reviewers assign each proposal section a score according to the method the customer will use to evaluate proposals. Reviewers will need a form of some type to summarize their scores and risk ratings for each proposal section. Again, the left-hand column of the proposal requirements matrix can be adapted easily for this task, as illustrated by Figure 15-3. (Note: Different proposal requirements matrices were used to create Figures 15-1 and 15-3. For Figure 15-1, I wanted to show a review guide that included multiple RFP requirements; for Figure 15-3, I wanted to show how the matrix could be tailored to summarize evaluation results at the proposal paragraph level. In reality, the same matrix would be used to create both evaluation forms.)
Combine scores and risk ratings for individual proposal sections to arrive at a score for each technical evaluation factor and subfactor, as defined by Section M of the RFP. Remember to apply the relative importance weights of subfactors to determine factor scores. The final evaluation score and proposal risk rating should be the consensus of all the reviewers who scored the section. This is the score that will be presented to the proposal team at the red team debriefing.
Likewise, all the proposal strengths and weaknesses should be agreed upon by the relevant reviewers and listed for each evaluation subfactor. All the reviewers for each proposal section should collaborate and agree upon the corrective action contained in the evaluation forms. Otherwise, authors may receive inconsistent or contradictory directions about how to correct a noted deficiency.
Figure 15-3. Sample Evaluation Recording Form for One Subfactor
Debriefing of Red Team Results
Assemble the entire proposal team and summarize the results of the red team review. Start with an overview of the entire proposal evaluation and highlight any general concerns or areas that need to be addressed. Explain how the review was conducted and what the proposal scores and risk ratings mean. Then present scores and risk ratings for each evaluation area and factor. Work from general to specific. Provide a separate presentation for each subfactor, including a list of strengths and weaknesses. Figure 15-4 provides a sample evaluation form that can be used to summarize evaluation results for major proposal sections or for technical evaluation subfactors.
Figure 15-4. Red Team Summary Evaluation Form
Maintain a positive tone during the debriefing. The evaluation scores will speak for themselves, good or bad. Emphasize the actions necessary to correct the deficiencies of the proposal and transform it into a winner. Avoid language or comments that assign blame or otherwise demean the efforts of the proposal team. The debriefing should be an objective presentation of red team results with a clearly defined road map for recovery and improvement.
Once the red team has completed its evaluation, have the assigned administrative assistant make copies of all evaluation forms. Organize them according to proposal section and log them into a deficiency tracking form or electronic database using the control number as an identifier. Figure 15-5 is an example of a tracking form. Use the tracking form to record and track the resolution of each deficiency identified by the red team.
Figure 15-5. Sample Proposal Deficiency Tracking Form
After the red team debriefing, distribute copies of the evaluation forms to the relevant authors or persons assigned to respond to the evaluation results. Each author then reviews the evaluation forms and determines how to respond to the recommendations of the red team. The red team is a recommending body. It does not provide direction, nor should all of its recommendations be followed absolutely. If the author disagrees with a red team recommendation, the proposal manager determines the final course of action.
After reviewing the evaluation forms, each author builds a plan and schedule to revise his or her proposal sections accordingly. Plan to revise the proposal sections that require the least amount of rework first to leave more time available for the most difficult ones. Every evaluation deficiency must be resolved and recorded in the deficiency tracking form. Resolution can be to ignore the recommendation, incorporate it, or amend it. The proposal administrator should maintain the tracking form and report progress to the proposal manager. Tracking deficiency resolution provides traceability between the red team results and the next revision cycle. It also ensures that each deficiency has been addressed.
GOLD TEAM REVIEW
Except for major proposals with ample preparation time, gold team reviews should be conducted informally as individual proposal sections are revised. The proposal manager should conduct the gold team review, with assistance from selected members of the capture team. Its primary purpose is to verify that deficiencies discovered by the red team have been corrected and that themes, discriminators, and strengths are presented effectively. Any deficiencies or further improvements identified during the gold team review are usually corrected by the proposal manager and capture team during the final revision of the proposal. However, individual proposal authors can be used for this task provided they have the requisite proposal experience and writing skills.
Strategically positioned and effectively conducted proposal reviews offer an excellent opportunity to gain competitive advantage. Seize this advantage. Use in-process reviews to provide timely course correction information to proposal authors. Schedule and manage reviews to maximize the time available to correct deficiencies and improve proposal quality. Conduct well-structured red team reviews with qualified and trained evaluators.
Provide clear, definitive guidance on how to correct deficiencies and enhance proposal strengths. Track deficiency resolution to completion, and verify that all legitimate deficiencies have been corrected. Allocate proposal revision duties to optimize the individual talents and strengths of the proposal team. Use your best people to make the final revisions and put the finishing touch on the proposal.