CHAPTER 3: DEFINE THE PLAN
Planning is the critical first step in the assessment process. Having a solid plan for the assessment sets the pace for the assessment, and has many benefits which will ensure a successful assessment and maximize the value of the findings. Some of the benefits of proper planning are listed below:
• Clarity of objectives – the project assessment is being conducted based on a specific stakeholder request or set of goals and objectives. Therefore, it is important to document what these objectives are, and level-set the assessment approach based on meeting these objectives. For example, if the assessment is based on following corporation standards, then the focus may be on ensuring the right use of templates. Whereas an objective for successful delivery may look at the content within these deliverables, and how they are used to manage the project.
• Begin with the end in mind – by starting with determining the final package layout and structure, the assessment can be conducted in such a way as to facilitate the results in a manner which aligns to that structure. This is a much better approach than gathering a collection of information and then trying to figure out how to synthesize it into a cohesive report.
• Focus on high value information – the assessment lead, or team, should spend time, upfront, identifying what information, documents, or evidence will be gathered, as well as what team members to interview and what questions to ask them. Planning out what information to look for will enable the assessment to be focused on specific documents, and not having to sort through extraneous materials which will not contribute to the goals of the project review.
• Be efficient with time – with proper planning, the assessment can target key information, documents, and questions, which will then minimize the amount of iterations and follow up questions. Proper assessment planning will result in requiring significantly less time for the project team members involved in the assessment, as well as the people conducting the assessment.
• Manage expectations for timing – a detailed plan should include milestones, activities and durations. This can be used to manage the expectations of people being interviewed which should allow team members to plan accordingly for the time commitments. This can also help manage expectations for management, as to when to expect the report and recommendations.
• Perception and credibility – having a well thought out assessment plan conveys the perception that the team knows what they are doing, and that they have everything under control. Challenges, rework, and a perceived lack of structure on the assessment, will only cause the stakeholders to focus on the assessment team, and not the findings and recommendations of the review.
• Tracking and reporting – having a plan with the key activities allows the assessment team to track progress against that plan, and then report out to stakeholders on the status of the work.
• “Eating your own dog food.” – it may seem hypocritical for the assessment team to be reviewing the plans and structure of a project if they do not demonstrate those things in how the assessment is set up and conducted, which comes back to having credibility.
Before developing the plan it is essential to confirm the objectives and criteria of the assessment, so it may be helpful to develop a charter, or statement of work. This document should outline the objectives of the assessment, the team members who will be working on the assessment, the project areas being reviewed, the approach, the stakeholders requesting the assessment, and any expected outcomes. The plan should then be tailored around meeting these objectives and criteria.
3.1 Determine assessment approach
As part of the planning phase, the assessment team should determine how the assessment will be approached. By determining the approach upfront, this will allow the team to develop a plan, and organize the work in a way that will be easier to assess, and package, later in the process. There are several types of approaches which can be taken to gather, assess and report on the review findings.
Risk assessment
This type of assessment focuses on the identification of project delivery risks. This approach can be subjective, and focuses on the perceptions and observations of the team members being interviewed, as well as the assessors. Risks can also be gathered from reviewing project documents, such as risk logs and issues logs, but generally interviews are a more effective means of soliciting risks.
This is a good approach for using with troubled projects, or high risk projects, as it focuses on those areas which are the most significant to project delivery. It also looks at the impact of those risks, and makes suggestions to mitigate the risks and improve the chances of successful project delivery.
Gap assessment
This type of assessment starts with a list of expected activities or deliverables, and then compares the project to that list to identify gaps. In this case, the assessment team could start with a corporation framework checklist and determine if the project is using the appropriate documents and standards. This approach is most effective for a framework or SDLC assessment, to confirm that standards have been followed. For example, the team would look to confirm that the project has all the appropriate project management documentation, such as a plan, risk log, issue log and resource inventory, and that the proper SDLC templates are used for requirements, design and testing.
Scoring
A scoring assessment compares the project to a specific set of benchmarks. This is meant to be more quantitative. For example, a rating of “high”, “medium”, or “low” could be given to score the adherence to standards, or level of adequacy that a particular project function is being managed. Graphical representations, such as Harvey Balls, could also be used to highlight how well defined a particular project function is (e.g. partial shading for not well defined and full shading for well defined).
Resource competency
In some cases, the project leadership and sponsors may ask the assessment team to review the competency of key resources on the project. For example, they may want to assess how well the project manager is running the project, or how certain leads on the teams are performing in their roles. Project success is as much a function of the competency of the resources on the team, as it is a function of processes and methodologies used, so the assessment team should pay attention to resource capabilities.
Some assessments may be conducted after the project has completed, to look for best practices which can be applied to other projects, or the corporation standard framework. The review can also facilitate lessons learned from this project which can be applied back to future projects and other organizational processes.
Note that project assessments can use all of these methods for approaching the assessment but it is a good idea to think through these first. By determining how the information will be analyzed and packaged, the assessment team can tailor the plan, approach, interview questions and document gathering approach accordingly.
3.2 Develop the plan
Based on the assessment approach and objectives, the plan should be developed. The plan should include all of the activities required to complete the assessment, along with their associated duration, and the name of the person accountable for the activity. Activity dependencies should also be captured in the plan. For example, there are some activities which can be concurrent, such as collecting documents and interviewing team members. Some activities are sequential and dependent on each other, such as presenting the final report after the analysis has been made and the findings have been packaged.
In creating the plan, the assessor should leverage existing materials to identify and confirm the activities required. For example, historical assessments may provide a good starting point for developing the plan, since they will already have many of the key activities and lessons learned. Also, corporations sometimes have standards for conducting assessments which may include additional activities to consider and plan for, such as governance reviews. Lastly, it is important to contemplate and document any logistics required to conduct the project assessment in the plan.
It is important to list out all activities and specifics of the assessment so that they can be tracked and communicated. In large project assessments there may be dozens of documents to review and many team members to interview, so it is important to be able to see which activities are outstanding and not yet completed. This progress can then be reported on during regular meetings with the project stakeholders which should be documented and agreed to in the communication plan. Some stakeholders may also want early indications of findings, and the team should discuss these expectations so that it is clear what is being reported, when, and in what forum.
The following list is an example of a high level plan for conducting an IT project assessment, organized by the steps in the assessment process and including some of the key activities to perform.
• Planning
Create charter or statement of work
Collect historical information on similar assessments or benchmarks
Collect assessment tools and corporation standards for assessments
Confirm skills needed and assessment team
Determine location for assessment working files
Determine roles to be interviewed
Obtain list of names of people for each role requested
Create list of questions to ask, by role
Determine approach for capturing information from documents and team member interviews
Confirm logistics for interviews (e.g. phone, meeting room, etc.)
Obtain corporation frameworks and standards
Create list of documents and areas to focus on
Obtain access to project documents
Develop the plan of all activities to undertake in the assessment
Determine estimate of hours required to perform the assessment
Develop communications plan for stakeholders
Announce the assessment with applicable information, such as intent, timing and key activities
Schedule and conduct kick off to review the objectives, scope, timing and involvement for the assessment
• Collect information
Conduct team member interviews (list out each interview by name or role, so they can be tracked)
Collect and review project documents (list out by area and document, so they can be tracked)
• Assess and recommend
Analyze findings from collected information
Document themes, gaps and observations
Conduct research necessary to score findings or assess the impact of the risks
Determine recommendations
Review findings
• Package and present
Document high level findings or executive summary
Document detailed findings with examples
Document recommendations
Package report with key project information and supplemental documentation, as appropriate
Present results
Schedule follow ups and next steps
3.3 Prepare the inventories
An important step in the planning process is to think through, and list out, the team members to interview, the questions to ask them, the inventory of project documents to review, and what to look for in those documents. Spending the time upfront preparing these inventories will ensure a smooth project assessment with minimal rework and follow ups, as well as attention on the most important parts of the project during the assessment. It is essential to ensure that the interviews and documents are focused on those areas which are identified in the objectives of the assessment. This will enable the assessment to meet its goals and be efficient with the time, resources, and materials being spent on it.
Planning for the interviews should start with the identification of which project roles will be reviewed, based on the stated objectives of the assessment and determined approach. Table 7.1 (page 197) lists out and describes the typical roles on an IT project. Once the targeted project roles are identified to be interviewed, then specific team member names can be gathered for each of these roles. This becomes the inventory of project team members and roles to be interviewed. Next, the assessment team should determine what questions are to be asked of each team member and role. These questions should be specific, target the objectives, and seek to get examples whenever possible. There are several types of questions which could be asked during the assessment interviewing process:
• General feeling – ask about risks or overall view of the project, to leave the discussion open-ended for the interviewee. This could include asking for any recommendations or challenges that the person can identify. These questions allow the assessment team to understand the perceptions and morale of the team, as well as any risks which may not be captured in project documentation.
• Role specific – questions that relate to the project role of the team member being interviewed and their ability to perform their job including processes, tools and interactions with other team members and roles. For example, if the team is interviewing a business analyst, the questions could relate to the collection of business requirements and obtaining business sign offs.
• Function specific – questions that relate to one of the project management or software delivery lifecycle functions that are being assessed which could include processes, tools, role interactions or best practices. For example, asking specific questions related to the schedule management, resource management, or vendor management practices on the project.
• Document specific – as project documents get reviewed, questions or clarifications may arise which could be asked during the appropriate interview. For example, the team could ask the project manager questions about the current status report, or numbers from a financial worksheet.
An inventory of documents should also be developed to identify which project documents will be asked for, gathered and reviewed during the assessment. This document inventory can also be used as a means to track which files have been received, which documents are still outstanding, and the status of the document in the assessment process. Tables 7.14 through 7.22 (pages 209–215) provide a sample of documents for each project functional area which should be considered for the document inventory, as well as some key areas to review for each. It may be helpful to create this inventory of documents and focus areas as a template, to provide the project team with a list of documents that are being requested and reviewed, and have them prepare the list as part of the planning process.
3.4 Plan for logistics
Lastly, once the assessment plan is developed and the inventories of documents and interviews are created, the assessment logistics need to be finalized and planned for. There are many logistics to prepare, and these activities may seem somewhat administrative in nature, but proper logistics can be the difference between a smooth assessment and a rough one. For example, not having meeting rooms for interviews, or not having access to project documents on a locked site, can slow down the assessment process. Below are some logistics to consider and prepare for when planning the project assessment.
• Project assessment
Have a central repository for assessment files where the review team can access documents in an organized structure.
Create a contact list for assessment team members with key contact information, such as email and phone numbers.
Consider having security on particular assessment files, if appropriate. For example, password protect key documents which may contain financial information or candid interview feedback.
Access and IDs for assessment team members if not part of the corporation where the project is being assessed.
Schedule a kickoff meeting with project team members to review the objectives, approach and plan.
• Project documents
Determine the approach for storing and retrieving project documents which could include a separate repository.
Obtain access to file shares for team members, as some sites may require users to be set up to view documents.
Understand email protocols, as some corporations have limits on the size of attachments which can be sent, or security around specific information contained within emails or documents.
Determine the location to conduct team member interviews which could include meeting at the project location site, a conference room at the corporation, or a separate location away from the corporation.
Schedule meeting rooms if conducting team member interviews at the location of the project.
Confirm access to the building for the interviews, or that someone will meet you to let you in.
Ensure that meeting invitations get sent so people being interviewed know the time and location.
Have dial-in numbers for conference calls and include them in meeting invitations for team member interviews.
Once the plans have been developed, including planning for interviewing team members, reviewing documents and logistics, the assessment can begin. Before the collection of information can be started, several initial activities are important to ensure a smooth start to the assessment. First, is that a message should come out from a senior executive announcing the assessment, along with the intent, timing and activities. This will provide the assessment team with the right level of support as they start reaching out to team members for information. Secondly, an initial kickoff meeting should be conducted with key team members to walk through the details of the assessment and set expectations for timing, who will be interviewed, what documents will be reviewed, and the key milestones for the assessment, including the timing of the final report.