It is easy to be misled when working with a small sample of data. This can be a serious issue when quoting a fixed fee for a big project if the quote is based on facts and figures that turn out to be inaccurate or irrelevant. Some factors that should be considered when calculating prices for quotes are:
- Experience of working on an individual project may not be a true reflection of how easy or difficult it is to work with a client.
- Different types of project have different levels of difficulty to implement.
- A client may have better skills in some areas than others.
- Client staff may move jobs and be replaced by new people throughout the project, resulting in whole new values generated for the client rating.
- The client's personality, commitment, competence and contribution skills may change over time—leading to an improvement or deterioration in efficiency of the working relationship and the decreased accuracy of the client rating data you have previously collected.
With the above in mind, it is possible to use more advanced methods to calculate the client rating and their resulting job quotes that better reflect evolving work relationships and the differing complexities of projects. These calculations, especially those for bigger projects, rely on the knowledge that:
- All projects are made of a series of smaller phases, hence each phase should have an associated cost to deliver.
Where the client insists on a quote for a project considered to be large and/or complicated, your risk can be reduced by providing an estimate for the entire project and quotes for individual phases as they occur.
- Accurate estimations are dependent on accurate client ratings and realistic perceptions of the ideal time required to complete a job.
Avoid the temptation to be overly optimistic, especially if you are trying to undercut the competition to offer a bid with the most attractive price, or where the client is pressuring you to commit to timescales that are unrealistic.
- The working relationship with the client is rarely either easy or difficult.
Clients will be easier to work with for some types of tasks and more difficult to work with in others. Avoid making assumptions about the client based on previous experiences of working with them that are not similar to the current type of task you are estimating.
With the client rating being the critical element in adjusting estimates to account for known factors relating to the client, the key to maintaining accuracy of estimates is there largely to do with how client ratings are calculated.
The most basic method of keeping quotes accurate is to base the client rating being used on the average of individual client ratings generated from each individual project engaged so far. Our original example used this method, in which each completed project was evaluated individual to identify its client rating, which in turn is added to the client ratings of previous projects in order to define a new average client rating that reflects the client's performance across all projects.
Using a client rating based on the average of all projects is good for situations where the project meets one or more of the following conditions:
- The project is reflective of the typical projects you have already delivered to the client.
- You are unaware of the bigger picture that the project exists within that will affect the client's performance on the project.
- There are not enough projects available to make use of the additional methods for generating the client rating.
Where the project doesn't meet any of the above conditions, there is an unnecessary uncertainty that poses a risk of the average client ratings being positively or negatively influenced by projects that are irrelevant to the current project being estimated.
An advantage using an average client rating in this type of scenario is that they tend to adjust themselves in the long run, providing that there are future projects/phases where they make enough improvements to cover any losses you experience from unexpected decreases in their efficiency.
To demonstrate how using an average client rating can introduce inaccuracies in a scenario that doesn't represent the typical average project, we will perform an estimate on a project scenario where the client hasn't previously worked with us at such a technical level and therefore requires more support—this being the part that has scope to significantly increase the time investment required.
To recap on the client rating produced in the original example:
We see from this that the client is steadily becoming more efficient to work with, even though the average client rating still provides a rate that is near the half way point between their best and worst project performance. With the average client rating being above the target value of 1, it tells us that we should expect the client to take longer to work with on the typical project—in this case, it is telling us to expect to take nearly double the amount of time.
With our project scenario containing elements that are unfamiliar to the client, we can expect the possibility for any of the following to happen:
- More effort to manage the client through communication and planning - With the client being unfamiliar with the technicalities of this type of project that they have no experience of being involved with, there is a fair chance that they will need at least some additional element of support to check that they are understanding communications as expected and are able to be aware of what contributions they are required to make and when they should be delivered.
- Miscommunication with the client that lead to the repetition of work - This could be a lack of clarity in how the client words some requests or completely forgetting to provide information. This happens when the client is unaware of the importance of information—if they are at all aware of what the information is in the first place.
- Requirements to provide the client with additional training so they can contribute to the project - This could be in the form of teaching them how to use software they are unfamiliar with so that the can provide the data that your software components depend upon, or it could be in the form of providing knowledge about some technical activities.
Our scenario could have an element of miscommunication that leads to some of the work being repeated, which isn't good if have provided a fixed price quote. The average client rating may accommodate for this if the client has already shown inefficiencies on previous projects, which fortunately would be the case in our example scenario—providing enough margin for any such inefficiency to cause the project time to be almost double the ideal project time. This shows that the average client rating is at least reliable to a point in this scenario, providing that the ideal time for the project was realistic to begin with.
Where the average client rating could go wrong in this project scenario is if the repetition of code or other time investments required to train and manage the client are a serious enough issue to more than double the time. Maybe you would be happy to accept a small reduction in profitability on this project, which average client rating calculations would adjust itself to recover unaccounted time in future projects/phases—providing that the client becomes more efficient. However, when the problem comes when the client increases the project time to an amount such as three, four or five times the amount of time it should take.
Average client ratings can be made more reliable through being selective in the projects used in the calculation to ensure that the result reflects the expected efficiency of the client in the upcoming project/phase. Considerations that could be taken into account when selecting suitable projects to base the client rating on include:
- Whether the client has been progressively improving with each project - As their skills and ability to engage in a working relationship develops, the client will become more efficient in their collaboration and communications.
- Characteristics that previous projects with the current project/phase - Evaluating the client's performance on previous projects can be an illusion if the characteristics of these projects are not closely analyzed to identify their similarities to the current project. Where previous projects are not similar to the current project, it can appear that the client is able to perform better than they are capable of achieving with characteristics of the current project that prove to be different, more complicated or problematic to deal with. These characteristics may come in the form of some type of technical knowledge, skills, processes or communication ability that are required to be performed efficiently.
- Elements that are standard across all projects - There are likely to be some skills and processes taken for granted that are repeated across all projects. Clients who are initially unfamiliar with these before working with you will not need to repeat learning and mistakes to acquire the knowledge of these skills and processes.
The vulnerability of this method of generating client ratings is in the selection of projects. Being only human, the selection is vulnerable to mistakes in human decision making, whether it be through assumptions made about the suitability of projects being selected or being overly optimistic about the client's abilities.
With our example scenario indicating that the client has little or no experience of participating in the current type of project, it would be more realistic to expect the efficiency of their engagement with the project to become similar to the first project. However, with the client already working with us on previous projects, it would be safe to assume that they will have gained skills and knowledge required to work with us that are transferrable to all projects, hence their performance in collaborating with us may not be as inefficient as the started with. For this reason, we will consider the following projects to reflect the characteristics of the upcoming project:
Project 1 - This project showed how efficient the client was to collaborate with when they first engage the type of project that they've not worked with before. Their lack of understanding of processes and skills required to collaborate at this point is similar to the current scenario.
Project 2 - This project showed how the client was able to make an initial improvement, which may have included elements that are repeated across all projects. With the previous projects being similar to each other, we are choosing to select the second project to accommodate the minimum amount of improvement we expect and to exclude later projects where the client has had more time to practice skills and understand processes that were specific to the previous type of projects.
With the projects with similarities identified for our selection, we are now ready to perform the calculation that identifies the client rating used for this project's estimation:
The result shows our selection indicating that we should expect the client's efficiency for this project to revert to a level that is only slightly better than when they started to work with us, which is what we would be expecting for a type of project that requires complexities that the client has never collaborated with us to produce.
The result also indicates that the client would be expected to become more inefficient than the average client rating would suggest—an increase of 0.31. Although this may sound like a small difference, it could be a large amount of money if the project is valued at thousands or tens of thousands of pounds instead of a few hundred pounds.
The third and final way that we look to enhance our client rating calculation is to estimate the client's efficiency for each individual phase of the project. This method requires more effort to evaluate the project, but stands to produce more accurate results if the project is evaluated correctly. It makes sense to identify that:
- The client's efficiency will vary throughout the project.
There will be some types of task that the client will be better at performing than others, especially if they already have relevant skills and/or knowledge that they use outside of the project.
- The client may learn and improve throughout the project.
Projects that have a longer duration will provide the client with the opportunity to improve their efficiency in the collaboration as they work on the project.
- Specific types of activity may involve barriers that other activities don't.
Barriers such as politics within the client's organization and their reliance on suppliers who are unreliable can affect the client's ability to be efficient in their contribution to the project. Without a full understanding of all of the factors relating to the client, it will not be possible to understand why they become inefficient, but analyzing the client's efficiency on each type of task or phase in a project will allow you to identify recurring patterns that lead to the client's efficiency decreasing when specific types of activities, people or suppliers are involved.
- There will be different levels of risk in estimating some types of activity, especially when we have to base opinions on third party sources.
Accurate estimation is dependent on being able to make calculations based on accurate information. We would ideally want all of this information be to be personally vetted by ourselves or someone we can trust to be knowledgeable on the subject so that the risk of anything being underestimated or overestimated can be minimized. However, mistakes can be made and it is difficult or impossible to fully verify all information, especially those that are outside the scope of our knowledge. We therefore need to reflect the scope for any unreliability of information used in our estimations as a risk.
With it being identified that the client's collaboration efficiency will differ throughout the project for different types of tasks, it makes sense that more accurate estimations can be generated by investing more effort into producing a breakdown of all of the project's activity types, which in turn can be used with existing knowledge of the client to produce a set of client ratings for each type of activity. These client ratings can then be used to produce estimates for the required time and budget required for each individual part of the project, as opposed to simply performing a single estimate to cover everything.
Any uncertainty about the reliability of the information we are using, whether it be about its accuracy, scope for unexpected results or increased complexities that could introduce unknown situations, can be applied by defining this uncertainty as a risk—see the previous section in this chapter on the subject. Applying risk to the client rating calculation allows for an adjustment to be made on any expected client contribution where we have an uncertainty about the reliability of judgments we are using about their abilities.
Our scenario has a level of inaccuracy because there is an assumption that the new project will have a similar progression pattern as the previous projects, even though their characteristics are not completely the same. We also want to incorporate the ability to estimate the level of inaccuracy within our estimation resulting from the possibility of inaccurate information or the occurrence of unlikely scenarios that may occur.
To solve this issue, we can consider the reliability of the information we have used to to produce the client rating for estimating individual tasks as a risk. Our example has the client claiming to be fully proficient in managing data, which will be a big part of their involvement in the testing phase of the project.
The problem is that we already know from previous experience of working with them that they 3 out of 7 times they claim to have skills, they are not as proficient as they claim to be—typically taking double their expected time to complete the task. We can use this information to identify the probability of the client's claim being inaccurate:
- Accuracy Probability = 3 ÷ 7 = 0.43
- Inaccuracy Probability = 4 ÷ 7 = 0.57
Now that we know the probability of the client's claims being correct and therefore incorrect, we are now in a position to scale the client rating that is being given according to the risk of the client being incorrect. We know that the client rating for testing activities in previous projects averages 1.2 and that their client rating doubles for activities they incorrectly claim to be competent at, hence we will expect their client rating to become 2.4 if they are wrong for this phase of the project. We would calculate this as:
We now have a client rating for the specific testing task that the client will be involved with that is proportionate their historical accuracy of their claims to be competent as well as reflecting the increase in their client rating for efficiency when they have been incorrect.