2.6. Popular Quality Techniques

We discussed at length the creation and management of teams, and the impact of sociological factors in these teams. This was important from a high-level view of organizing the quality function. We now delve into the lower-level techniques of actually carrying out quality functions. As was suggested in Chapter 1, ensuring model quality of UML diagrams includes a detailed checklist for each of the UML diagrams that ensures their correctness and completeness by checking their syntax, semantics, and aesthetics. The quality techniques that help us carry out these checks from elements (for example, a use case or a class) to complete models (MOPS, MOSS, and MOBS) include those of walkthrough, inspection, reviews, and audits. Each of these techniques can use the additional techniques of checklists, interviews, and workshops.

These same techniques can also be used to ensure that all the necessary activities and tasks of a process-component have been carried out in its enactment. Detailed audits of a process can also reveal its malleability. As shown in Figure 2.10, one or more of the quality techniques can be used, in combination, to accomplish the quality goals of the process and the models.

2.6.1. Walkthroughs

A walkthrough is the least formal of all quality techniques. It is simply a quick check of whatever has been produced. Walkthroughs are done to simply look through a UML diagram, a piece of code, or a database schema. The producer of the model or code himself can do a walkthrough. While it is a casual or relatively informal quality mechanism, it is important to treat this as a separate activity from the activity of developing a model or writing code. Walkthroughs are not necessarily done with the intention of finding errors (although if errors, incompleteness, or inconsistency is discovered it should be recorded, or corrections made immediately).

Walkthroughs may not check the artifact against a set of rules but rules can be kept in the back of our minds as we conduct a walkthrough. Sometimes, a checklist may be used to conduct a walkthrough, and still the intention is not to locate errors formally, but to simply ensure that no major gaps have been left in the model. A walkthrough can take up to an hour for a particular UML-based diagram—or deliverable such as a use case or a project plan.

2.6.2. Inspections

An inspection is done with the specific aim of finding errors. Thus an inspection is more formal and more robust in ensuring the quality of a particular artifact than a walkthrough. Although an inspection can also be done by the producer of the artifact (now with an aim of finding errors), it is highly advisable that the inspection of, say, a use case or a use case diagram is done by someone other than the one who has produced that artifact. If another person does the inspection, the chances of picking up errors are much higher than if it is done by the producer of the artifact.

A valid question at this point is, “If peers do inspections, what does a quality person do?” Quality analysts well versed in the techniques of producing the artifact can also conduct inspections. Alternatively, they can facilitate such inspections in a workshop environment. A checklist used at this stage provides the ground rules against which the inspection is carried out. These ground rules are also very helpful in keeping the personalities out of the inspection process.

Because more than one party is usually involved, preparation for inspections is very helpful. A time and place for inspections should be set aside and inspections on UML diagrams, and even quality and project plans, should be carried out on a regular basis. Results from the inspections should be formally documented, and fixes should be subject to reinspections. Inspections of UML-based diagrams may take between one and two hours. This is estimated on the fact that two people are usually involved in an inspection, and there is usually the need to ask questions and seek clarifications.

It is worth noting that formal inspections of usability, called usability inspections, still satisfy the requirements of inspection as discussed here. In usability inspections the concept of inspection is combined with that of a workshop, leading to a more comprehensive inspection involving the producer of the artifact, the user, the quality person, and a peer who asks the maximum number of questions as she inspects the product. Such inspections will most certainly go beyond the two hours and may last up to a day.

2.6.3. Reviews

A review is a formal quality technique that ensures that a particular deliverable is meeting its syntax, semantics, and aesthetics criteria. In a UML-based project, a review can be carried out on an entire model—such as a MOPS or a MOSS. A review makes sense at the level of a model or a collection of diagrams, because we want to spend more formal time reviewing a model for the inconsistencies or incompleteness that are not apparent when only a single artifact is inspected.

A review is invariably done in a workshop environment. The ideal way to review UML-based models is to provide the models to be reviewed to the reviewers at least a day before the review—enabling them to familiarize themselves with the models created, and to allow them enough time to prepare their questions.

Reviews are ideally carried out by presenting the artifacts using formal presentation facilities such as the data projector, failing which the producer of the artifact would step through the model on the screen and the reviewers would stop him at any point to ask questions and seek clarification.

While the producer of the models has the responsibility of walking through the models, during this process he is not allowed to defend his model. All the criticisms that are heaped on the model are recorded. Although the reviewers can seek clarifications, these clarifications cannot be initiated by the producer of the model. For example, if a business analyst is presenting his use case diagram and sequence diagrams, he can continue to do so until he is stopped by a peer who wants to know the particular use case in which the message shown in the sequence diagram will appear. The business analyst must then explain the documentation of that use case, and show how an instance of that use case is explained by the sequence diagram.

Sometimes, in a quality review, it's necessary to allow the producer to refrain from providing an explanation immediately. For example, if a reviewer asks, “Why has this use case not been stereotyped?” the business analyst can provide the explanation during the review workshop. However, if a reviewer says, “You must stereotype this use case,” and if the business analyst had a reason not to stereotype it, the explanation should be provided toward the end of the review workshop, and not during it. This is important in order to encourage the reviewers to ask as many questions as possible, and for the producer to sort out the issues before the system goes live. Detailed explanations of why a use case has not been stereotyped, although valid, prevent an average reviewer from asking the next question or passing the next comment—one that might be genuinely important for the quality of the artifact. Once again, the human aspect of keeping the personalities out of the review process is important to achieve the aims of an impartial review.

Reviews can be conducted against the detailed checklists for the UML diagrams (for suggested checklists, see the accompanying CD). They can also be conducted against the objectives of the system identified earlier on in the problem space. Prototypes can be used to review a particular deliverable—ensuring that the deliverable produced satisfies whatever was promised in the prototype.

Users should be involved in reviews whenever possible. Users will be able to make an important contribution during these quality techniques, as they bring with them the field knowledge. For example, a class diagram that satisfies all quality checks may still need improvement if the user mentions an operational requirement of higher hits per minute than originally specified. While such a requirement may not be directly related to a class diagram, eventually this requirement may end up modifying the class diagram in order to improve, say, the performance of the system. Overall, though, users are able to make more contributions to the MOPS reviews, as opposed to the MOSS or MOBS reviews.

Although reviews can be conducted at any point during the project, it is preferable to schedule them at the end of each iteration. While a review at the end of the first iteration is mandatory, the major iteration may have more than one review schedule. Furthermore, each of the packages or subsystems within the overall architecture may have its own formal reviews. Because of the more formal nature of the reviews, one envisages the participation of at least three roles in a review process:

• The person who has produced the artifacts

• The person who has sufficient knowledge to inspect the artifacts

• The quality person who is ensuring that the review process is correctly followed

Results from reviews should be formally documented in a review report. They are then circulated to all participating members and reported to the steering committee. Each review should end with a follow-up task list, including brief meetings to ensure that all errors and criticisms have been addressed by the producers of the artifact. A review may take up to one day, and a follow-up meeting around one hour.

2.6.4. Audits

An audit is the most formal of all quality techniques. It requires preparation, followed by a rigorous inspection and review of whatever has been produced. Thus, in a way, it encompasses the previous two techniques discussed. However, an audit has many more characteristics than an inspection or review. For example, the producer of an artifact being reviewed is usually a team, not a person. It is the responsibility of the entire project team to subject their deliverables—which could be an entire system—to a rigorous audit. Another interesting thing to note is that an audit can be done on a deliverable without the presence of the producer. Thus, an audit may use checklists and interviews more than a workshop.

Audits can be further divided into internal audits and external audits. Both audits have the capability to bring in not only the technological and sociological elements discussed so far, but also legal issues. For example, an internal auditor will be well within her rights to ask questions about the copyrights and license fees paid on a CASE tool being used to create UML diagrams. Thus, audits will look at finances, project plans, organizational plans, budgets, costs against budgets, and other such issues that may not be a part of a programmer's daily routine.

External audits are conducted when the organization lacks internal resources, when the project itself has been outsourced, or when there is need for an unbiased appraisal of the quality of what went on in the project. It is especially helpful in an outsourced project for the organization to commission a third party to perform an external audit on the state of the project. Depending on the legal contracts, such audits may or may not involve checking the financial aspects of the organization—in which case only project-level finances may be audited. Technical and project management issues are also subject to audits. An audit can last from a day to a week or more. However, keeping the scope of the audit itself in mind is important—in fact, auditors must be provided with written objectives of the audit, and should be made to stick to it. Furthermore, audits should result in a formal report on the status of the project, as well as possible suggestions for risk reduction and risk rescheduling.

2.6.5. Checklists

Checklists provide the most basic form of quality techniques. They can be used to conduct a walkthrough or an inspection. They can be used in a formal quality review or a detailed audit. Checklists provide the baseline against which the quality of a model or a product is verified.

For checklists to be successful, they should be carefully created by people who have sufficient experience in creating the artifacts themselves. For example, a quality checklist for a deployment diagram should be created only by someone who has previous UML-based knowledge and experience creating many deployment diagrams. Furthermore, checklists should be graded according to the urgency of the checks and their importance.

Other forms of grading and grouping are also acceptable—but throwing a checklist together with no consideration of the sequence can jeopardize the quality effort. For example, it is advisable to check the syntactical correctness of an attribute within a class before checking the correctness of an operation—because operations may depend on an attribute. The sequencing of checks can be further extended to the UML diagram level. An example is checking a state chart diagram for a class without having checked the class itself. A well-constructed checklist ensures that the class is checked first before the state chart is checked.

Checklists can appear in various mediums—paper-based checklists, electronic checklists that can do some basic calculations on the results of the checks (such as a checklist in an Excel spreadsheet that totals the results of “OK” and “not OK” responses), and checklists deployed on the Internet. Internet checklists are helpful in an outsourced project by providing feedback to the outsourcing partner on the status of a use case diagram or a state chart diagram.

Dynamicity of checklists is also important. If a part of the checklist is to be stepped through depending on the results of the previous check, then the “if then else” situation must be explained in the checklist itself. This is done by means of comments underneath the checks. For example, in checking the attribute of a class, a comment would be “go to the checks of initial values if this is a MOPS quality check.” Dynamicity of checklists also means that the checklists should be continually updated to reflect the newer requirements of quality checks arising out of previous runs of the checklists.

The manner in which checklists are executed or used can also differ. For example, we can use the checklists provided in the accompanying CD to conduct detailed quality checks. But these checklists only provide the reference list—there is no mandate to fill out any results at the end of each check. However, other checklists may have a need for the quality personnel to fill out results or pass comments against each of the checks. Surveys of a social nature, such as checking the e-factor or quality environment within a project, may use such subjective checklists, where those who go through the checklist also write their answers, comments, and criticisms at the end of each check.

2.6.6. Interviews

Interviewing is a technique that not only validates and verifies whatever has been created, but also plays a decisive role in creating a good quality UML model and software in the first place. This is because many of the requirements in their early stages are best elicited and documented using interviews with users, end users, and project sponsors. Interviews are used by business analysts to deal with a large cross section of users, more than just the one or two users who might be onboard in a project.

Interviews require preparation. It is important for the business analysts or the quality analysts to prepare their questions in advance. This sets the agenda and tone for the interview, and also keeps the interviewer free from the responsibility of creating questions on the fly (running the risk of forgetting to ask an important question). However, provisions should be made within the preparatory lists to allow for extra questions based on the answers provided to previous questions.

Preparation for interviews also requires setting up time, and time-boxing the interview to stick to the prearranged schedule. Questionnaires for the interview may be passed on to the interviewee earlier—especially if it requires some research on the part of the interviewee to provide a satisfactory answer. For example, if a business analyst wants to interview an insurance specialist on the recent business regulations governing the claims process for a worker injured while away from the home city on business, then it is advisable to put that question in a list and send it to the insurance specialist well in advance of the interview.

Interviews are best conducted face to face. Telephone interviews, however well conducted, simply do not have the same effect as face-to-face interviews have. Even videophone interviews fail to draw the same response, in my opinion, as a physical interview. The reason for this is perhaps outside the current scope of our discussion but it can be certainly said with confidence that face-to-face interviews should be conducted wherever possible. A follow-up interview can be conducted over the phone, but it will still lack the descriptiveness resulting from an interviewer being physically present.

Recording of interviews can be a sticky point in some interviews. If interviewees are even slightly uncomfortable with a tape recorder, it should be avoided. If there is likely to be a lot of technical material resulting from the interview, someone should accompany the interviewer in order to document everything that is being said during the interviews. However, if the comfort level of the interviewee is not affected, then a tape recorder is the most helpful interview instrument—as we can replay the tape later to document the answers.

The manner in which interview questions are framed also affects the outcome. The choice is between objective questions that result in a plain “yes or no” answer, versus a subjective question that results in a detailed descriptive response. If the nature of your interview is a survey, wherein you have to receive, collate, and analyze responses from hundreds of participants, then an interview with “yes or no” or a grade of one through five or ten is helpful.

However, in most UML-based work in the problem space, subjective questions are more important than objective ones. For example, a good business analyst trying to interview an end user for an Internet-based query feature of an insurance claim will not ask a question like “Would you like to know the date you made your claim?” Instead, a better question is “What would you like to know about your claim?” If the response includes “date,” a follow-up question would be “Why would you like to know about the date?” This approach results in a more comprehensive requirement model, as the end user in this case is likely to describe more features that she wants in the query than the business analyst herself can think of. Subjective interviewing is most beneficial in eliciting quality requirements. Long answers should also be tolerated, but the interviewer should try to bring the interviewee back on course by asking counter-questions that relate to the subject matter. Irrelevant answers can always be ignored, but the cost of not getting the answers at all is too high, especially in the problem space.

2.6.7. Workshops

Workshops, as a formal quality technique, provide immense benefits of group work. Workshops bring together project participants with varying expertise. Workshops can be organized for the purposes of inspections, reviews, and audits. They can also be conducted for requirements capturing and requirements modeling. Workshops provide the forum to express business requirements in the MOPS, but they are equally effective in reviewing a technical design, or getting a consensus on the use of a particular technology within the overall system architecture.

A workshop, however, is a rich source of gaming, as described earlier in the sociology of a project. Meetingitis, the game so popular with all project teams, and which results in a phenomenal amount of unproductive work, finds workshops an ideal playing field. Despite that risk, it is important to have well-organized workshops in a project; they bring together user groups, business analysts, and developers and enable them to sort out the objectives, the requirements, and the designs that would otherwise not be easy to sort on a one-to-one basis. It is therefore important to undertake some preworkshop planning to ensure that the administrative part of the workshop is addressed before the workshop proceeds.

A typical workshop administration may include a whiteboard, flipcharts, data projectors, and other audiovisual equipment. These are the basic things in a workshop, yet they have the tendency to disrupt the workshop. If functioning well, they tend to make the workshop more productive. For example, a whiteboard with a printing facility is a tool I can't praise enough. It is also recommended, especially during requirements workshops wherein a CASE tool is being used for UML diagrams, that the diagrams are projected on a whiteboard from the data projector. This technique provides the dual advantage of the good look and feel with color, provided by the CASE tool, and the ability to conduct discussions and corrections directly on the whiteboard using a marker, before the changes are incorporated in the tool itself.

All workshops must have a facilitator. In a review workshop, this can be the reviewer or it can be the quality person. A quality analyst is called upon to facilitate the workshop and to review the models. However, the skills to review a model and the skills to facilitate a workshop are not the same. A facilitator, for example, must be on her feet most of the time—cajoling and encouraging and at the same time keeping the objectives of the workshop in front of the participants all the time.

There are times in a workshop (typically after lunch) when discussions tend to die down and the room is left with a few seconds of awkward silence. Such situations can be best resolved by asking a leading question, or going through the parking lot of difficult issues thus far. The reverse of this situation is also true, wherein a workshop gets heated with about three different raucous conversations going on in the room at the same time. The facilitator in such situations should ensure that things don't become uncontrollable—the value of a coffee break, even if it is not scheduled, should never be underestimated in a workshop. In the absence of such special situations, the facilitator should stick to the time-boxes, especially the breaks. Workshop sessions should not last more than two hours at the most.

Play acting during workshops is an important aspect of quality techniques. Workshops, during quality reviews and audits, does not mean just sitting and looking at a UML diagram or deliverable from a quality angle. Play acting, by assigning elements on a diagram (actors) to people and stepping through the use cases, is very helpful for quality. While techniques like CRC cards are made up of such play acting, what is suggested here is extending the technique of play acting to verifying the quality of almost all UML-based artifacts.

For example, if you are doing a workshop to verify the quality of a state chart diagram, it is worthwhile to assign the states to the people present in the workshop, and make them responsible for the states as the workshop members step through the state chart diagram. People should be encouraged to voice their views during workshops, but the facilitator has the important job of keeping the workshops from degenerating into a game. Reviews of UML-based requirements can especially benefit by keeping a separate area on the whiteboard titled “parking lot”—wherein all issues related to, say, a use case that can't be resolved immediately are parked.

I tend to look at the emergence of difficult issues, or issues that are likely to hold up the progress of the workshop, with relief rather than concern. This is because the sooner we start having debates and the sooner the confusion surfaces in requirements and quality workshops, the better. If there is no discussion and no confusion early on, I view it with some trepidation—because there are no projects without confusion. It's valuable to allow those confusions to surface as early as possible in the project lifecycle.

Such issues, when surfacing during a workshop, should be welcome but should be parked in the parking lot. In addition to enabling the workshop to move forward, often you will notice that the issues that appeared insurmountable at the start of the workshop, and which were prominently placed in the parking lot, get resolved as the workshop progresses.

Finally, it is important to consolidate the workshop at the end, and to report on the progress. Summarize at the conclusion of the workshop what has been achieved, what should be done next (in terms of action items), and the strategy to handle parking-lot issues. A summary of the minutes of the workshop is circulated to all concerned parties, and the next workshop scheduled tentatively, if possible.