34. False Quality Gates

The project’s quality assurance activities focus on format checks that do nothing to improve real quality.

Upon reaching a milestone or finishing an iteration, many organizations execute an institutionalized quality check of the results. Very often, this quality check is split into two parts: a check on the existence and form of planned artifacts and a check of the content. The first step is intended to make sure that the expected deliverables have been produced in the expected format. The second step confirms that the content of each deliverable is relevant, accurate, and complete enough for the purposes of the project. The first is syntactic and the second is semantic.

But there is no point in worrying about the syntax of a deliverable if the semantics are lacking.

The languages of our everyday communication—French, English, Albanian, Urdu, and so on—all have their own grammatical syntax. We use this syntax to help us discover and communicate meaning. For example, English grammatical rules say that all sentences must contain a verb. “We eat breakfast in the evening” is a sentence that would pass the syntactical test—but does it make sense? We can only answer that question by doing a semantic analysis to determine whether this is the intended meaning within the context.

Similarly, all the languages of system development have their own syntax. Common examples are a UML use-case model that must have (1) an actor to trigger it and (2) a use-case name; design models that must have a definition of each one of the interfaces; data field dictionary entries that must include ranges of values. However, these syntactic checks do not help if the wrong actor is defined as triggering the use case, the definition of the design interface contains the wrong details, or the range of values for the data field is incorrect. Semantic checking of the use case asks whether the trigger really is the correct, essential trigger for that use case and if there could be other triggers for the same process.

Just because a document passes its syntactic completeness check does not mean that it is fit for a purpose. A data field defined in the dictionary does not mean that its content has been understood. A process description in the style of “get input, do some work, produce output” is just a waste of paper.

We reviewed a functional specification document that had a heading “Glossary of Terms” in the table of contents. The document was intended to specify functional requirements so that potential suppliers could bid on the job. When we looked at the body of the document, we found that the glossary contained ten entries at varying degrees of vagueness—and these entries did not match the terms used in other parts of the document. But this document had satisfied the quality check because it contained a section called “Glossary of Terms.”

—SQR

Organizations that have false quality gates focus on the syntax and form of a deliverable but neglect the content. There are three common reasons for this behavior:

• Persons assigned to do the QA job are not part of the project team and have no interest whatsoever in closely reading and commenting on the deliverable. So, they take the easy way out and comment on the form. On an international project, one of the partners commented on a specification that had been sent out to all partners, in order to freeze the requirements for the next big version, adding this insight: “This hundred-page text document still contains an enormous number of double spaces and thus looks totally unreadable to me. Correct this and resubmit.”

• Persons assigned to the job have no education in the method used for creating a document and its associated quality aspects or they lack knowledge of the domain area. So, they concentrate on headers and numbering schemes, or they point to deliberately blank paragraphs demanding that there should be an entry under every prescribed heading.

• The process model in the company or its organizational structure encourages this kind of behavior by separating the quality assurance people from the actual work of the project.

In more than one organization—usually in large ones—I have found written instructions informing the QA department that its job is to check the completeness, consistency, and formal correctness of documents. But the people assigned to these jobs are not specialists for requirements, design, programming, testing, or any other discipline of system development. They are “QA people.” They are supposed to use (tons of) prescribed checklists for (tons of) different kinds of documents and tick them off without looking for the meaning. Process models in such companies often explicitly state that the quality assurance of the content lies with the original authors, those that developed the artifact in the first place. They are supposed to be the specialists in their respective areas.

—PH

An indicator that you are using false quality gates is that the majority of feedback from quality checks is concerned with the form of the deliverable rather than the meaning of the content. The cost of this pattern is the time wasted in unproductive procedures, but more importantly, the cost of the content-related defects that slip through into the final product.