Understanding the Role of the Six Sigma Tools—A Case Study

The successful deployment of Six Sigma depends on leaders having an understanding of the Six Sigma tools—what they are, when they are to be used, when they work, and when they don't. Certainly the more leaders know about the tools the better, although familiarity is more important to acquire than a high degree of technical expertise. The tools we refer to are general improvement tools, some of which may involve the use of two or more statistical methods.

The case study we will discuss is an elaboration of the Newspaper Accuracy case from Hoerl and Snee (2002). While this case is about reducing errors in newspaper publishing, it is illustrative of reducing errors in processes in general.

One of the first questions leaders should ask is “Is this problem important?” The answer here was a clear yes. Nothing is more important to a newspaper than the accuracy of the names, facts, figures, and other information it publishes. In this case the newspaper reported the promotion of a new CEO in a major U.S. corporation and misspelled the new executive's name. The newspaper received a call from an unhappy reader—not the new CEO, not the company's public relations department, not the CEO's spouse, but the CEO's mother! This points out that the requirement of accurate information can come from many different sources, some of which may not be anticipated.

Before delving further into this case study, let us look at the DMAIC process with emphasis on the tools used in each phase, the purpose of each tool, and what we can learn from the application of each tool. A brief description of the most common Six Sigma improvement tools is below. A depiction of the purpose, deliverables, and key tools of each step in the DMAIC process is shown in Figure 8-1.

Key Six Sigma Tools

• Process Map— A schematic of a process showing process inputs, steps and outputs.

• Cause and Effect Matrix— A prioritization matrix that enables you to select those process input variables (Xs) that have the greatest effect on the process output variables (Ys).

• Measurement System Analysis— Study of the measurement system typically using Gage R&R studies to quantify the measurement repeatability and reproducibility.

• Capability Study— Analysis of process variation versus process specifications to assess the ability of the process to meet the specifications

• Failure Mode and Effects Analysis— Analytical approach for identifying process problems by prioritizing failure modes, their causes, and process improvements

• Multi-vari Study— A study that samples the process as it operates and by statistical and graphical analysis identifies the important controlled and uncontrolled (noise) variables.

• Design of Experiments— A method of experimentation that identifies, with minimum testing, how key process input variables affect the output of the process

• Control Plan— document that summarizes the results of a Six Sigma project and aids the operator in controlling the process.

Define Phase

In the define phase we select a project to work on, and define the specific problem to be solved and process to be improved. We use key process metrics to guide project selection and identify the goals, and we summarize the resulting project and its objectives in the project charter. The project charter is a key tool of the project definition phase.

A common cause of failure in improvement projects is lack of common understanding of what the projects will do and accomplish. This can lead to disappointment and finger pointing at project closure.

In the newspaper accuracy study leadership established error reduction as an important issue. In order for a Black Belt to work on this issue the newspaper needed a project charter which, as discussed in Chapter 4, defines the work to be done including the process involved, problem statement, the metrics (process baseline, project goal, process entitlement) associated with the problem, the project objective, the financial impact of the project, the team members, and project scope.

Common problems encountered in constructing an effective project charter are discussed in Chapter 4. Leadership must ensure that for each project a charter is created by the project Champion, is refined by the Champion and Black Belt, contains all the relevant metrics, defines the proper scope, specifies all the metrics, names the team members¸ and so on. In short, the charter must be complete with no shortcuts taken. It then serves as an internal contract among all project stakeholders. The project scope is very important because it defines the boundaries of the project. The scope of the newspaper accuracy project was the editorial process beginning with assignment of the story and ending with transmission of the story to production. Anything before or after these boundaries would be out of scope for this project.

Some baseline data showed that while the copy desk might catch and fix as many as 40 errors per day, the rate on a typical day was 20 errors. Each error caused significant problems in the production process, so improvement was needed. A goal was set for this project to reduce the errors by 50 percent to less than 10 per day.

The financial impact of an error was established as $62 if caught at the copy desk, $88 if caught at the composing room, $768 if a page had to be redone, and $5,000 if the presses had to be stopped and restarted. Of course, the cost of an error being published is unknown and unknowable.

The team developed an operational definition for an error before any data were collected, so that the data would be accurate, and everyone would be talking about the same thing when errors were discussed. This definition was (1) any deviation from truth, accuracy, or widely accepted standards of English usage, or (2) a departure from accepted procedures that causes delay or requires reworking a story or a graphic. The team also decided to divide the errors into nine categories: misspelled words, wrong number, wrong name, bad grammar, libel, word missing, duplicated word, fact wrong, and other. Obviously, all of these errors are not equally serious.

The 11-person team consisted of the Black Belt, the editor, two copy editors, two graphics editors, one reporter, and four supervisors. This team was large (we recommend teams of four to six), but it was effective. Larger teams have difficulty finding mutually agreeable meeting times, reaching consensus, and making decisions. Fortunately, team size turned out not to be a problem in this case.

Measure Phase

The measure phase is intended to ensure that you are working on improving the right metrics, those that are truly in need of improvement and that you can measure well. In this phase you select the appropriate process outputs to be improved, based on the objectives of the project and customer needs. You determine what is acceptable performance and gather baseline data to evaluate current performance. This work includes the evaluation of the performance of the measurement system, as well as the performance of the process being studied. The tools used during the measure phase include the process map, cause and effect diagram or cause and effect matrix, measurement system analysis, capability analysis, and a control chart analysis of the baseline data on the process output. Quality function deployment (QFD) may also be used to refine and validate the output metrics (often referred to as CTQs).

Process mapping, cause and effect diagrams, capability studies, and control charts are popular improvement tools discussed by Hoerl and Snee (2002) and other authors. More detailed discussions of the analysis of measurement systems are found in AIAG (1990) and Wheeler and Lyday (1989). Breyfogle (1999) elaborates on the use of the cause and effect matrix.

The process map is prepared by the team, not the Black Belt alone, and provides a picture of the process, as well as identifying non-value-added work and the hidden factory where the reworking is done. Reworking refers to redoing substandard work done previously, such as finding and correcting errors in financial reports. Non-value-added work refers to that which adds no value to the product or service, but is currently required due to inefficiencies in the process. For example, warehousing finished goods adds no value from a customer point of view, but some level of warehousing is typically needed to maintain a supply chain.

The first process map is prepared at a macro level, and usually consists of five to ten steps. If the team needs further detail for some key steps, it can map them further, creating substeps for each macro step.

The team usually identifies important process input and output variables during the process mapping work. The newspaper writing and editing five-step process map is shown in Figure 8-2. Note the revision cycle that may be a source of both non-value-added work and rework. The size of the paper (number of pages), number of employees absent each day, and a major change in front page (cover) story (yes, no) were identified by the team as variables that could have an effect on errors.

The purpose of both the cause and effect diagram and cause and effect matrix is to enable the Black Belt and the team to study the relationships between the process input variables and the process output variables. In the case of the newspaper accuracy study, the key output variable was errors. The diagram serves as a visual display of the effect, or output variable, and its important causes or input/process variables. The cause and effect diagram for errors is shown in Figure 8-3. The cause and effect matrix rates the process input variables in terms of their relative impact on the process output variables. It is actually one of the houses (typically the third house) created in a full-blown QFD analysis (see Breyfogle [1999]).

For this project the cause and effect diagram was adequate and the cause and effect matrix was not needed. As a general rule, we do not have to use every tool on every project; rather we use whatever tools are needed to successfully complete each phase of the DMAIC methodology. Note also that both the cause and effect diagram and matrix are examples of knowledge-based tools; they are developed based on our existing knowledge of the process, rather than on objective data. Eventually, we need objective data to ensure our current understanding is correct, and to enhance this understanding.

In this case the measurement system analysis consisted of developing the measurement system and errors collection scheme, and validating it. In other instances, particularly in manufacturing, gage repeatability and reproducibility studies (AIAG 1990) are used to evaluate the adequacy of the measurement system. Gage repeatability and reproducibility studies evaluate your ability to replicate results when you take multiple measurements (repeatability), and the ability of several people or pieces of measurement equipment to obtain similar measurements (reproducibility). Such analysis is a particularly important step because, in our experience, as many as 50% of the measurement systems in use are in significant need of improvement. Of course, there can be other measurement issues besides repeatability and reproducibility, such as accuracy (ability to achieve the correct average measurement).

In Six Sigma projects outside manufacturing, such as the newspaper study or projects in finance, most of the measurement system work focuses on the creation of the measurement system and the construction of the data collection process.

A process capability study is conducted to measure how well the process is capable of meeting the customer specifications. Typical outputs of these studies are short-term capability indices (short-term sigma level, Cp, Cpk) in the measure phase and long-term capability indices (long-term sigma level, Pp, Ppk) in the control phase. See AIAG (1995a) for definitions of these capability indices. Black Belts often conduct such studies using control charts (AIAG [1995a], Montgomery [2001])—graphical depictions of performance level and variation over time. A control chart analysis of 44 days of baseline data for the newspaper study showed that the errors were being produced by a stable process with an average value of approximately 20 errors per day, and daily variations from just below 10 to just below 40.

A control chart is a plot of data over time with statistically determined limits of normal variation. A run chart is analogous to a control chart, but does not have the statistically determined limits. Figure 8-4 shows a run chart of this data that illustrates the degree of stability, average level, and variation of the process.

Analyze Phase

The analyze phase helps us avoid the ready, fire, aim approach by accurately diagnosing the root causes of problems. In this phase we evaluate the baseline data to further document current performance and to identify root causes of the observed variation and defects, collecting additional data as needed.

Two of the improvement tools most commonly used in the analysis phase are multi-vari studies and FMEA. Multi-vari studies are process studies in which we collect data on the key process and input variables as well as on the key outputs. We then analyze the data using graphical and statistical tools such as regression analysis and hypothesis testing, to identify the variables having the most significant impact on the output variables. FMEA is a disciplined methodology for identifying potential defects, and taking proactive steps to avoid them. More detailed explanations of these tools are given in the references on page 220.

When we evaluate data on errors we often use a Pareto analysis to determine which categories of errors are the biggest problems. The Pareto chart (Figure 8-5) is basically a bar graph whose bars are ordered by number or magnitude of occurrence. The theory is that a few categories will account for the majority of the errors, and it holds in the newspaper errors study. In Figure 8-5 we see that the majority of the errors during the March-April time period were due to misspelling, wrong names, numbers, and facts, and poor grammar.

The team initially focused attention on addressing these categories, and identified one root cause: reporters were not using the spell checker. The typical attitude was “I don't have time to spell check. Besides, the copy editors will catch the errors anyway.” The reporters were also not routinely checking their facts and their sources, which was a job requirement. We address how to deal with these root causes in the improve phase.

A multi-vari study is conducted to identify variables that may be producing the errors. The variables studied in the newspaper case were those identified in the measure phase; namely, size of paper, number of employees absent, and major changes in the front-page story. While the size of the paper is controllable, the other two variables are not. The team also studied day of the week and month of the year differences. Work teams often perform differently on Mondays and Fridays than on the other days of the week. Analysis of the data indicated that the only variables with an effect on errors were size of the paper (more pages leads to more errors) and changes in the front cover story (new stories had to be created under very tight schedules increasing the error rate).

Improve Phase

In the improve phase teams figure out how to change the process to address the root causes identified in the analyze phase, and thereby improve the performance of the process. Each process change is tested with a confirmatory study to verify that the predicted improvements in fact happen. It may take several rounds of improvements to reach the desired level of performance. Note that this is the only phase in the DMAIC process that actually makes improvement. The other phases are intended to properly set up (DMA) and maintain (C) the improvements from this phase.

At this point in the newspaper case, management reaffirmed that reporters were responsible for checking the accuracy of their articles. Three job aids were also created: a “Spell Check How-To,” a list of “10 Rules of Grammar,” and the “Pyramid of Trust” which detailed the sources that could be trusted to produce accurate names, facts, and numbers.

These new working methods were communicated in an all-hands meeting in July. The importance of being careful when the front cover story changed with little notice and problems of large editions of the newspaper were also discussed. The interim goal of letting less than 10 errors reach the copy desk per day was also reviewed and reaffirmed.

One month went by, and it was now time to test whether the changes were having an effect. Data for the month of August were analyzed, and total errors had not changed! The leadership team assembled and reviewed the situation. Why were errors still high? Leadership learned that the new procedures were simply not being used. Many employees did not feel that leadership was serious about the changes, and therefore did not take them seriously. This emphasizes the point that deciding on improvements, and actually implementing them effectively, are two different things. The editor reiterated that the new procedures were to be used and that the leadership team was expected to lead this new way of working. Another all-hands meeting was held to address the issue.

One month later when the latest data were analyzed, total errors had dropped significantly (Figure 8-6). In another month the total errors had dropped by approximately 65%, compared to the goal of 50%. The new procedures were clearly working.

It is not uncommon to find that new procedures are not being used. It is leadership's responsibility to ensure that the new way of working is used. Otherwise, the benefits of the project will not be realized. Project reviews, confirmatory studies, and process audits are effective ways to identify whether the process changes are being used and are effective.

In some instances, particularly in manufacturing processes, additional work needs to be done in the improve phase to obtain the desired process performance. Typical studies include verifying cause-and-effect relationships identified in the measure and analyze phases, identifying optimum operating conditions, and defining process specification limits. The tools of statistical design of experiments (DOE) and response surface methodology can be very helpful in these instances (Box, Hunter, and Hunter 1978; Myers and Montgomery [1995]). Designed experiments have been successfully used in non-manufacturing studies and their use continues to grow in this important area of improvement (Koselka [1996]).

Control Phase

In the control phase we implement a system that will ensure that the improved performance of the process is sustained after the Six Sigma team has completed its work and moved on to another project. The key tools of this phase are the control plan (AIAG 1994) including control charts (part of statistical process control—Montgomery [2001] and AIAG [1995a]) and long-term capability studies.

Returning to our newspaper case, we see that while the errors were significantly reduced, errors were not yet at zero, and more work was needed to achieve further improvement. In the meantime, a control plan was put in place to hold the gains of the work done to date, and to keep the errors at the level obtained. This is the purpose of the control phase in Six Sigma, to hold the gains. The control plan specified that the following measures would be monitored using control charts:

• Total errors

• Errors by category

• Percent of articles checked by the author

• Percent of articles spell-checked

The latter two measurements were particularly useful in detecting when the reporters were not following procedures and thus when errors would be a problem.

The team created checklists and defined roles and responsibilities, including backups, to reduce handoff problems between departments. This enabled people to view their work processes as part of an overall system. To obtain further improvements, a team was chartered to find the sources of errors in the newspaper graphics, and eliminate them.

Results

In addition to reducing the errors by 65 percent, the new way of operating had other benefits, including:

• Fewer missed deadlines, including the ability to deal effectively with extremely tight deadlines.

• Improved morale at the copy desk. Copy editors were freed up to make better use of their talents and training.

• Reduced rekeying of names (rework).

• Identification of more efficient and less costly sources of information, resulting in reduced errors and less number input time (less keying of data). News assistants were freed up to do more valuable work

Fewer errors resulted in less non-value-added work and a more streamlined and effective process. Such effects are characteristic of what happens when the errors and defect levels of business processes are reduced. The processes work more effectively and efficiently, costs are reduced, employee moral improves, and customer satisfaction increases.

It is not uncommon for a project, when completed, to achieve improvements to the process that were not anticipated at the start. Six Sigma leaders should be on the lookout for these situations and make sure that all the benefits are included in the financial impact calculation of the project. It is also important that the benefits of the project be communicated widely and that the success of the project be appropriately celebrated.

We also note that two commonly applied improvement tools, FMEA and DOE, were not used in this newspaper accuracy project. This is not unusual. In any given project, some tools are not needed; in other cases, the work may have already been done in a previous project. Leaders should assess whether the proper tools were used, and provide input to the Black Belt as appropriate. Sometimes the unused tools will be needed in subsequent studies. In the newspaper accuracy case the error rate was reduced to less than 10 per day, which is not zero, the desired state. FMEA and DOE may be useful in subsequent projects intended to make further improvements.