Chapter 25. Accelerated Testing Fundamentals

“A stitch in time saves nine.”

Let us examine the time frame of bringing a project to market from different perspectives and examine the impact on the value of the information versus its timeliness.

To do this with some level of quantification, a measure on the “value” of the information must be made. This can be straightforward. For example, if you knew what a lottery number was going to be, the value of the information would equal the jackpot. However, in most cases the value of information is subjective, and in some cases, knowing the information precludes an objective quantification of its value.

For example, knowing that a particular design feature will break under service conditions in the field is valuable. A design fix can be made, and the problem can be avoided. But how valuable is it? The only way to know for sure is to leave the design alone and allow it to fail in the field and then measure the economic impact. The value of the information is in the cost avoided, proving what this cost is means proving a negative … this design change prevented a serious warranty problem associated with the design feature.

Have you heard the one about clapping keeping the lions away? A man standing on the corner clapping furiously is asked by a police officer what he is doing. “Keeping lions away” was his answer. Looking around bewildered, the officer said, “I don't see any lions around.” To which the man replied, “Works well, doesn't it?”

Knowing a design feature will break in the field and cause some warranty rate does not provide the information needed to quantify the value of the information. On the other hand, knowing the economic impact of a particular failure mode without knowing the design feature or failure mechanism would quantify the value of the information, but not provide the information to fix the problem.

Different tests can provide different types of information. Knowing the economic impact of a particular failure mode without knowing the failure mechanism or how to fix it is not very useful. Knowing a failure mechanism and how to fix it without knowing the economic cost of fixing it can be frustrating. And finally, the economic cost of quantifying the economic value of fixing a failure mode is usually cost prohibitive.

What is the value of a piece of information during a project?

Information types:

• Feasibility
• Physical properties
• Failure mechanisms
• Warranty rate
• Nominal operating conditions
• Maximum and minimum operating conditions
• Storage conditions
• Potential failure modes
• Component failure rate
• Mean time to failure
• Mean time between failures
• System reliability

Project stages:

• Research
• Feasibility
• Development/design
• Design validation
• Production ramp
• Production validation
• Production
• Service

Quantifying the value of each type of information at each step of a project is not possible. But providing a relative gauge of the value of the information is possible.

For example, the value of knowing at the feasibility stage that a product is not feasible is worth much more than discovering it at production. And discovering at production that the product is not feasible is worth more than discovering it after three months of production. This is an extreme case and unlikely to occur in real life. It is far more likely that a product is discovered to have serious design flaws during production ramp or later. The value of knowing about serious design flaws during design validation instead of during production ramp or production validation is obvious, but not always quantifiable—unless you did not know.

The value of a piece of information is only known when it comes too late. However, the cost of getting the information can be anticipated. The cost of determining if a material will work (not just feasible) in the design is very expensive during the feasibility stage, there are no tools made, no prototypes—everything would be from scratch. The cost of determining if a material will work at production or field use is fairly low, lots of parts available. Two curves can then be used for the ranking of cost to find information vs. the cost of not knowing. The “sweet spot” is where the sum of the two curves is lowest. (See Figure 25.1.)

Figure 25.1. Ranking on the value of information at a particular level of product development: 10 being the most valuable, 0 being the least. Different job responsibilities would find different pieces of information of different values at various times during product development.

Since the only way to examine the value of a piece of information at a particular time is to examine the cost of not having the piece of information, let us examine three scenarios:

1. A key physical property is wrong
2. A primary failure mode of a product
3. The mean time to failure

In each case, we will consider the potential economic impact of not knowing this information at different levels of development.

25.1. Scenario 1. A Key Physical Property Is Wrong

Suppose that a modulus of elasticity on a structural plastic component is assumed to be 2000 MPa and designed accordingly. Now consider the consequence of not discovering that the material only has a 500-MPa modulus under certain temperature conditions at different times during the development process.

• Research During research, this information would be of some interest, but no economic impact would have been incurred by not knowing yet.
• Feasibility This may impact the feasibility of a product, one-fourth the stiffness is a big difference. However, choosing a different material at this point has very little economic impact provided the material costs are comparable.
• Development/design Getting into the design phase and then discovering the stiffness problem would be annoying, but the economic impact would only be slightly greater than if the discovery was made during feasibility.
• Design validation Having the design process mostly complete and then discovering the stiffness problem would cause some significant redesign and material selection rework.
• Production ramp Tools have been cut, cost of discovery now could be devastating if the replacement material could not be used in the same tooling. Plus, add in all the design and validation rework.
• Production validation Same as production ramp, but add any more time lost and missed production start. Even more cost if there are penalties for stopping another production line the part supplies.
• Production Production lines are being shut down, product deliveries are being missed. This is now a major emergency.
• Service Product has made it out the door and is in the hands of the customer. Add up everything previous and then calculate lost customers, customer dissatisfaction, and so forth. And that assumes that the stiffness problem does not cause any harm; if so, add in liability costs.

25.2. Scenario 2. A Primary Failure Mode of a Product

A resistor in a power circuit is marginally sized for the application. The normal variation in the population is not a problem, but colder operating temperatures can push the resistance too low and result in a rectifying circuit receiving excessive current.

• Research During research, this information would be of some interest, but no economic impact would have been incurred by not knowing yet.
• Feasibility This would not affect the feasibility of the product. A different resistance value or a tighter resistance tolerance would solve the problem.
• Development/design Getting into the design phase and then discovering the resistance problem would be annoying, but the economic impact would only be slightly greater than if the discovery was made during feasibility.
• Design validation Discovering the problem during design validation would not cause much more harm than during development. A few more validation tests would be necessary after the design correction was made.
• Production ramp Cost of discovery now could be significant. It would involve revalidating the design, and resetting the production.
• Production validation Same as production ramp, but add more time lost and missed production start. Even more cost if there are penalties for stopping another production line the part supplies.
• Production Now production lines are being shut down or all components are being reworked, product deliveries are being missed. This is now a major emergency.
• Service Product has made it out the door and is in the hands of the customer. Add up everything previous and calculate lost customers, customer dissatisfaction, and so forth. And that assumes that the overcurrent does not cause any harm (like a fire); if so, add in liability costs.

25.3. Scenario 3. The Mean Time to Failure

A serviceable product has a mean time to failure (MTTF). Consider two possibilities: low MTTF or high MTTF.

• Research During research this information would be of some interest, but no economic impact would have been incurred by not knowing. The reality is that whatever MTTF the product may exhibit during research, feasibility, or development may very well change due to design or production process changes or due to end-use changes. More important, the cost of determining the MTTF would be wasted since the design will go through changes.
• Feasibility This would not affect the feasibility of the product.
• Development/design Knowing the product has a low MTTF would be useful only if the cause of the failure was known.
• Design validation Marginal value.
• Production ramp Marginal value.
• Production validation Marginal value.
• Production Marginal value.
• Service Knowing the MTTF (low or high) is valuable at this point so that proper service can be planned for and provided. If the MTTF is low, the cost of raising it is high.

In reading through these scenarios, two salient points should be noted: (1) The cost of responding to information goes up the later the information is learned. (2) Information has no value unless it can be responded to.

The other fact that comes out of this discussion is the problem of not knowing or noninformation. If you recall Hiesenburg’ uncertainty principle: Measuring the energy of a particle precludes accurately knowing its position, however, measuring its position precludes accurately knowing its energy. This principle was observed in particle theory because the smallest unit that could be used to measure either position or energy was a photon, and the photon would either change the particle’ position or its energy depending on how it was used to measure the state of the particle. The quantity not measured becomes unknown or noninformation.

In a macro sense, the same thing happens in development. If a piece of information is gained during development, then the consequence of not knowing the information until service cannot be found. This is not to say that there is not a definite consequence of discovering the information too late, but that knowing the information early causes those involved in making decisions about the project to act differently. Like the particles in Hiesenburg’ uncertainty principle: When and how the information is acquired precludes the ability to know other pieces of information.

This is not to say that the energy of a particle for which a position has been measured does not exist, it simply cannot be known with certainty. Likewise, the consequences of not knowing a serious design flaw at the development stage exists but is not knowable with any certainty. The energy in the particle can be extrapolated from past measurements, estimated, and theorized. Likewise, the consequences of not knowing a piece of information that is known can be extrapolated from past experience, estimated, and theorized.

Dealing with noninformation in an efficient development plan is essential. An organization must have the self-discipline to make good decisions about what information to gain and what information to not know. (See Figure 25.2.)

Figure 25.2. Relationship between known and unknown information.

What information to know and what information to not know is affected by the business structure. A design and manufacturing firm that separates the design function from the production and service efforts will make decisions about what information to know and what information to not know within the divisions. In other words, the design team will tend to choose to know information about the designs characteristics and less about the warranty or serviceability. Measuring and demonstrating that the design is feasible and can be made reliable does not quantify what the warranty is. However, if the organization of the company dictates that the design group is doing the testing, then the testing will be conducted to focus on the design feasibility and reliability. The warranty or serviceability issues will get second seat. Reverse the situation and have the testing conducted by a manufacturing reliability supervisor who is responsible for warranty and service rates and the testing will focus on those issues. These situations forces a body of noninformation, not because knowing one piece of information precludes another, but because the business unit spending the time, effort, and money to get the information dictates what is known and what is marginalized.

Not only will the business organization dictate the type of testing and what information is known and not known, but the business type will also. An entrepreneurial business will choose to test the feasibility and features of the product. But spending money on features or functional testing reduces what will be spent on durability, reliability, or serviceability.

The type of product being made is important also. A manufacturer of a component that sells its component to another company for assembly into a larger system will not and may not be able to test the product in the assembly. I recently consulted with one company that manufactured seat hinges for automotive applications. In discussing options for testing setup, the option of using the full seat as a fixture for the product was brought up. The only problem was it could not get the seating from its clients in time to do any testing. It could not purchase aftermarket seating because of small changes and because the seating was often sold as all of the individual parts instead of the whole seat, therefore, making the cost excessive. The result was that the seating hinge supplier could not know how its part would perform in the full system until the client conducted testing.

The development stage also creates two kinds of noninformation. The first is simple, there are some characteristics that cannot be known about the product until final production. During development, most injected-molded dies do not reflect any final texturing (that rough surface that is given to some plastic parts like orange peel, leather, and so forth). The final texturing is the last thing that is done to an injection-molding die. This means any affect of the texturing on durability or more likely interference issues with matting parts is not known during early development. The same is true of the effect of any design characteristic that cannot be realized until the final production is started.

The second type of development stage noninformation is a little more subtle. During development certain design assumptions are made, decisions about which design option to use. The cost of developing a particular design option and bringing it to production often precludes switching to the other design option after tooling and production has been set. The performance, characteristics, and benefits of the other design become noninformation. This is not to say that the design could not be investigated later, but an economic barrier develops the further the development goes on a different design. Once one design has been in use for a couple of years of production, the cost of switching to a different design concept becomes a serious economic challenge. This also happens on different levels, from small components to entire transportation systems. Currently, the automotive companies are trying to use some hybrid cars that have both gas-fired engines and electric motors in them. They are also looking at fuel cells, natural gas, and hydrogen powered. The fuel-cell cars hold the most promise for clean fuel-efficient cars, but the hybrid cars are already on the road. The hybrid cars are not necessarily better—more likely worse—but the infrastructure of gasoline fuel stations around the country make it possible. There is no infrastructure for the fuel-cell cars. The lack of infrastructure makes trying the fueled-cell cars on the same scale as the hybrid cars nearly impossible. The fair comparison of fuel cell verses hybrid in the real market place becomes noninformation.

You don't know, what you don't know.

Why is noninformation important? Ignorance of your ignorance is not a good thing. Being conscious that a business structure has precluded knowing certain information is critical to making good business decisions. Suppose you have a company where the manufacturing department is in charge of the testing. The testing focuses primarily on the population’ behavior and optimizing manufacturing and not much design-related information. The company decides it needs to improve reliability, because its warranty rates are higher then its competition. Where to focus its efforts? Well—the information available in the records of all the testing highlight areas of strength and weakness in the manufacturing process. Great—focus on the weaknesses in manufacturing to improve the reliability. But what about the design group? There is no evidence that they have a real problem because the testing focuses on the manufacturing side. A good manager who is conscious of what he or she does not know will be asking some tough questions of the design group also.