Chapter 8
Conformity Assessment
8.1 Introduction
While standards are essential to making widespread consistency of products and processes possible, the confidence in actual compliance with the standards is made possible by conformity assessment. The importance of conformity assessment is reflected in the fact that it is almost always included in any policy statements regarding standards.
There are many definitions of conformity assessment depending on the field of application. One definition, as it pertains to engineering applications, is given by ISO/IEC Guide 2:
Any activity to determine, directly or indirectly, that a process, product, or service meets relevant technical standards and fulfills relevant requirements.
A second definition, similar but somewhat more explicit, is provided by OMB Circular A-119, the direction from the Office of Management and Budget, Office of the President of United States, as implementation of the National Technology Transfer and Advancement Act of 1995:
What is Conformity Assessment?
“Conformity assessment” is a demonstration, whether directly or indirectly, that specified requirements relating to a product, process, system, person, or body are fulfilled. Conformity assessment includes sampling and testing, inspection, supplier's declaration of conformity, certification, and management system assessment and registration. Conformity assessment also includes accreditation of the competence of those activities.
Conformity assessment programs are vital for acceptance and use of engineered products because they ensure not only compliance with the standards and requirements used during manufacturing and/or operation (verification), but also that the correct requirements were put in place in the first place (validation). Failure to use conformity assessment programs effectively may lead to catastrophic results such as the one shown in Figure 8.1: Genesis was a space probe intended to sample the solar wind. While it was successful in doing so, the return to earth of the samples was marred by failure of parachute deployment, caused by a G-switch that was installed backwards. The chair of the NASA mishap investigation board stated that a pretest procedure skipped by the supplier could have easily detected the problem.
Figure 8.1 Genesis space probe sample return capsule
(Courtesy of NASA)
8.2 Users of Conformity Assessment
Conformity assessment in engineering is usually applied by one of the following interested parties:
- Producers and manufacturers.
- Users and their designated agents
- Inspection/insurance agencies and inspection jurisdictions
- Governmental agencies such as the United States Nuclear Regulatory Commission (NRC) and the Environmental Protection Agency (EPA) that may have an interest in the safety of certain engineered products and their operation.
8.2.1 Producers and Manufacturers
Producers and manufacturers have multiple reasons to be interested in the conformity of their products. A product that does not meet the requirements is not going to be welcomed by customers whether it simply fails to do what the customer wants or fails to comply with the required standards and codes. Producers and manufacturers demonstrate their compliance with requirements by following conformity assessment programs. An example is the assessment procedure, referred to as the Quality Control Procedure, of manufacturers showing their compliance with the requirements of the ASME Boiler and Pressure Vessel codes. ASME publication CA-1 titled “Conformity Assessment Requirements” gives detailed instructions for the quality control procedures of manufacturers. Similar conformity procedures are required for many manufactured products ranging from cranes to computer chips to threads and pumps.
8.2.2 Users and their Designated Agents
Many users must have in-house conformity assessment programs in order to ensure that they operate engineered equipment safely, consistently, and reliably. An example is hospitals where complicated equipment such as CAT scanners, sterilizers, and X-ray machines are used on a daily basis for functions in which the patient's welfare depends on proper operation. Hospitals usually have detailed conformity assessment programs to assure that personnel operating specific equipment are qualified to do so and that proper procedures are followed. They also make use of outside accrediting organizations to perform independent conformity assessments.
8.2.3 Inspection/Insurance Agencies and Inspection Jurisdictions
Companies that insure products such as boilers, pressure vessels, and certain manufacturing and processing facilities protect their interests and those of their clients by performing conformity assessments. They typically have authorized inspectors or other assessors who need to be familiar with the details of operation and maintenance of the equipment in order to perform their conformity assessments effectively. The inspectors must comply with a number of requirements, including certification, continuing education (periodic classes to ensure currency of their knowledge), and minimum experience levels. Accordingly, insurance companies have detailed conformance assessment programs to monitor and ensure compliance with internal and external requirements of authorized inspectors performing assessments for them.
Some cities, counties, and states have their own authorized inspectors. In such situations, these jurisdictions must likewise have conformance assessment programs in place.
8.2.4 Governmental Agencies
Conformity assessment programs may also be required at such governmental agencies as the NRC for nuclear components, the EPA for commercial equipment, and the FAA for critical flight programs. Highly specialized personnel are needed for these jobs and a conformity assessment program also ensures proper training and performance of such personnel. US Government regulation 15CFR287 has guidance in this area. It also includes the “legal” definition of conformity assessment (and a number of other related terms) as it applies to the US Government. While written as guidance, and fairly generally, it delineates conformity assessment responsibilities of federal agencies.
8.3 Applicability of Conformity Assessment
Conformity assessment may pertain to the following:
- Employees
- Processes
- Products
- Services
- Systems.
8.3.1 Employees
Conformity assessments are applicable to designated employees of manufacturers who manage the in-house conformity assessment programs, as well as to many of those performing critical operations. Again, using the case of pressure vessel manufacturing, an employee assigned to overseeing the conformity assessment program for the company must meet certain requirements. For example, in addition to possessing certain capabilities, this employee, who works in the plant, must not report to the plant manager but to another arm of the company since her/his responsibilities include monitoring both the plant and the plant manager for conformance. If this employee were to report to the plant manager, there would be an obvious conflict of interest that could make it impossible for the employee to perform her/his role successfully.
8.3.2 Processes
Failure to follow an established manufacturing or operating process may lead to undesirable consequences. This happens, for example, in welded products when the welder does not follow the exact welding parameters related to such items as voltage, current, and speed of welding. It may not be possible to nondestructively determine certain critical properties (low temperature toughness, crack growth rates, etc.) of the weld so the quality of the final product is dependent on effective control of the process. For this reason, conformity assessment programs must always be in place to assure that the process is followed. Similarly, the process of integrating the design, acquiring raw materials, and assembling the product must follow certain procedures to avoid unwanted results.
8.3.3 Products
There are numerous instances in which a product has been shipped to a given location only to have the purchaser discover that it is unacceptable, whether failing to comply with user needs or violating existing local law, standard, or other requirement. The cost of remedying such a situation is sometimes very high. Thus, it behooves the organization specifying the product to have in place a conformance program that includes either checking local requirements regarding acceptance of such products or a procedure for contacting appropriate parties for product approval prior to shipping, along with the program that ensures the quality of the product itself.
8.3.4 Services
Many products such as boilers, bridges, and buildings must be serviced in place and cannot be shipped back to the fabricator for repairs. Servicing such products requires recognition of the local standards and requirements. Personnel, equipment, and procedures must be properly qualified in accordance with the standards applicable on site. To accomplish this, a conformity assessment program needs to be in place to avoid unnecessary delays and cost to the project.
8.3.5 Systems
Conformity assessment is vital to the operating system of a company for product manufacturing and/or operation. For example, the sequence of startup and shutdown of many pressure vessels in a refinery or chemical plant is extremely important in maintaining the integrity of the equipment. A somewhat different example of the need for a conformity assessment program for operations within a company relates to repairing equipment whose operation could be hazardous to maintenance personnel. To perform the repair safely, all sources of energy must be cut off, using a procedure referred to as “Lockout–Tagout.” Sources of electrical power need to be shut off and valves of incoming and outgoing pipes connected to the equipment being repaired need to be closed, and the equipment may need to be properly ventilated to allow personnel to enter or otherwise work on the unit safely. Some of these power sources and valves may be a long distance away from the unit and operators may not be aware of their location without a detailed plan, yet their lives may depend on the switches and valves being in the proper configuration. An appropriate conformity assessment program is used to ensure that they are.
8.4 Verification and Validation Process
One manifestation of the conformity assessment process is referred to as Verification and Validation (V&V). This terminology is frequently used in software development.
One definition of V&V, given by IEEE standard 610, is
Validation: The process of evaluating software during or at the end of the development process to determine whether it satisfies specified requirements.
Verification: The process of evaluating software to determine whether the product of a given development phase satisfies the conditions imposed at the start of that phase.
Thus, validation deals with the question, “Have we correctly specified the product we are planning to build,” while verification deals with, “Does the product we are building meet the specification?”
Simply put, validation consists of processes implemented throughout the life cycle to ensure that the product being produced is correctly defined and specified, while verification consists of processes, again throughout the life cycle, to make certain that the product is correctly built to meet the specification, requirements document, statement of work, etc.
Initially, the V&V process was used to assure that complex software programs do what they are supposed to in order to produce correct and accurate results. In recent years, this process has been formalized as a quality assurance process used in software development to ensure that the final product does what it is intended to. While the development of appropriate requirements for a product (software or otherwise) may be for the most part manageable, the complexity of software products, often with millions of lines of code, has become so great that it is often not practical, perhaps not even feasible, to check every possibility of how the product might work. V&V is one tool used to greatly enhance the chances that the product will work as planned.
As new industries develop and grow, it is common for standards to ensure the quality and/or safety of the products of those industries to develop and grow with them. An example of this has occurred with the software industry. ISO/IEC 12207:2008, Systems and Software Engineering – Software Life Cycle Processes, is an international standard used to ensure the quality of software products. It is notable that this standard was first issued in 1995 (relatively recently) and that it first came out as an international standard (as opposed to proprietary, or an individual country standard). The current standard includes 43 system and software processes dedicated to providing a common structure so that stakeholders in the software development process can use a common language in product development, procurement, and assessment, established in terms of the processes.
A portion of ISO/IEC 12207 deals with V&V, two of the processes used to ensure that software products are correctly specified and that they are built to meet the specification. These processes are a formalization by the software industry, tailored to the needs and capabilities of that industry, of an approach that has long been used in a wide range of industries to ensure that a product being developed meets its intended purpose. Whether in the software industry or elsewhere, when a complex product is produced, it is necessary to plan processes to ensure quality and to implement them throughout the life cycle of the product.
ASME is another organization heavily involved with writing standards related to V&V. Standards such as V&V in Computational Solid Mechanics, V&V in Computational Simulation of Nuclear System Thermal Fluid Behavior, V&V in Computational Fluid Dynamics and Heat Transfer, and V&V in Computational Modeling of Medical Devices are examples of the standards being developed in this field by ASME.
Currently, the V&V planning process is used for software development as well as other product development. The process must assure that key parties involved with the product development discuss, at key points during the development state, such items as progress, path forward, and adjustments to the initial plan based on progress of the product. The nature of the discussion is negotiations and consensus among the primary stakeholders to assure a satisfactory product.
8.5 Conformity Assessment Organizations
There are numerous organizations that offer third-party conformity assessment reviews. Some of these organizations are listed here:
| Acronym or Abbreviation | Name and web address | Major fields of action |
| ACB | American Certification Body (acbcert.com) | Certification services for wireless equipment |
| ASCI | Automation Standards Compliance Institute (isa.org) | Charter includes compliance assistance for software and hardware products |
| ASME | American Society of Mechanical Engineers (asme.org) | Approves manufacturers for fabricating boilers, pressure vessels, and nuclear components. Also publishes ASME CA-1-2014 for Conformity Assessment Requirements |
| ASSE | American Society of Sanitary Engineering (asse-plumbing.org) | Plumbing product certification |
| Bureau Veritas | Bureau Veritas Group (bureauveritas.com) | A global company in testing, inspecting, and certifying a wide range of consumer products |
| CSA | CSA Group (csagroup.org) | Conformity services in the consumer products area |
| Dekra | Dekra Certification, Inc. (dekra.com) | Vehicles, wind turbines, and aviation products |
| DQS | DQS, Inc. (dqsus.com) | Certifies management systems |
| FM Approvals | FM Global Corporation (fmapprovals.com) | Fire prevention, sprinkler systems, and insulation products |
| ICAP | IEEE Conformity Assessment Program (ieee.org) | Electrical engineering, electronics, and telecommunications |
| Acronym or Abbreviation | Name and web address | Major fields of action |
| ICC-ES | ICC Evaluation Service (icc-es.org) | Performs conformity assessment and evaluation of building products in the construction industry |
| INTERTEK | Intertek, Inc. (Intertek.com) | Conformity assessment as well as testing and certification in various industries such as chemical, construction, energy, and transportation |
| NBBI | National Board of Boiler and Pressure Vessel Inspectors (nationalboard.org) | Approves manufacturers for fabricating and repairing boilers, pressure vessels, and nuclear components |
| UL | Underwriters Laboratories (ul.com) | Various areas such as electrical, fire, and HVAC |
8.6 Summary
While conformity assessment has only been named and formalized as such in recent decades, the field has been in place for much longer. Any organization using a formal or informal program to ensure the quality and compliance of its products or processes is employing elements of conformity assessment. Many companies and government agencies have long had formal and well-documented programs to ensure that what they do or produce is properly specified and that the actual work and product comply with that specification. The formalization of the field of conformity assessment has resulted in higher expectations on the part of consumers that the product they purchase will really deliver what they want.
8.7 Case Studies
Case Study 1
A manufacturer wishes to produce engineered hospital beds.
What are the key items it must consider when developing the conformity assessment program regarding (1) product features, (2) manufacturing details, (3) documentation?
Case Study 2
A company has determined through business studies, surveys, and focus groups that customers for the products that it sells would be willing in some cases to pay premium prices for delivery within 2 h of ordering. It wishes to produce the drone version of the grocery store delivery boy of the 1950s.
Discuss the components of a drone delivery system and the conformity assessments that might apply to each.