13 Standards
Standards define generally recognized rules reflecting the “state of the art” of technology. As such, they constitute the frame of reference within which projects operate, promoting consistency of products and processes and providing, at least from a legal point of view, the minimum requirements for professional work.
In this chapter, standards relevant to test management are listed, characterized, and put in relation to each other.
13.1 Objectives and Positioning
In software development, it is increasingly important—not the least for legal reasons—to observe generally recognized engineering rules that have been tried and tested in industry practice and are valid according to the prevailing majority of practitioners. These rules are often described in national and international standards developed and published worldwide.
Standards are developed and maintained by national and international organizations
The following organizations and committees are responsible for standardization, that is for creation, publication and maintenance of respective documents:
At the international level, the “International Organization for Standardization” (ISO, [URL: ISO]) and the “International Electrotechnical Commission” (IEC [URL: IEC])
At the European level, the “European Committee for Standardization” (CEN [URL: CEN]) and the “European Committee for Electrotechnical Standardization” (CENELEC [URL: CENELEC])
In Germany, the “German Institute for Standardization” (DIN [URL: DIN]) and in Austria the “Austrian Standard Institute” (ON [URL: ON])
In the United States, the “American National Standards Institute” (ANSI [URL: ANSI])
In the United Kingdom, the “British Standards Institution” (BSI [URL: BSI])
These standardization bodies governed by public law work in cooperation with many domain-specific organizations such as these:
Domain-specific organizations cooperate with standardization organizations.
The “Electronic Industries Alliance” (EIA [URL: EIA])
The “International Telecommunication Union” (ITU [URL: ITU])
The “European Telecommunications Standards Institute” (ETSI [URL: ETSI])
The “Institute of Electrical and Electronics Engineers” (IEEE [URL: IEEE])
The “Association of German Engineers” (VDI [URL: VDI])
The “Gesellschaft für Informatik” (GI, [URL: GI])
In the area of software testing, the “International Software Testing Qualifications Board” (ISTQB [URL: ISTQB]) and its national boards, such as, for instance, the “American Software Testing Qualifications Board” (ASTQB [URL: ASTQB]) and the “German Testing Board” (GTB [URL: GTB]), define the training contents for professional software testers, thereby contributing to further standardization and international harmonization of technical terminology.
ISTQB defined training contents for the professional software tester
Domain-specific specifications reflecting the state of the art in technology are, for example, published by the following:
The “Object Management Group” (OMG [URL: OMG])
The “World Wide Web Consortium” (W3C [URL: W3C])
The “Motor Industry Software Reliability Association” (MISRA [URL: MISRA])
The “European Computer Manufacturers Association” (ECMA [URL: ECMA])
Often, these publications enjoy the status of preliminary or draft standards that after consolidation will be incorporated into the international standards catalogue. Valid national and international patents, too, are to be considered part of the state of technology.
In addition, some international alliances or bodies corporate, such as the “North Atlantic Treaty Organization” (NATO [URL: NATO]), the “American Department of Defense” (DoD [URL: DOD]), and the “European Organization for Civil Aviation Equipment” (EUROCAE [URL: EUROCAE]), maintain their own “domain-specific” rules and standards.
It is one of the tasks of quality or test managers to determine which norms, standards, and perhaps statutory regulations are applicable either for the product under test (product standards) or the project (process standards) and to ensure their compliance.
Audits assess product and process adherence to norms and standards.
Adherence of software products or of development processes to applicable standards, guidelines, and specifications is assessed in audits (see [IEEE 1028]).
This chapter considers different types of standards, structured by their increasing applicability:
Corporate standards
Best practices and technical specifications
Domain-specific standards
Generally valid standards
13.2 Corporate Standards
Particularly in large, international corporations and organizations with a large number of products and projects, corporate internal directives and process instructions (possibly also specified by the customer) are applied to ensure that processes run smoothly locally and beyond national borders.
Corporate internal guidelines and process instructions
To these belong, for example, the following:
Specifically tailored process models
Quality management manuals to be applied across an entire organization
Product-area-specific test manuals
Document templates
Coding and design conventions
Standardized coding conventions, in particular, are important to foster exchange and reusability of developed software and hence the establishment of product lines. Since developers are known to resist having their freedom to design code curtailed, code conventions must be mutually defined and agreed upon by quality assurance and development and regularly checked for their usefulness and applicability.
Keep coding conventions lean and practicable!
Moreover, standard templates are necessary for the entire documentation. Test managers create and use such templates, e.g., for test plans (see chapter 5), deviation reports (see chapter 8), and metric definitions (see chapter 11).
Support members of staff by providing electronic templates and appropriate tooling (macros, scripts, etc.) for document creation.
Do not provide any code conventions that cannot be verified through static analyzers.
13.3 Best Practices and Technical Specifications
So-called “best practices” are not yet standardized but widely accepted methods and procedures representing the state of the art in their field of application.
Best practices represent the state of the art in particular fields of application.
In many areas, technical specifications are developed as a preliminary step toward standardization, and products are then manufactured, marketed, and used according to them. The line between these kinds of technical specifications and corporate standards is blurred, especially if such companies have a sufficiently large market share (“de facto standards”).
For this reason, companies and organizations often combine their efforts to create larger market opportunities for their specifications, evolving, for instance, into consortiums such as W3C [URL: W3C], OMG W3C [URL: OMG], ECMA [URL: ECMA], MISRA [URL: MISRA], HIS [URL: HIS], AUTOSAR [URL: AUTOSAR], and others. All of them develop specifications and publish them as soon as possible. However, this can lead to certain premature, ad hoc publications that are then revised several times and available in as many versions, forming the basis of concrete products that, regrettably, may in some cases be quite incompatible.
Project-oriented consortiums establish technical specifications that also reflect the current state of the art.
Best practices are listed together with corresponding references in so-called “bodies of knowledge”. The standard [ISO 19759], “Guide to the Software Engineering Body of Knowledge” (SWEBOK [URL: SWEBOK]), for instance, categorizes relevant methods regarding all sections of software engineering, dedicating a chapter each to the description and listing of test techniques and software quality methods. In its regularly revised syllabus, the “International Software Test Qualifications Board” refers to important practices in the area of testing [URL: ISTQB].
Body of knowledge lists best practices of an area of application.
Part of the best practices in software development are practices listed in process models such as the life cycle model of the Federal German republic (V-model XT) and the Rational Unified Process (RUP) (see chapter 3).
Many technical specifications are freely available and may be used free of charge. The OMG specifications, for instance, are available for free (see also [URL: OMG]).
13.4 Domain-Specific Standards
In many areas of life, software has replaced man as a controlling and regulating body. Defects often had, and have, catastrophic consequences in industries such as the aerospace and medical industries, whereas in the consumer sector defects have at most been considered a nuisance to users or as having an image-damaging effect on the manufacturers. But even here the consequences of defects are becoming more and more noticeable. One example is the automotive sector where many electronics failures are traced back to software defects.
Software (and its defects) are omnipresent!
Many fields of application pose very specific requirements on the systems being deployed or the software used in them. National and international organizations, partly governed by public law, have been established to create standards for product development, quality assurance, and deployment, ranging from informal guidelines up to international standards.
Most of the domain-specific standards relate to software development for safety-critical applications in sectors such as the aerospace industry, the military, railroad engineering, medical technology, pharmaceutics, and power plant engineering. Some corresponding standardization bodies and related domain-specific standards are summarized in table 13-1. Documents marked (*) are considered in some more detail in the course of this chapter.
Domain-specific standards, especially for safety critical software
Table 13–1 Domain-specific standards
Sector |
Body |
Norm/Standard |
Aviation |
RTCA (USA) |
DO 178 B/ED-12B (*) |
Space industry |
NASA (USA) |
NASA-STD-8719.13B |
Military |
DoD (USA) |
MIL-STD-498 (replaced by IEEE/EIA Std 12207:1998) |
Medical technology |
FDA (USA) |
FDA-535, FDA-938 |
Pharmaceutics |
FDA (USA) |
FDA-21 CFR Part 11 |
Railroad engineering |
CENELEC SC 9XA |
EN 50128 (*) |
Nuclear technology |
DOE (USA) |
DOE G 414.1-4 |
Telecommunications |
ITUT-SG17 |
ITU X.290-X.296 (ISO/IEC 9646-x) |
In the aviation industry, the “Software Considerations in Airborne Systems and Equipment Certification” (DO 178 B/ED12B, [DO 178 B]) standard developed in cooperation between RTCA and EUROCAE contains guidelines for the development of software for equipment deployed in avionic systems. DO 178 B distinguishes five different categories depending on defect severity (table 13-2).
DO 178 B applies to “flying” software
Table 13–2 DO 178B failure categories
Software level |
Potential failure condition |
Potential consequences |
A |
Catastrophic |
Continued safe flight and landing impossible |
B |
Hazardous/severe-major |
Severe failure condition for the aircraft (e.g., flight characteristics or controls severely restricted, safety at great risk, serious injuries possible) |
C |
Major |
Major failure condition for the aircraft (e.g., flight management system could be down, safety at risk, minor injuries possible) |
D |
Minor |
Minor failure condition for the aircraft (e.g., safety could be at risk, some inconvenience possible) |
E |
No effect |
No effect on safety or aircraft operation |
Based on these categories, software is divided into (safety) levels A through E depending on the consequences of a potential software failure.
The DO 178 B demands on software development and testing increase with the software level.
DO 178 B also contains technical guidelines, such as, for example, the application of test techniques “Modified Decision/Condition Coverage” (MC/DC) (see [Spillner 07, section 5.2.3]) for level A software. The standard also requires the measurement of code coverage at source code level for levels B through E. Moreover, for level A object code, instrumented instructions that cannot be directly traced back to source code (for instance, compiler-inserted array bounds checks).
DO 178 B requires MC/DC coverage for level A software.
It is up to the test manager to ensure that the methods and techniques required by the standard are implemented, although he does not have much leeway left for the organization of the test activities. It’s important that evidence can be provided of the methods and techniques applied and that required coverage has been achieved.
For railroad control and monitoring applications within the scope of the European railroad authorities, standard EN 50128 is to be complied with in all software development activities. This applies to completely new software development as well as to small or large software changes, independent from whether the regulatory authority is involved or not.
EN 50128 applies to software in railroad control and monitoring applications.
However, in the case of existing software, it only applies to the modifications made to the software.
Similar to the DO 178 B safety levels, EN 50128 contains so-called safety integrity levels (SILs) and distinguishes between safety-relevant (SIL > 0) and non-safety-relevant software (SIL=0). Categorization into SIL = 0 or SIL > 0 is proposed by the software manufacturer and certified either by the railroad authority itself or by a recognized expert. In this context, the integration of the software into the complete automotive or railroad system in terms of absence of feedback or error propagation to the basic system (e.g., the bus system) should be considered, taking into account the currently applicable guidelines of the railroad authority. This happens before the software is developed.
EN 50128 contains detailed work instructions.
EN 50128 contains detailed working instructions for all software development and quality assurance activities. With regard to test management, for example, it states that for component testing, a component test specification must exist that checks its intended function. This test specification must also define the necessary degree of test coverage. Tests must be repeatable and, if practicable, should be automated.
An auditable component test report is to be drawn up comprising the following points:
1. Test results and statement whether each component does meet the requirements of its design specification
2. Test coverage to prove that each source code line has been executed at least once (complete instruction coverage)
3. Test cases and their results, which are to be recorded in machine-readable form for subsequent analysis
EN 50128 requires CO coverage for component testing.
Such guidelines facilitate the work of the test manager because he does not need to think about which basic tasks are to be performed. In most cases, however, the exact methods and techniques that are to be employed to complete the required activities need to be defined. Above all, all activities and their achieved results must be documented in detail.
Another central issue of EN 50128 and DO 178 B is the traceability of all documents across the entire development process. Here test managers are particularly asked to prove requirements and code coverage through corresponding test cases.
To a large extent, the other standards listed in table 13-1 also provide domain-specific working instructions; some of them even contain concrete descriptions of applicable methods and techniques. Since these have been developed by boards whose members come mostly from their respective subject areas, they are in most cases more directly applicable and therefore appear to be more practicable than those general standards presented in the next section.
Often the standard documents are expensive and/or difficult to obtain. Provide for sufficient number of printed copies or licenses for electronic versions so that the norms or standards are actually available for use to each member in the project.
13.5 Generally Applicable Standards
In addition to domain-specific standards, there are, in the area of information technology, currently more than 280 generally applicable standards registered at ISO (see [URL: ISO]), starting from basic definitions of terminology to detailed working instructions and reference software source code for multimedia applications. Limiting the scope to norms or standards relevant to software development, we still have almost 90 standards, to which we need to add the IEEE Software Engineering Standards collection with its over 40 standards.
ISO/IECJTC 1/SC 7 and IEEE S2ESC develop important standards for software development.
In the following sections, we shall list and briefly characterize only those standards that due to their content and technical relevance are important to test management. Many of these standards were developed by the following bodies or organizations:
“ISO/IEC Joint Technical Committee 1/Subcommittee 7: Software & System Engineering” (ISO/IEC JTC 1/SC 7, [URL: JTC1SC7])
“ISO/IEC Joint Technical Committee 1/Subcommittee 27: IT Security Techniques” (ISO/IEC JTC 1/SC 27, [URL: ISO])
“IEEE Software and Systems Engineering Standards Committee (S2ESC, [URL: S2ESC]) of the Institute of Electrical and Electronics Engineers” (IEEE, [URL: IEEE])
To keep track in view of such a large number, the simple categorization into process and product standards provided in Software Testing Foundations is no longer sufficient. We therefore use additional categories for classification of these standards, for instance, with respect to their normative objectives:
Categorization into process and product standards is not granular enough.
Terminology, vocabulary: standards defining the terminology of a specific area of application
Principle standards: standards explaining the underlying principles of an area of application
Element standards: standards containing detailed conformity requirements for products and processes
Application guides and supplements: documents containing application guidelines and documentation standards
Toolbox of techniques: descriptions of methods and techniques applied to support adherence to standards
On the other hand, standards are often categorized based on their applicability to the process models, shown in figure 13-1.
Figure 13–1 Process model of the ISO/IEC and IEEE SE standards
Customer relationships: standards that help with the definition of customer or contractor responsibilities
Processes: standards that describe the activities to be performed during the software life cycle in generic terms
Products: standards that contain concrete particulars relating to the characteristics, specifications, metrics, and evaluation techniques of the artifacts to be developed
Resources: standards that deal with the documentation and methods, models, and tools for the development of software products and associated processes
Ultimately, standards can also be classified according to the field of application’s granularity (left of figure 13-2).
Figure 13–2 Structure of the ISO/IEC and IEEE SE standards
The subsequent deliberations regarding generally applicable standards are based on the following list, simplifying the above categories in a pragmatic way:
Terminology and contractual standards
Process standards
Product and documentation standards
Methods and engineering standards
13.5.1 Terminology and Contractual Standards
The standards listed in this section form the basis for all other standards, since their content and statements can be written down and understood only in a consistent and unambiguous way using standardized terminology. The most important terminology standards are among the following:
Terminology standards define the terminology of a particular area of application.
ISO/IEC 2382 part 1-33 [ISO 2382] and IEEE 610.12-1990, which is the standard glossary of software engineering terminology [IEEE 610.12].
ISO 9000 [ISO 9000] is important for test management and explains the basis for quality management systems and the terminology used in the ISO 9000 and ISO 900x series of standards.
The British standard BS 7952-1 defines terms used in software testing [BS 7952-1] and is currently revised to harmonize it with the ISTQB Certified Tester glossary [URL: ISTQB].
It is particularly important for the development, procurement, and use of software to unambiguously define the responsibilities of customer (buyer, client, user) and contractor (vendor, developer, broker) for all parties concerned. There is a whole range of corresponding contractual standards, among them the following:
Contractual standards deal with customer/buyer and contractor/developer relationships.
“IEEE Recommended Practice for Software Acquisition” (IEEE Std 1062-1998, [IEEE 1062]), helping the test manager in the acquisition of test tools.
The “Software Engineering – Product Evaluation” series of standards (ISO/IEC 14598, [ISO 14598-4]), which provides the test manager with advice on how to design acceptance testing while supporting him in the evaluation of test tools. The underlying quality model is described in the ISO/IEC 9126 series of standards, which belongs to the product standards.
In addition, relevant for test management are also standards for the design of requirements specifications (IEEE 1233, “System Requirements Specifications” [IEEE 1233]) and general requirements for system safety (e.g., the ISO/IEC 15408 series of standards, “Information technology – security techniques; evaluation criteria for IT security”).
13.5.2 Process Standards
Processes are correlative activities that, using resources, turn inputs into results (see figure 13-1). Process standards subsume all across-the-board standards that specify minimum requirements for processes or process evaluation and improvement, in some cases without stating concrete requirements regarding implementation.
The best-known example for process standards is the ISO 9000 family, whose standards help organizations of all shapes and sizes in the realization of effective quality management systems.
The ISO 9000 family belongs to the important process standards.
ISO 9000 [ISO 9000] comprises the fundamentals as well as the terminology of quality management systems. It may therefore also be considered a terminology standard.
ISO 9001 [ISO 9001] contains requirements on quality management systems by means of which organizations can prove their capability to meet customer and official requirements with their products and to enhance customer satisfaction.
ISO 90003 [ISO 90003] forms a guideline for the application of ISO 9001 on the development, release, installation, and maintenance of computer software. Amplifications of ISO 9001 specific to test management are, for instance, sections on quality planning, risk management, validation, and test as well as software quality criteria. For example, ISO 90003 requires the planning of suitable (intermediate) checks during the manufacturing process (also applicable to the special case of a software development process) without, however, specifying when and how these are to be performed. ISO 90003 can be used to build a bridge from ISO 9001 to process improvement models such as ISO 15504 (SPICE, [ISO 15504]) and CMMI ([URL: CMMI], see chapter 7) since they are all based on the ISO/IEC/IEEE Standard 12207 (see below).
ISO 9004 is a guideline that considers the effectiveness and efficiency of the quality management system. The objective of this standard lies in organizational performance improvement and in enhancing the satisfaction of customers or other interested parties.
ISO 19011 is a guideline for quality and environmental management systems audits.
In combination, these standards facilitate mutual understanding of quality management in national and international trade relations.
Standard [ISO 12207.0] describes in detail issues to be considered in designing software development processes without, however, providing concrete methods and techniques. Relevant implementation guidelines are provided by [ISO 12207.2]. Interesting for test management is the early integration of testing in the overall development process.
ISO/IEC 12207-the generic “standard software development process.”
The process described in chapter 12 concerning tool selection follows ISO/IEC 14102:1995, “Evaluation and Selection of CASE Tools”, the procedure for the introduction of test tools is based on ISO/IEC TR 14471:1999, “Guidelines for the Adoption of CASE Tools”.
Part 3 of IEC 61508-3 describes functional software safety requirements in safety-related systems and represents a hybrid between process and product standard.
As defined in the domain-specific standards DO 178 B and EN 50128, IEC 61508-3 defines safety integrity levels (SILs) 1 to 4, which describe the assumed hazard potential. The higher the level, the higher the hazard potential, accompanied by stricter requirements on system reliability [IEC 61508-3].
Using such standards for guidance makes sense even where compliance with them is not mandatory.
Finally, when it comes to litigation, “state of the art” development must be proved, including compliance with standards. Naturally, this also applies to all other standards whose application is not mandatory.
13.5.3 Product and Documentation Standards
Standards relating to quality requirements and the concrete design of material and immaterial products fall under the category “product and documentation standards”, providing document and product specifications together with quality attributes and appropriate means of verification. These standards can be further divided into those relating to intermediate software development products such as requirements, design, and test specifications and those relating to the end product software together with maintenance and user documentation.
With respect to quality goals, the ISO 9126 series [ISO 9126] contains a quality model as well as concrete specifications including metrics for evaluation of the quality in use and product quality. Consequently, it supports the test manager in his test planning; i.e., the definition of test objectives for individual quality features as well as measurement and test procedures (see chapters 5 and 11).
The ISO/IEC 12119:1994, “Software Packages: Quality Requirements and Testing” [ISO 12119], defines a quality model for application software and supports the test manager in the design of system and acceptance testing.
The [ISO 9241] series describes in 17 parts ergonomic requirements for office work with visual display terminals and, with regard to test management, serves as a basis for the specification of usability tests.
Concrete specifications concerning the creation of documents are provided by standard [IEEE 730] for the test plan and by standard [IEEE 829] for the entire test documentation. Both were already discussed in Software Testing Foundations. IEEE 829 prescribes the documents referenced in figure 13-3.
Figure 13–3 Test document reference structure according to IEEE 829
IEEE Standard 1044-1993, “IEEE Standard Classification for Software Anomalies” [IEEE 1044], is also important for the test manager and provides a detailed classification for the management of incident reports. IEEE 1044 has already been explained at length in section 8.4.
IEEE 1044: Incident management
Additional standards relevant for documentation are the IEEE Standard 1063-2001, “IEEE Standard for Software User Documentation” [IEEE 1063], and the ISO/IEC standard 15910:1999, “Information Technology – Software User Documentation Process” [ISO 15910]. They can assist test management in drawing up checklists for corresponding reviews.
Last but not least there are standards that define specific data formats regarding the interoperability of systems. EN 9735, for instance, defines syntactic rules for the preparation of messages for electronic data interchange between partners in the fields of administration, commerce, and transport (EDIFACT, [ISO 9735]). The ISO/IEC 13818-x family describes MPEG formats for encryption of multimedia data. Such standards are invaluable to test management when specifying concrete test cases such as syntax tests.
13.5.4 Methods and Engineering Standards
This category comprises concrete working instructions for the constructive, testing, and supporting activities in software development. Several software test standards are of central importance to test management and, independent from concrete products, define how software tests are professionally planned, specified, and performed.
Based on the IEEE Standard 730, “IEEE Standard for Software Quality Assurance Plans” [IEEE 730], the ANSI/IEEE Standard 730.1-1995, “IEEE Guide for Software Quality Assurance Planning”, supports test managers in the creation, evaluation, and maintenance of test schedules.
The IEEE Standard 1044.1-1995, “IEEE Guide to Classification for Software Anomalies” [IEEE 1044.1], provides assistance for deviation management according to the IEEE 1044-1993 standard, “IEEE Standard Classification for Software Anomalies” [IEEE 1044].
IEEE 1059-1993, “IEEE Guide for Software Verification and Validation Plans” [IEEE 1059], provides guidelines and practical advice on validation/verification planning and documentation in all software development phases.
The ISO/IEC standard 16085:2004, “Information Technology – Software Life Cycle Processes – Risk Management” [ISO 16085], gives test managers concrete instructions for risk management (see chapter 9).
The following standards are already cited in Software Testing Foundations [Spillner 07]:
The British Standard 7925-2, “Software Testing, Part 2: Software Component Testing” [BS 7925-2], describing component testing methods and techniques.
The IEEE Standard 1028-1997, “IEEE Standard for Software Reviews” [IEEE 1028], which deals in detail with the planning, performing, and documentation of reviews.
In addition there exist a series of standards relating to methods and techniques that test managers should know. To these belong, for instance, [IEC 60812] for Failure Mode and Effects Analysis (FMEA, see section 9.8.1), [IEC 61025] for Fault Tree Analysis, and [ISO 5806] for decision table practices.
13.5.5 Application of Standards
For the application of standards, we recommend the following approach (see [Schmidt 00]):
1. First, examine your own approach regarding gaps and inconsistencies in applied process and quality management standards and align the terms used for performed activities and artifacts with the standard terminology.
2. Second, align and complete your own artifacts such as documents, specifications, and guidelines with respect to product and documentation standards
3. Building on this, complete your own activities in reference to the methods and engineering norms.
4. Observe your own processes over time using productivity and quality measurements. Optimize them applying standards for process improvement and adapt them to suit organizational and technological changes.
At the end of this chapter, table 13-3 lists all generally valid test management standards, grouped according to the different phases of the test process and the type of standard. Standards applicable to several areas are listed in each area. Documents vital to the test manager are listed in bold.
Table 13–3 standards in the phases of the test process
Type of standard Test phase |
Terminology and contracts |
Processes |
Products |
Documentation |
Methods and techniques |
Test planning Test control |
BS 7925-1 |
ISO 12207-0 |
ISO 9126-x |
IEEE 730 |
ISO 12207-2 |
Test analysis Test design |
[ISO 14598-4] |
IEEE 1008 |
ISO 9241 |
IEEE 829 |
BS 7925-2 |
Test realization and test execution |
|
IEEE 1008 |
ISO 9241 |
IEEE 829 |
IEEE 1028 |
Test evaluation and reporting |
|
IEEE 1008 |
IEEE 982.2 |
IEEE 829 |
IEEE 1044.1 |
Always involve developers in the selection of standards, distinguishing between those that have to be applied at all times and those whose application is only recommended.
Consider standards as an opportunity to improve professional practices; do not use them as surrogates.
Standards to be applied are to be reviewed regularly with respect to changed methods, approaches, and technology. If necessary, a new selection must be made.
If possible, use templates and tools for easier application of standards.
13.6 Summary
Test managers must know which standards are relevant for the product under test or for the (test) process and need to ensure compliance, if necessary through audits.
Categorized by increasing general validity and applicability, there are corporate standards, best practices, technical specifications, domain-specific standards, and generally applicable standards.
Domain-specific standards and generally applicable standards define generally recognized codes of practice and form the framework within which projects are performed. They promote consistency of products and processes and constitute the minimal requirements for professional work.
We can distinguish between terminology and contractual standards, process standards, product and documentation standards, and methods and engineering standards.
In defining the test strategy and the test techniques to be applied, general and domain-specific standards such as [IEC 61508-3], [DO 178 B], [EN 50128], nuclear industry standards, pharmaceutical standards, and the MISRA Guidelines for Vehicle Based Software are to be applied.
IEEE Standard 829-1998, “IEEE Standard for Software Test Documentation” [IEEE 829], is a recognized standard for the form and content of the test documentation. Test plans and test level plans, in particular, are developed in compliance with this standard.
The four standards of the ISO/IEC 9126 series, “Software Engineering – Product Quality” [ISO 9126-x], contain a model and measurements for the evaluation of the quality of software products.
The IEEE Standard 1028-1997, “IEEE Standard for Software Reviews” [IEEE 1028], describes in detail the planning, execution, and documentation of reviews.
The IEEE Standards 1044-1993, “IEEE Standard Classification for Software Anomalies” [IEEE 1044], and 1044.1-1995, “IEEE Guide to Classification for Software Anomalies” [IEEE 1044.1], provide a process and detailed classification for the management of incident reports.
The two standards BS 7925-2, “Software Testing – Part 2: Software Component Testing” [BS 7925-2], and IEEE Std 1008, “IEEE Standard for Software Unit Testing” [IEEE 1008], describe methods, practices, and processes for the test level “component testing”.