A decision to evaluate a system formally must take into consideration the many trade-offs between security and cost, such as time to market and the number of features. Groups seeking formal evaluation may have to pay the evaluator's charge as well as staffing costs for skilled experts to develop security documentation and assurance evidence. Interaction with the evaluator for training, clarification, or corrections takes development staff time and could affect development and delivery schedules. Unfortunately, security evaluation cannot prove that a system is invulnerable to attack. Most systems today must operate in hostile environments, and the systems must provide their own protections from attacks and inadvertent errors.

Security and trust are no longer the exclusive realm of the government and military, nor are they of concern only to financial institutions and online businesses. Computers are at the heart of the economy, medical processes and equipment, power infrastructures, and communications infrastructures. Systems having no security are unacceptable in most environments today. Systems providing some security are a step in the right direction, but a trusted system that reliably addresses specifically defined security issues engenders stronger confidence. Evaluation provides an independent assessment by experts and a measure of assurance, which can be used to compare products.

The independent assessment by experts of the effectiveness of security mechanisms and the correctness of their implementation and operation is invaluable in finding vulnerabilities and flaws in a product or system. An evaluated product has been scrutinized by security experts who did not design or implement the product and can bring a fresh eye to the analysis. Hence, the evaluated product is less likely to contain major flaws than a product that has not been evaluated. The analysis of such a system begins with an assessment of requirements. The requirements must be consistent, complete, technically sound, and sufficient to counter the threats to the system. Assessing how well the security features meet the requirements is another part of the evaluation. Evaluation programs require specific types of administrative, user, installation, and other system documentation, which provide the administrators and maintainers the information needed to configure and administer the system properly, so that the security mechanisms will work as intended.

The level of risk in the environment affects the level of trust required in the system. The measure of trust associated with an evaluated product helps find the optimum combination of trust in the product and in the environment to meet the security needs.

Government and military establishments were the early drivers of computer security research. They also drove the creation of a security evaluation process. Before evaluation methodologies were available for commercial products, government and military establishments developed their own secure software and used internal methodologies to make decisions about their security. With the rapid expansion of technology, government and military establishments wanted to use commercial products for their systems rather than developing them. This drove the development of methodologies to address the security and trustworthiness of commercial products.

Evaluation methodologies provide functional requirements, assurance requirements, and levels of trust in different formats. Some list requirements and use them to build trust categories. Others list the requirements only within the description of a trust category. To help the reader compare the development of the methodologies, we present each methodology in a standard manner. We first present overview information about the methodology. Descriptions of functional requirements (when they exist), assurance requirements, and levels of trust follow. If the methodology was widely used to evaluate systems, we describe the evaluation process. The final discussion for each methodology addresses its strengths, its weaknesses, and the contributions it makes to the evaluation technology. Unfortunately, the methodologies use slightly different terminologies. In the discussion of each methodology, we will describe the terminology specific to that technique and relate it to the specific terminologies of previous methodologies.

The TCSEC is organized by evaluation class and uses an outline structure to identify named requirement areas. It defines both functional and assurance requirements within the context of the evaluation classes. The actual requirements are embedded in a prose description of each named area. The divisions and subdivisions of the document are of lesser importance than the actual requirement areas found within them.

Class C1, called discretionary protection, has minimal functional requirements only for identification and authentication and for discretionary access controls. The assurance requirements are also minimal, covering testing and documentation only. This class was used only briefly, and no products were evaluated under this class after 1986.

Class C2, called controlled access protection, requires object reuse and auditing in addition to the class C1 functional requirements and contains somewhat more stringent security testing requirements. This was the most commonly used class for commercial products. Most operating system developers incorporated class C2 requirements into their primary product by the end of the lifetime of the TCSEC.

Class B1, called labeled security protection, requires mandatory access controls, but these controls can be restricted to a specified set of objects. Labeling supports the MAC implementation. Security testing requirements are more stringent. An informal model of the security policy, shown to be consistent with its axioms, completes class B1. Many operating system vendors offered a class B1 product in addition to their primary products. Unfortunately, the B1 products did not always receive the updates in technology that the main line received, and they often fell behind technically.

Class B2, called structured protection, is acceptable for some government applications. At class B2, mandatory access control is required for all objects. Labeling is expanded, and a trusted path for login is introduced. Class B2 requires the use of the principle of least privilege to restrict the assignment of privilege to the users least required to perform the specific task. Assurance requirements include covert channel analysis, configuration management, more stringent documentation, and a formal model of the security policy that has been proven to be consistent with its axioms.

Class B3, called security domains, implements the full reference validation mechanism. It increases the trusted path requirements and constrains how the code is developed in terms of modularity, simplicity, and use of techniques such as layering and data hiding. It has significant assurance requirements that include all the requirements of class B2 plus more stringent testing, more requirements on the DTLS, an administrator's guide, and design documentation.

Class A1, called verified protection, has the same functional requirements as class B3. The difference is in the assurance. Class A1 requires significant use of formal methods in covert channel analysis, design specification, and verification. It also requires trusted distribution and increases both test and design documentation requirements. A correspondence between the code and the FTLS is required.

Government-sponsored evaluators staffed and managed TCSEC evaluations at no fee to the vendor. The evaluation had three phases: application, preliminary technical review (PTR), and evaluation. If the government did not need a particular product, the application might be denied. The PTR was essentially a readiness review, including comprehensive discussions of the evaluation process, schedules, the development process, product technical content, requirement discussions, and the like. The PTR determined when an evaluation team would be provided, as well as the fundamental schedule for the evaluation.

The evaluation phase was divided into design analysis, test analysis, and a final review. In each part, the results obtained by the evaluation team were presented to a technical review board (TRB), which approved that part of the evaluation before the evaluation moved to the next step. The TRB consisted of senior evaluators who were not on the evaluation team being reviewed.

The design analysis consisted of a rigorous review of the system design based on the documentation provided. Because TCSEC evaluators did not read the source code, they imposed stringent requirements on the completeness and correctness of the documentation. Evaluators developed the initial product assessment report (IPAR) for this phase. Test analysis included a thorough test coverage assessment as well as an execution of the vendor-supplied tests. The evaluation team produced a final evaluation report (FER) after approval of the initial product assessment report and the test review. Once the technical review board had approved the final evaluation report, and the evaluators and vendor had closed all items, the rating was awarded.

The Ratings Maintenance Program (RAMP) maintained assurance for new versions of an evaluated product. The vendor took the responsibility for updating the assurance evidence to support product changes and enhancements. A technical review board reviewed the vendor's report and, when the report had been approved, the evaluation rating was assigned to the new version of the product. RAMP did not accept all enhancements. For example, structural changes and the addition of some new functions could require a new evaluation.

The TCSEC created a new approach to identifying how secure a product is. The approach was based on the analysis of design, implementation, documentation, and procedures. The TCSEC was the first evaluation technology, and it set several precedents for future methodologies. The concepts of evaluation classes, assurance requirements, and assurance-based evaluations are fundamental to evaluation today. The TCSEC set high technical standards for evaluation. The technical depth of the TCSEC evaluation came from the strength of the foundation of requirements and classes, from the rigor of the evaluation process, and from the checks and balances provided by reviews from within the evaluation team and the technical review boards from outside the evaluation team.

However, the TCSEC was far from perfect. Its scope was limited. The evaluation process was difficult and often lacked needed resources. The TCSEC bound assurance and functionality together in the evaluation classes, which troubled some users. Finally, the TCSEC evaluations were recognzed only in the United States, and evaluations from other countries were not valid in the United States.

The ITSEC assurance requirements were similar to those in the TCSEC, although there were substantial differences in terminology. As in the TCSEC, assurance requirements were defined within the constraints of the evaluation levels. ITSEC assurance was viewed in terms of correctness and effectiveness. The six effectiveness requirements applied equally to all levels of ITSEC evaluation. The first two effectiveness requirements were as follows.

These requirements applied to the security target and provided an analysis of the security target that contained the security requirements. There was no correspondence between the ITSEC and the TCSEC in this area because the corresponding analysis was done in defining the TCSEC evaluation classes.

This section discusses the remaining four effectiveness requirements along with the correctness requirements. The correctness requirements are subdivided, and, as with the TCSEC, the subdivisions are not as significant as the individual requirement areas. This section will identify the differences between the requirements of the ITSEC and those of the TCSEC.

The ITSEC levels were listed from lowest to highest. Each level included the requirements of the preceding level. If a product or system did not meet the requirements for any level, it was rated as level E0 (which corresponded to the TCSEC level D).

Level E1 required a security target against which to evaluate the product or system. It also required an informal description of the product or system architecture. The product or system had to be tested to demonstrate that it satisfied its security target.

Level E2 also required an informal description of the detailed design of the product or system TOE, as well as configuration control and a distribution control process. Evidence of testing had to be supplied.

Level E3 had more stringent requirements on the detail design and also required a correspondence between the source code and the security requirements.

Level E4 also required a formal model of the security policy, a more rigorous, structured approach to architectural and detailed design, and a design level vulnerability analysis.

Level E5 also required a correspondence between the detailed design and the source code, and a source code level vulnerability analysis.

Level E6 also required extensive use of formal methods. For example, the architecture design had to be stated formally and shown to be consistent with the formal model of the security policy. Another requirement was the partial mapping of the executable code to the source code.

Each participating country had its own methodology for doing evaluations under the ITSEC. All were similar and followed well-defined guidelines. This discussion uses the U.K. methodology.

Certified, licensed evaluation facilities (CLEFs) performed evaluations for a fee. The U.K. government certified the CLEFs. Evaluations typically started much later in the development cycle than did TCSEC evaluations. CLEFs often had an evaluation division and a consulting division. Vendors sought guidance and support from the consulting division to prepare for the evaluation, and consequently the products and systems were better prepared before evaluation began. Because fees were involved, all parties were motivated to finish the evaluation quickly. The evaluation process was much more structured and did not have the lengthy (but technically sound) checks and balances that were provided by TCSEC technical review boards.

The process began with an evaluation of the security target, based on the suitability and binding of assurance requirements. When the security target was approved, the evaluators evaluated the product against the security target. The documentation required for the ITSEC followed a slightly more rigid structure than that for the TCSEC, making it easier in some ways for vendors to provide useful evidence to the evaluators. ITSEC evaluators read the code for clarification when documentation proved inadequate.

The U.K. Scheme for the ITSEC had a very straightforward and simplistic certificate maintenance scheme. It required a plan and evidence to support correct implementation of the plan. Like the evaluation process, it did not have technical reviews such as those of the technical review boards of the TCSEC.

The ITSEC evaluation allowed flexibility in requirement definition and in mixtures of assurance and functionality. Commercial labs performed the ITSEC evaluations, which effectively reduced the length of the evaluation process. Additionally, the ITSEC methodology lent itself to any kind of products or systems. ITSEC provided guidance on what documentation was required. Reciprocity of evaluation existed within the European states. The four effectiveness requirements were a very useful addition to assurance requirements.

In spite of the somewhat stronger assurance requirements in some areas, the ITSEC evaluations were often viewed as technically inferior to the TCSEC evaluations for two reasons. The first was a fundamental potential weakness in the development of functional requirements. The second dealt with the evaluation process itself, which was somewhat lacking in rigor.

Another limit of the ITSEC was the lack of reciprocity of evaluation with Canada and the United States.

The CISR had its roots in the TCSEC evaluation class C2. Because one level of trust was involved, the functional and assurance requirements were stated directly and not embedded in the description of several levels of trust. The CISR functional and assurance requirements included only those requirement areas required by the TCSEC evaluation class C2. This effectively eliminated design specification and verification, labeling, mandatory access control, trusted path, trusted facilities management, and trusted recovery. Assurance requirements were identical to the TCSEC C2 requirements with one small addition. The administrator guide had to contain a threat analysis that identified the protection measures addressing each threat. CISR functional requirements for object reuse and system integrity were identical to the TCSEC class C2 requirements. The other C2 functional requirement areas were enhanced.

CISR added several new categories of requirements that were not found in the TCSEC. Session controls included login attempt thresholds, limits on multiple concurrent sessions, and keyboard locking. System entry constraints could be set to limit a user's access to the system based on time, location, and mode of access. CISR provided a set of workstation security requirements that included the use of encryption, virus deterrents, and restrictions on use of peripheral devices and operating commands. CISR network security requirements included the use of a centralized administrative interface as well as alternative user authentication methods such as tokens, challenge response techniques, and public key cryptography.

Although the CISR never became a generally available evaluation methodology, it did contribute to the rapid growth of evaluation technology in the early 1990s. Perhaps the most significant contribution of this work was the awareness it brought to the U.S. federal government regarding the security evaluation needs of the commercial sector. The CISR influenced the Federal Criteria, which included many of the new requirements stated by the CISR.

The FC included a catalogue of all functional requirements of the TCSEC. New functional requirements adopted from the CISR included the system entry constraints based on time, mode of entry, and location, and other functional issues. Possibly for the first time, there appeared an availability policy based on requirements for resource allocation and fault tolerance. Security management requirements were identified, enhanced, and added to a new section of the functional requirements. Assurance requirements met both TCSEC and ITSEC requirements. The FC included a new assurance requirement for a life cycle process.

The most significant contribution of the FC was the concept of an evaluated protection profile. This approach also appears in the 1993 CTCPEC. The functional requirements sections of protection profiles are similar to the ITSEC functionality classes, but the protection profile requirements were selected from the FC functional requirements catalogue. The FC methodology supported evaluation of protection profiles. In contrast, the ITSEC functionality classes were not included in the ITSEC evaluation methodology.

The FC protection profile included the information needed for identification and cross-referencing as well as a brief description of the nature of the problem that the profile addressed. The rationale portion included identification of threats, the environment, and assumptions and provided the justification for the profile. The subsequent sections of the protection profile contained the functional and assurance requirements as stated in the FC. The FC also introduced the concept of a product-dependent security target that implemented the requirements of an approved protection profile.

A second significant contribution was the development of a profile registry that made FC-approved protection profiles available for general use.

FIPS 140-1 and FIPS 140-2 provide the security requirements for a cryptographic module implemented within federal computer systems. Each standard defines four increasing, qualitative levels of security (called security levels) intended to cover a wide range of potential environments. The requirements for FIPS 140-1 cover basic design and documentation, module interfaces, roles and services, physical security, software security, operating system security, key management, cryptographic algorithms, electromagnetic interference/electromagnetic compatibility, and self-testing. The requirements for FIPS 140-2 include areas related to the secure design and implementation of cryptographic modules: specification; ports and interfaces; roles, services, and authentication; a finite state model; physical security; the operational environment; cryptographic key management; electromagnetic interference/electromagnetic compatibility; self-testing; design assurance; and mitigation of other attacks.

In this section we present an overview of the security levels of FIPS 140-2. Changes from those of FIPS 140-1 reflect changes in standards (particularly the move from the TCSEC to the Common Criteria), changes in technology, and comments from users of FIPS 140-1.

Security Level 1 provides the lowest level of security. It specifies that the encryption algorithm be a FIPS-approved algorithm but does not require physical security mechanisms in the module beyond the use of production-grade equipment. Security Level 1 allows the software and firmware components of a cryptographic module to be executed on a general-purpose computing system using an unevaluated operating system. An example of a Level 1 cryptographic module is a personal computer board that does encryption.

Security Level 2 dictates greater physical security than Security Level 1 by requiring tamper-evident coatings or seals, or pick-resistant locks. Level 2 provides for role-based authentication, in which a module must authenticate that an operator is authorized to assume a specific role and perform a corresponding set of services. Level 2 also allows software cryptography in multiuser timeshared systems when used in conjunction with an operating system evaluated at EAL2 or better under the Common Criteria (see Section 21.8) using one of a set of specifically identified Common Criteria protection profiles.

Security Level 3 requires enhanced physical security generally available in many existing commercial security products. Level 3 attempts to prevent potential intruders from gaining access to critical security parameters held within the module. It provides for identity-based authentication as well as stronger requirements for entering and outputting critical security parameters. Security Level 3 requirements on the underlying operating system include an EAL3 evaluation under specific Common Criteria protection profiles (see Section 21.8.1), a trusted path, and an informal security policy model. An equivalent evaluated trusted operating system may be used.

Security Level 4 provides the highest level of security. Level 4 physical security provides an envelope of protection around the cryptographic module with the intent of detecting and responding to all unauthorized attempts at physical access. Level 4 also protects a cryptographic module against a security compromise resulting from environmental conditions or fluctuations outside the module's normal operating ranges of voltage and temperature. Level 4 allows the software and firmware components of a cryptographic module to be executed on a general-purpose computing system using an operating system that meets the functional requirements specified for Security Level 3 and that is evaluated at the CC evaluation assurance level EAL4 (or higher). An equivalent evaluated trusted operating system may be used.

The CMV program has improved the quality and security of cryptographic modules. By 2002, 164 modules and 332 algorithms had been tested. Of the 164 modules, approximately half had security flaws and more than 95% had documentation errors. Of the 332 algorithms, approximately 25% had security flaws and more than 65% had documentation errors. Vendors were able to correct these problems before their modules and algorithms were deployed and used.

The CC supports two kinds of evaluations: evaluations of protection profiles and evaluations of products or systems against security targets (STs). Product evaluations are awarded at one of seven predefined EALs or at another, user-defined, EAL. All CC evaluations are reciprocal to the signers of the CCRA.

The concept of a protection profile evolved from the Federal Criteria, the CTCPEC profiles, and the ITSEC functionality classes. The form, structure, and terminology of a CC protection profile differs from that of an FC protection profile, although the concepts are similar.

The PP provides a thorough description of a family of products in terms of threats, environmental issues and assumptions, security objectives, and CC requirements. The requirements include both functional requirements, chosen from the CC functional requirements by the PP author, and assurance requirements, which include one of the seven EALs and may include additional assurance requirements as well. The final section of the PP provides the assurance evidence in the form of a rationale that the PP is complete, consistent, and technically sound. PPs do not have to be evaluated and validated. PPs that are evaluated must undergo evaluation in accordance with the methodology outlined in the CC assurance class APE: Protection Profile Evaluation.

A PP consists of six sections.

The second form of evaluation offered by the CC is the evaluation of a product or system against a security target (ST). The results of the evaluation are recognized by all signatories to the CCRA. This type of evaluation has two parts. The first is the evaluation of the ST in accordance with assurance class ASE: Security Target Evaluation (see Section 21.8.4). The product or system itself is then evaluated against the ST.

Under the CC, the functional requirements for a specific product or system are defined in an ST, just as was done under the ITSEC. The concept of a security target evolved from the ITSEC, and the idea of evaluating a security target against an evaluated protection profile evolved from the FC.

There are two approaches to developing an ST. The first approach is to develop an ST based on a PP. The second approach is to develop an ST directly from the CC. If an evaluated PP is used, the ST process is generally simpler because much of the rationale in the ST can reference the PP directly. The ST addresses the same fundamental issues as the PP, with some notable differences. A significant difference is that the ST addresses the issues for the specific product or system, not for a family of potential products or systems.

An ST consists of eight sections.

As shown in the list above, in addition to the PP issues, the ST includes a product or system summary specification that identifies specific security functions and mechanisms. It also describes the strength of the functional requirements and the assurance measures used to analyze those requirements. A PP claims section identifies claims made to PPs that the ST implements. The ST rationale section contains a summary specification rationale that shows how the security functional requirements are met, how any strength-of-function claims are met, and that the assurance measures are sufficient for the assurance requirements. An ST that claims to implement a PP must state those claims and justify them in the rationale.

The CC also has a scheme for assurance maintenance. The goal of such activities is to build confidence that assurance already established for a product or system will be maintained and that the product or system will continue to meet the security requirements through changes in the product or system or its environment.

The heart of the CC is the requirements themselves. The CC defines both functional and assurance requirements and then builds EALs out of the assurance requirements. The requirements are organized into a somewhat elaborate naming and numbering scheme. However, this scheme is much easier to use than the textual descriptions of multiple requirements in a single section, as is done in other methodologies. Functional and assurance requirements are divided into classes based on common purpose. Classes are broken into smaller groups called families. Families contain components, which contain definitions of detailed requirements as well as dependent requirements and a definition of hierarchy of requirements.

There are 11 classes of security functional requirements, each having one or more families. Two of the security functional requirement classes are auditing and security management. The related requirements are unique in the sense that many requirements in other classes generate auditing and/or management requirements. A management section of each family overview provides specific information about management issues relevant to the subdivisions and requirements of the family. Similarly, the audit section of the family overview identifies relevant auditable events associated with the requirements of the family. Requirements may be hierarchical in nature. Requirement A is hierarchical to requirement B if the functional elements of requirement B contain the functional elements of requirement A along with some additions. Finally, nonhierarchical dependencies, which may cross classes, are also identified with each requirement. These four structural approaches (identification of management requirements, audit requirements, hierarchical issues, and nonhierarchical dependencies) help define a consistent and complete specification using the CC.

Consider the security functional requirements of the CC by class and family. The class is indicated by the title, and the families are identified in the descriptive text. All other requirements are derived from previously discussed methodologies.

Class FAU: Security Audit. This class contains six families of requirements that address audit automatic response, audit data generation, audit analysis, audit review, audit event selection, and audit event storage.

Class FCO: Communication. This class contains two families that address nonrepudiation of origin and nonrepudiation of receipt. The CC is the first methodology to contain this requirement.

Class FCS: Cryptographic Support. This class contains two families that address cryptographic key management and cryptographic operation. Encryption algorithms and other implementation issues can be addressed using FIPS 140-2.

Class FDP: User Data Protection. This class has 13 families. It includes two different types of security policies, each with one family for each type of policy and another family that defines the functions for that type of policy. These are access control and information flow policies. The difference between these two types of policies is essentially that an access control policy makes decisions based on discrete sets of information, such as access control lists or access permissions, whereas an information flow control policy addresses the flow of information from one repository to another. A discretionary access control policy is an access control policy and a mandatory access control policy is an information flow control policy. These families are also represented in other methodologies, but they are generalized in the CC, for flexibility.

The residual information protection family addresses the issues called “object reuse” in previous criteria. Other families address data authentication, rollback, stored data integrity, inter-TSF user data confidentiality transfer protection, inter-TSF user data integrity transfer protection, exporting to outside the TSF control, and importing from outside the TSF control.

Class FIA: Identification and Authentication. This class has six families that include authentication failures, user attribute definition, specification of secrets, user authentication, user identification, and user/subject binding.

Class FMT: Security Management. This class contains six families that include management of security attributes, management of TSF data, management roles, management of functions in TSF, and revocation.

Class FPR: Privacy. The CC is the first evaluation methodology to support this class. Its families address anonymity, pseudonymity, unlinkability, and unobservability.

Class FPT: Protection of Security Functions. This class has 16 families. TSF physical protection, reference mediation, and domain separation represent the reference monitor requirements. Other families address underlying abstract machine tests, TSF self-tests, trusted recovery, availability of exported TSF data, confidentiality of exported TSF data, integrity of exported TSF data, internal product or system TSF data transfer, replay detection, state synchrony protocol, timestamps, inter-TSF data consistency, internal product or system TSF data relocation consistency, and TSF self-tests.

Class FRU: Resource Utilization. The three families in this class deal with fault tolerance, resource allocation, and priority of service (first used in the CC).

Class FTA: TOE Access. This class has six families. They include limitations on multiple concurrent sessions, session locking, access history and session establishment, product or system access banners, and limitations on the scope of selectable attributes (system entry constraints).

Class FTP: Trusted Path. This class has two families. The inter-TSF trusted channel family is new to the CC, but the trusted path family was in all previous criteria.

There are ten security assurance classes. One assurance class relates to protection profiles, one to security targets, and one to the maintenance of assurance. The other seven directly address assurance for the product or system.

Class APE: Protection Profile Evaluation. This class has six families, one for each of the first five sections of the PP and one for non-CC requirements.

Class ASE: Security Target Evaluation. This class contains eight families, one for each of the eight sections of the ST. They are similar to the PP families and include families for product or system summary specification, PP claims, and non-CC requirements. Like the requirements of class APE, these requirements are unique to the CC.

Class ACM: Configuration Management (CM). This class has three families: CM automation, CM capabilities, and CM scope.

Class ADO: Delivery and Operation. This class has two families: delivery and installation, and generation and start-up.

Class ADV: Development. This class contains seven families: functional specification, low-level design, implementation representation, TSF internals, high-level design, representation correspondence, and security policy modeling.

Class AGD: Guidance Documentation. The two families in this class are administrator guidance and user guidance.

Class ALC: Life Cycle. There are four families in this class: development security, flaw remediation, tools and techniques, and life cycle definition.

Class ATE: Tests. There are four families in this class: test coverage, test depth, functional tests, and independent testing.

Class AVA: Vulnerabilities Assessment. There are four families in this class: covert channel analysis, misuse, strength of functions, and vulnerability analysis.

Class AMA: Maintenance of Assurance. This class has four families: assurance maintenance plan, product or system component categorization report, evidence of assurance maintenance, and security impact analysis. These were not formal requirements in any of the previous methodologies, but the TCSEC Ratings Maintenance Program (RAMP) addressed all of them. The ITSEC had a similar program that included all these families.

The CC has seven levels of assurance.

EAL1: Functionally Tested. This level is based on an analysis of security functions using functional and interface specifications, examining the guidance documentation provided, and is supported by independent testing. EAL1 is applicable to systems in which some confidence in correct operation is required but security threats are not serious.

EAL2: Structurally Tested. This level is based on an analysis of security functions, including the high-level design in the analysis. The analysis is supported by independent testing, as in EAL1, as well as by evidence of developer testing based on the functional specification, independent confirmation of developer test results, strength-of-functions analysis, and a vulnerability search for obvious flaws. EAL2 is applicable to systems for which a low to moderate level of independent assurance is required but the complete developmental record may not be available, such as legacy systems.

EAL3: Methodically Tested and Checked. At this level, the analysis of security functions is the same as at EAL2. The analysis is supported as in EAL2, with the addition of high-level design as a basis for developer testing and the use of development environment controls and configuration management.

EAL4: Methodically Designed, Tested, and Reviewed. This level adds low-level design, a complete interface description, and a subset of the implementation to the inputs for the security function analysis. An informal model of the product or system security policy is also required. Other assurance measures at EAL4 require additional configuration management including automation. This is the highest EAL that is likely to be feasible for retrofitting of an existing product line. It is applicable to systems for which a moderate to high level of independently assured security is required.

EAL5: Semiformally Designed and Tested. This level adds the full implementation to the inputs for the security function analysis for EAL4. A formal model, a semiformal functional specification, a semiformal high-level design, and a semiformal correspondence among the different levels of specification are all required. The product or system design must also be modular. The vulnerability search must address penetration attackers with moderate attack potential and must provide a covert channel analysis. Configuration management must be comprehensive. This level is the highest EAL at which rigorous commercial development practices supported by a moderate amount of specialist computer security engineering will suffice. This EAL is applicable to systems for which a high level of independently assured security is needed.

EAL6: Semiformally Verified Design and Tested. This level requires a structured presentation of the implementation in addition to the inputs for the security function analysis for EAL5. A semiformal low-level design must be included in the semiformal correspondence. The design must support layering as well as modularity. The vulnerability search at EAL6 addresses penetration attackers with high attack potential, and the covert channel analysis must be systematic. A structured development process must be used.

EAL7: Formally Verified Design and Tested. The final level requires a formal presentation of the functional specification and a high-level design, and formal and semiformal demonstrations must be used in the correspondence, as appropriate. The product or system design must be simple. The analysis requires that the implementation representation be used as a basis for testing. Independent confirmation of the developer test results must be complete. EAL 7 is applicable in extremely high-risk situations and requires substantial security engineering.

The following table gives a rough matching of the levels of trust of various methodologies. Although the correspondences are not exact, they are reasonably close. The table indicates that the CC offers a level that is lower than any previously offered level.

The CC evaluation process in the United States is controlled by the CC Evaluation Methodology (CEM) and NIST. Evaluations are performed by NIST-accredited commercial laboratories that do evaluations for a fee. Many of the evaluation laboratories have separate organizations or partner organizations that can support vendors in getting ready for evaluations. Teams of evaluators are provided to evaluate protection profiles as well as systems or products and their respective security targets. Typically the size of the team is close to the size of a TCSEC team (four to six individuals) but this may vary from laboratory to laboratory.

Typically, a vendor selects an accredited laboratory to evaluate a PP or a product or system. The laboratory performs the evaluation on a fee basis. Once negotiations and a baseline schedule have been developed, the laboratory must coordinate with the validating body. Under the U.S. scheme, the evaluation laboratory must develop a work plan and must coordinate on the evaluation project with the validator and with an oversight board. The evaluation of a PP procedes precisely as outlined in the CEM and according to schedules agreed to by the evaluation laboratory and the PP authors. When the PP evaluation is complete, the laboratory presents its findings to the validating agency, which decides whether or not to validate the PP evaluation and award the EAL rating.

Evaluation of a product or system is slightly more complex because there are more steps involved and more evaluation evidence deliverables. A draft of the product or system ST must be provided before the laboratory can coordinate the project with the validating organization. The vendor and the evaluation laboratory must coordinate schedules for deliverables of evaluation evidence, but otherwise the process is the same as described above for a PP. When the laboratory finishes the evaluation, it presents its findings to the validating agency, which decides whether or not to validate the product or system evaluation and award the EAL rating.

The CC addresses many issues with which other evaluation criteria and methodologies have struggled. However, the CC is not perfect. At first glance, one might think that the protection profiles and security targets of the CC suffer the same weaknesses as those that plagued the security targets of the ITSEC. In some sense, this is true. A PP or ST may not be as strong as TCSEC classes because fewer security experts have reviewed it and it has not yet faced the test of time. Some of the CC requirements were derived from requirements of the previous methodologies. Such requirements may inherently have more credibility. Mature requirements and the CC process of identifying dependencies, audit requirements, and management requirements can contribute to the completeness, consistency, and technical correctness of a resulting PP or ST. The clarity of presentation of the requirements also helps, but ultimately the correctness of an ST lies in the hands of the vendor and the evaluation team.

The CC is much more complete than the functional requirements of most preceding technologies. However, it is not immune to “criteria creep.” A CC project board manages interpretations to support consistent evaluation results. Interpretations can be submitted by any national scheme for international review. The final interpretations agreed on become required on all subsequent evaluations under the CC and form the basis for future CC updates. Although this is a well-managed process, it does not address the fact that a newer evaluation may have more stringent requirements levied on it than an older evaluation of the same type.

Having a team member who is not motivated by financial issues to complete the evaluation quickly lends support to the depth of the evaluation and in some respects addresses the functions of a technical review board by providing impartial review. The evaluation process itself is very well-defined and well-monitored by the validating body. The process itself is less subjective than some of the preceding methodologies because every step is well-defined and carefully applied. Because many U.S. CC evaluators were part of the TCSEC evaluation system, the U.S. CC evaluations are probably close to the TCSEC evaluations in depth.

The CC documentation and methodology continue to evolve. A new version of the CC is planned for release in mid-2003. The revision will include approved interpretations as well as other changes currently under consideration.

The SSE-CMM is organized into processes and maturity levels. Generally speaking, the processes define what needs to be accomplished by the security engineering process and the maturity levels categorize how well the process accomplishes its goals.

The SSE-CMM contains 11 process areas.

The definition of each process area contains a goal for the process area and a set of base processes that support the process area. The SSE-CMM defines more than 60 base processes within the 11 process areas.

Eleven additional process areas related to project and organizational practices adapted from the SE-CMM are

The five Capability Maturity Levels that represent increasing process maturity are as follows.

Application of the SSE-CMM is a straightforward analysis of existing processes to determine which base processes have been met and the maturity levels they have achieved. The same process can help an organization determine which security engineering processes they may need but do not currently have in practice.

This is accomplished using the well-defined base processes and capability maturity levels that were overviewed in the preceding section. One starts with a process area, identifying the area goals and base processes that SSE-CMM defines for the process area. If all the processes within a process area are present, then the next step of the analysis involves determining how mature the base processes are by assessing them against the Capability Maturity Levels. Such an analysis is not simple and may involve interactions with engineers who actually use the process. The result of the analysis culminates in identification of the current level of maturity for each base process in the process area.

The analysis continues as described above for each process area. Processes within an area may have varying levels of maturity, and the level of maturity for the process area would be the lowest level represented by the set of levels for the base process. A useful way of looking at the result of a complete SSE-CMM analysis is to use a Rating Profile, which is a tabular representation of process areas versus maturity levels. An example of such a profile is provided in Figure 21-1. In a similar fashion, process area rating profiles can be used to show the ratings provided for individual base processes within a process area.

Figure 21-1. Example of a rating profile for the 11 process areas of the SSE-CMM (from [347]).