Appendix B. The MSwA Body of Knowledge with Maturity Levels Added
The following content comes from the Master of Software Assurance Reference Curriculum report [Mead 2010a], with maturity levels added for each BoK component.
1. Assurance Across Life Cycles
Outcome: Graduates will have the ability to incorporate assurance technologies and methods into life-cycle processes and development models for new or evolutionary system development, and for system or service acquisition.
1.1. Software Life-Cycle Processes
1.1.1. New development [L4]
Processes associated with the full development of a software system
1.1.2. Integration, assembly, and deployment [L4]
Processes concerned with the final phases of the development of a new or modified software system
1.1.3. Operation and evolution [L4]
Processes that guide the operation of the software product and its change over time
1.1.4. Acquisition, supply, and service [L3]
Processes that support acquisition, supply, or service of a software system
1.2. Software Assurance Processes and Practices
1.2.1. Process and practice assessment [L3]
Methods, procedures, and tools used to assess assurance processes and practices
1.2.2. Software assurance integration into SDLC phases [L2/3]
Integration of assurance practices into typical life-cycle phases (for example, requirements engineering, architecture and design, coding, test, evolution, acquisition, and retirement)
2. Risk Management
Outcome: Graduates will have the ability to perform risk analysis and tradeoff assessment and to prioritize security measures.
2.1. Risk Management Concepts
2.1.1. Types and classification [L4]
Different classes of risks (for example, business, project, technical)
2.1.2. Probability, impact, severity [L4]
Basic elements of risk analysis
2.1.3. Models, processes, metrics [L4] [L3—metrics]
Models, process, and metrics used in risk management
2.2. Risk Management Process
2.2.1. Identification [L4]
Identification and classification of risks associated with a project
2.2.2. Analysis [L4]
Analysis of the likelihood, impact, and severity of each identified risk
2.2.3. Planning [L4]
Risk management plan covering risk avoidance and mitigation
2.2.4. Monitoring and management [L4]
Assessment and monitoring of risk occurrence and management of risk mitigation
2.3. Software Assurance Risk Management
2.3.1. Vulnerability and threat identification [L3]
Application of risk analysis techniques to vulnerability and threat risks
2.3.2. Analysis of software assurance risks [L3]
Analysis of risks for both new and existing systems
2.3.3. Software assurance risk mitigation [L3]
Plan for and mitigation of software assurance risks
2.3.4. Assessment of Software Assurance Processes and Practices [L2/3]
As part of risk avoidance and mitigation, assessment of the identification and use of appropriate software assurance processes and practices
3. Assurance Assessment
Outcome: Graduates will have the ability to analyze and validate the effectiveness of assurance operations and create auditable evidence of security measures.
3.1. Assurance Assessment Concepts
3.1.1. Baseline level of assurance; allowable tolerances, if quantitative [L1]
Establishment and specification of the required or desired level of assurance for a specific software application, set of applications, or a software-reliant system (and tolerance for same)
3.1.2. Assessment methods [L2/3]
Validation of security requirements
Risk analysis
Threat analysis
Vulnerability assessments and scans [L4]
Assurance evidence
Knowledge of how various methods (such as those above) can be used to determine if the software or system being assessed is sufficiently secure within tolerances
3.2. Measurement for Assessing Assurance
3.2.1. Product and process measures by life-cycle phase [L1/2]
Definition and development of key product and process measurements that can be used to validate the required level of software assurance appropriate to a given life-cycle phase
3.2.2. Other performance indicators that test for the baseline as defined in 3.1.1, by life-cycle phase [L1/2]
Definition and development of additional performance indicators that can be used to validate the required level of software assurance appropriate to a given life-cycle phase
3.2.3. Measurement processes and frameworks [L2/3]
Knowledge of range of software assurance measurement processes and frameworks and how these might be used to accomplish software assurance integration into SDLC phases
3.2.4. Business survivability and operational continuity [L2]
Definition and development of performance indicators that can specifically address the software/system’s ability to meet business survivability and operational continuity requirements, to the extent the software affects these
3.3. Assurance Assessment Process (collect and report measures that demonstrate the baseline as defined in 3.1.1.)
3.3.1. Comparison of selected measurements to the established baseline [L3]
Analysis of key product and process measures and performance indicators to determine if they are within tolerance when compared to the defined baseline
3.3.2. Identification of out-of-tolerance variances [L3]
Identification of measures that are out of tolerance when compared to the defined baselines and ability to develop actions to reduce the variance
Outcome: Graduates will have the ability to make a business case for software assurance, lead assurance efforts, understand standards, comply with regulations, plan for business continuity, and keep current in security technologies.
4.1. Making the Business Case for Assurance
4.1.1. Valuation and cost/benefit models, cost and loss avoidance, return on investment [L3]
Application of financially-based approaches, methods, models, and tools to develop and communicate compelling cost/benefit arguments in support of deploying software assurance practices
4.1.2. Risk analysis [L3]
Knowledge of how risk analysis can be used to develop cost/benefit arguments in support of deploying software assurance practices
4.1.3. Compliance justification [L3]
Knowledge of how compliance with laws, regulations, standards, and policies can be used to develop cost/benefit arguments in support of deploying software assurance practices
4.1.4. Business impact/needs analysis [L3]
Knowledge of how business impact and needs analysis can be used to develop cost/benefit arguments in support of deploying software assurance practices, specifically in support of business continuity and survivability
4.2. Managing Assurance
4.2.1. Project management across the life cycle [L3]
Knowledge of how to lead software and system assurance efforts as an extension of normal software development (and acquisition) project management skills
4.2.2. Integration of other knowledge units [L2/3]
Identification, analysis, and selection of software assurance practices from any knowledge units that are relevant for a specific software development or acquisition project
4.3. Compliance Considerations for Assurance
4.3.1. Laws and regulations [L3]
Knowledge of the extent to which selected laws and regulations are relevant for a specific software development or acquisition project, and how compliance might be demonstrated
4.3.2. Standards [L3]
Knowledge of the extent to which selected standards are relevant for a specific software development or acquisition project, and how compliance might be demonstrated
Knowledge of how to develop, deploy, and use organizational policies to accelerate the adoption of software assurance practices, and how compliance might be demonstrated
5. System Security Assurance
Outcome: Graduates will have the ability to incorporate effective security technologies and methods into new and existing systems.
5.1. For Newly Developed and Acquired Software for Diverse Systems
5.1.1. Security and safety aspects of computer-intensive critical infrastructure [L2]
Knowledge of safety and security risks associated with critical infrastructure systems such as found, for example, in banking and finance, energy production and distribution, telecommunications, and transportation systems
5.1.2. Potential attack methods [L3]
Knowledge of the variety of methods by which attackers can damage software or data associated with that software by exploiting weaknesses in the system design or implementation
5.1.3. Analysis of threats to software [L3]
Analysis of the threats to which software is most likely to be vulnerable in specific operating environments and domains
5.1.4. Methods of defense [L3]
Familiarity with appropriate countermeasures such as layers, access controls, privileges, intrusion detection, encryption, and code review checklists
5.2. For Diverse Operational (Existing) Systems
5.2.1. Historic and potential operational attack methods [L4]
Knowledge of and ability to duplicate the attacks that have been used to interfere with an application’s or system’s operations
5.2.2. Analysis of threats to operational environments [L3]
Analysis of the threats to which software is most likely to be vulnerable in specific operating environments and domains
5.2.3. Designing of and plan for access control, privileges, and authentication [L3]
Design of and plan for access control and authentication
5.2.4. Security methods for physical and personnel environments [L4]
Knowledge of how physical access restrictions, guards, background checks, and personnel monitoring can address risks
5.3. Ethics and Integrity in Creation, Acquisition, and Operation of Software Systems
5.3.1. Overview of ethics, code of ethics, and legal constraints [L4]
Knowledge of how people who are knowledgeable about attack and prevention methods are obligated to use their abilities, both legally and ethically, referencing the Software Engineering Code of Ethical and Professional Conduct [ACM 2016]
5.3.2. Computer attack case studies [L3]
Knowledge of the legal and ethical considerations involved in analyzing a variety of historical events and investigations
6. System Functionality Assurance
Outcome: Graduates will have the ability to verify new and existing software system functionality for conformance to requirements and to help reveal malicious content.
6.1. Assurance Technology
6.1.1. Technology evaluation [L3]
Evaluation of capabilities and limitations of technical environments, languages, and tools with respect to creating assured software functionality and security
6.1.2. Technology improvement [L3]
Recommendation of improvements in technology as necessary within project constraints
6.2. Assured Software Development
6.2.1. Development methods [L2/3]
Rigorous methods for system requirements, specification, architecture, design, implementation, verification, and testing to develop assured software
6.2.2. Quality attributes [L3—depends on the property]
Software quality attributes and how to achieve them
6.2.3. Maintenance methods [L3]
Assurance aspects of software maintenance and evolution
6.3. Assured software analytics
6.3.1. Systems analysis [L2 architectures; L3/4 networks, databases (identity management, access control)]
Analysis of system architectures, networks, and databases for assurance properties
6.3.2. Structural analysis [L3]
Structuring the logic of existing software to improve understandability and modifiability
6.3.3. Functional analysis [L2/3]
Reverse engineering of existing software to determine functionality and security properties
6.3.4. Analysis of methods and tools [L3]
Capabilities and limitations of methods and tools for software analysis
6.3.5. Testing for assurance [L3]
Evaluation of testing methods, plans, and results for assuring software
6.3.6. Assurance evidence [L2]
Development of auditable assurance evidence
6.4. Assurance in acquisition
6.4.1. Assurance of acquired software [L2]
Assurance of software acquired through supply chains,1 vendors, and open sources, including developing requirements and assuring delivered functionality and security
1. For more information about software security supply chain risk, download the SEI report Evaluating and Mitigating Software Supply Chain Security Risks [Ellison 2010].
6.4.2. Assurance of software services [L3]
Development of service level agreements for functionality and security with service providers and monitoring compliance
7. System Operational Assurance
Outcome: Graduates will have the ability to monitor and assess system operational security and respond to new threats.
7.1. Operational Procedures
7.1.1. Business objectives [L3]
Role of business objectives and strategic planning in system assurance
7.1.2. Assurance procedures [L3]
Creation of security policies and procedures for system operations
7.1.3. Assurance training [L4]
Selection of training for users and system administrative personnel in secure system operations
7.2. Operational Monitoring
7.2.1. Monitoring technology [L4]
Capabilities and limitations of monitoring technologies, and installation and configuration or acquisition of monitors and controls for systems, services, and personnel
7.2.2. Operational evaluation [L4]
Evaluation of operational monitoring results with respect to system and service functionality and security
7.2.3. Operational maintenance [L3]
Maintenance and evolution of operational systems while preserving assured functionality and security
7.2.4. Malware analysis [L2/3]
Evaluation of malicious content and application of countermeasures
7.3. System Control
7.3.1. Responses to adverse events [L3/4]
Plan for and execution of effective responses to operational system accidents, failures, and intrusions
7.3.2. Business survivability [L3]
Maintenance of business survivability and continuity of operations in adverse environments (See also Outcome 3, Assurance Assessment.)
Association of Computing Machinery & Institute of Electrical and Electronics Engineers. Software Engineering Code of Ethics and Professional Practice. Association of Computing Machinery. June 9, 2016 [accessed]. http://www.acm.org/about/se-code.
Ellison, Robert J.; Goodenough, John B.; Weinstock, Charles B.; & Woody, Carol. Evaluating and Mitigating Software Supply Chain Security Risks. CMU/SEI-2010-TR-016. Software Engineering Institute, Carnegie Mellon University. 2010. http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=9337.
Mead, Nancy R.; Allen, Julia H.; Ardis, Mark A.; Hilburn, Thomas B.; Kornecki, Andrew J.; Linger, Richard C.; & McDonald, James. Software Assurance Curriculum Project Volume I: Master of Software Assurance Reference Curriculum. CMU/SEI-2010-TR-005. Software Engineering Institute, Carnegie Mellon University. 2010. http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=9415.