Chapter 3. Lifecycle Management

This chapter helps you prepare for the Certified Information Systems Auditor (CISA) exam by covering the following ISACA objectives, which include understanding the importance of lifecycle management and items such as the system development lifecycle. This chapter pairs with Chapter 4 to cover all of the tasks/knowledge statements and requirements you need to know that are included under the Systems and Infrastructure Lifecycle Management job practice area.

---

Tasks

Evaluate the business case for the proposed system development/acquisition to ensure that it meets the organization’s business goals.

Evaluate the project management framework and project governance practices to ensure that business objectives are achieved in a cost-effective manner while managing risks to the organization.

Perform reviews to ensure that a project is progressing in accordance with project plans, is adequately supported by documentation, and status reporting is accurate.

Evaluate the process by which systems and/or infrastructure are maintained to ensure the continued support of the organization’s objectives and are subject to effective internal control.

Evaluate the process by which systems and/or infrastructure are disposed of to ensure that they comply with the organization’s policies and procedures.

Evaluate proposed control mechanisms for systems and/or infrastructure during specification, development/acquisition, and testing to ensure that they will provide safeguards and comply with the organization’s policies and other requirements.

Evaluate the readiness of the system and/or infrastructure for implementation and migration into production.

---

Knowledge Statements

Knowledge of quality assurance methods

Knowledge of benefits management practices, (e.g., feasibility studies, business cases)

Knowledge of project governance mechanisms (e.g., steering committee, project oversight board)

Knowledge of project management practices, tools, and control frameworks

Knowledge of project success criteria and risks

Knowledge of configuration, change, and release management in relation to development and maintenance of systems and/or infrastructure

Knowledge of requirements analysis and management practices (e.g., requirements verification, traceability, gap analysis)

Knowledge of acquisition and contract management processes (e.g., evaluation of vendors, preparation of contracts, vendor management, escrow)

Knowledge of system development methodologies and tools, and an understanding of their strengths and weaknesses (e.g., agile development practices, prototyping, rapid application development [RAD], object-oriented design techniques)

Knowledge of the management of testing processes (e.g., test strategies, test plans, test environments, entry and exit criteria)

Knowledge of data conversion tools, techniques, and procedures

Knowledge of system and/or infrastructure disposal procedures

Knowledge of software and hardware certification and accreditation practices

Knowledge of post-implementation review objectives and methods (e.g., project closure, benefits realization, performance measurement)

Knowledge of system migration and infrastructure deployment practices

---

Outline

Introduction

Project Management

Roles, Responsibility, and Structure

Project Culture and Objectives

Project-Management Practices

Project Initiation

Project Planning

Project Control and Execution

Closing a Project

Business Application Development

Systems-Development Methodology

Alternative Application-Development Techniques

Application-Development Approaches

Information Systems Maintenance Practices

Chapter Summary

Key Terms

Apply Your Knowledge

Exercises

Exam Questions

Answers to Exam Questions

Need to Know More?

---

Study Strategies

This chapter addresses lifecycle management. Project management, business application development, and infrastructure and acquisition practices are the primary topics. A CISA candidate should review the following primary topics for the exam:

Project-management structure

The formalized steps of the project-management process: initiating, planning, executing, controlling, and closing

Identification and definition of the steps of the system development lifecycle

Alternative approaches to application development, such as prototyping and agile development

Process-improvement practices

Information systems maintenance practices

Introduction

Did you ever stop to think about how much of an auditor’s work actually revolves around project management? After all, projects are temporary endeavors that have a defined beginning, middle, and end. Each audit you will be involved with will be unique. Each will have different requirements and specifications. This means that CISAs need to understand project management. This includes initiating, planning, executing, controlling, and closing projects.

Lifecycle management also requires that auditors understand the system development lifecycle (SDLC). Auditors can become deeply involved in the SDLC process. Auditors are responsible for helping to ensure that sufficient controls are designed during SDLC and that these controls work as expected. Controls must be tested and the overall design plan must be reviewed. Not all projects will use the same development method. Today many alternate development methods, such as prototyping, rapid application development, and agile development, are used. The auditor must understand each of these to fulfill his job duties. After the rollout of new applications, the auditor’s job is not done. Systems require maintenance, review of changes, and review and redesign of processes. Throughout the lifecycle, auditors play a key role.

Project Management

Knowledge Statements

Knowledge of project governance mechanisms (e.g., steering committee, project oversight board)

Knowledge of project management practices, tools, and control frameworks

To be able to discuss project management, one must first define project management. Projects are temporary endeavors. The purpose of this one-time effort is to meet a defined goal of creating a specific product, service, or result. Projects are unique, in that when all the objectives are met, the project is terminated. Projects can be large or small, and can involve hardware, software, or networks. A project can even be used to create a product or service. Projects have unique attributes:

A unique purpose

A temporary nature

A primary customer and/or sponsor

Uncertainty

Projects are constrained by their scope, time, and cost; therefore, you must consider the following:

Scope—How much work is defined? What do the sponsor and the customer expect from this project?

Time—How long is this project scheduled to run? Does it have a defined schedule? When does the product need to be launched, or when does the service need to be operational? Answering these questions will help determine how long the project will run.

Cost—How much money is this project expected to cost? Has the sponsor approved it?

Many approaches and standards exist to meet this triple constraint. The most well known of these approaches and standards are PMBOK (IEEE Standard 1490) Prince 2 (projects in a controlled environment) and Project Management Institute (PMI). Each of these standards is somewhat different, but they share common attributes. The following section looks at some of the common roles and responsibilities in project-management activities.

Roles, Responsibility, and Structure

The auditor should play an active part in the project-management process and know the various roles and responsibilities. Auditors should understand who is responsible and be able to identify key stakeholders, some of which include the following:

Senior management—Provides necessary resources to complete project.

Stakeholders—A person, group, or business unit that has a share or an interest in the project activities.

Project steering committee—Ultimately responsible. Must ensure that the stakeholders’ needs are met and oversee direction and scope of project. The committee acts as project-oversight board.

Project sponsor—Works with the project manager to ensure success and is responsible for allocating funding for the project.

Project manager—Responsible for day-to-day management of the project team.

Project team—Responsible for performing operational tasks within the project.

Quality assurance—Responsible for reviewing the activities of the project-management team and ensuring that output meets quality standards. This role/group does not necessarily act as part of the project team as a whole. QA activities can be carried out by external parties to the project team; including auditors.

Projects must take on an organizational form or framework, which can be either loosely structured or very rigid, in that the program manager has complete authority over the group and is assigned to the group for the duration of the project. Table 3.1 shows the primary types of project organizational forms.

Table 3.1 Project Organizational Forms

Project Culture and Objectives

Teams take on a unique culture. Project managers can play a part in developing a healthy culture by holding a kick-off meeting that allows team members to get to know each other. The kick-off meeting also gives the project manager a forum in which to discuss the goals of the project and what tasks need to be completed to meet the desired goal. The program manager might also want to use this time to perform some team-building activities, such as having the group develop a team name, establish a mission statement, or even create a project logo.

As the team progresses, it typically goes through four stages:

Forming

Storming

Norming

Performing

Throughout these ups and downs, the team must stay clearly focused on the deliverables. Some deliverables are considered main objectives, and others are considered nonobjectives. The main objectives are directly linked to the success of the project; nonobjectives add value by clarifying and defining the main objectives. The project-management process usually starts with the team working on an object breakdown structure (OBS), which defines each component of the project and their relationship to each other. The OBS can help make sure the team doesn’t leave anything out and that all requirements are clearly defined. Next, a work breakdown structure (WBS) can be developed. A WBS is process oriented and shows what activities need to be completed in a hierarchical manner. It allows for easy identification of tasks that need to be completed. One advantage of the WBS is that tasks are defined as achievable work units. Figure 3.1 shows an example of a WBS.

Figure 3.1 Work breakdown structure.

Project-Management Practices

Tasks

Evaluate the project management framework and project governance practices to ensure that business objectives are achieved in a cost-effective manner while managing risks to the organization.

Perform reviews to ensure that a project is progressing in accordance with project plans, is adequately supported by documentation, and status reporting is accurate.

Knowledge Statement

Knowledge of project success criteria and risks

Project-management practices are designed to increase the likelihood of overall success. Good project-management practices can help ensure that the goals of the project are met. The three constraints of project management include the following:

Scope—The scope of the project can be better defined by understanding areas/activities/personnel needed to complete the project. For example, software projects must define how big the applications will be. Will the project involve a few thousand lines of code or millions of lines of code?

Time—Time can be better established by building a project timeline that lists each task and specifies a timeframe for each.

Cost—Cost can be determined by examining the lines of code, the number of people in the project team, and the time needed for each phase of the project.

The following sections discuss each of these constraints in more detail.

Project Initiation

Project initiation is the first stage. Sometimes attention and effort are focused on the endpoint and final deliverable. Projects must be managed carefully during the initiation stage. At this point, a project sponsor seeks to obtain funding and approval for a project through a project initiation document (PID). The PID is used to obtain authorization for the project to begin. It justifies the project to management, clearly defines the scope of the project, documents all roles and responsibilities, and sets up and runs a project office environment.

Project Planning

Project planning is the part of project management that relates to schedules and estimation. The project manager must develop a realistic estimate of how much time the project will take and determine what tasks must be accomplished. A big part of project planning involves not just time, but also the identification and quantification of all required resources. Task management is not just about handing out tasks to each member of the team; it is about determining who is most capable of accomplishing each task. Program Evaluation and Review Technique (PERT) charts and Gantt charts are two tools to help; each is discussed later in the chapter. Project planning requires that the sequence of tasks be determined. Some tasks can be performed in any order, whereas others must flow in a specific order. As an example, building a house requires the foundation to be laid before the walls can be built. Each of these tasks must have a time estimate performed; the project manager must determine how long each task will take. Needed resources will vary depending on the task. As in our previous example, a foundation requires concrete and rebar, while walls require wood and nails. The following sections look at some of the pieces of project planning. The first is estimating software cost and size.

Software Cost Estimation

Most of us put a lot of effort into cost estimation in our personal lives. When considering a new job offer, most of us look closely at the cost of living in a different area; likewise, when shopping for a new car, most people check with several dealerships to find the best deal. The business world is constrained by the same budget factors. These components drive up the cost of software:

The chosen source code language—Using an obscure or unpopular language will most likely drive up costs.

The size of the application—The size or complexity of the application has a bearing on cost. As an example, the level of security needed is something that will affect the complexity of a given application. This also has a direct correlation to the scope of the project.

The project time constraints—If a project is projected to be completed in one month versus three months, this might mean that more overtime needs to be paid, along with fees for rushed services.

Computer and resource accessibility—If resources are available only during certain times, the output of the project team will most likely be reduced.

Project team experience—Every individual has a learning curve that adds cost to inexperienced team members.

The level of security needed—A project that needs very high levels of security controls will take additional time and effort to develop.

One early cost model, developed by Barry Boehm, is known as the Constructive Cost Model (COCOMO). It was designed as a simple cost model for estimating the number of person-months required to develop software. It is now considered obsolete, replaced by COCOMO II in 1995. This model considers “what if” calculations to determine how changes to personnel, resources, or staffing affect project cost. Figure 3.2 shows what the COCOMO II model looks like. It can be run online at http://sunset.usc.edu/research/COCOMOII/index.html. COCOMO II can also be used with standard spreadsheet applications such as Microsoft Excel.

Figure 3.2 COCOMO software estimation.

Now that you have a reasonable idea of the software cost estimates, the next step is to examine methods to determine software size, in terms of lines of code.

Software Size Estimates

Traditional software sizing has been done by counting source lines of code (SLOC). This method of software sizing was developed because early programs were written in FORTRAN or other line-oriented languages. To give some idea of how big programs can be, Linux 2.6 had about 5.7 million lines of code. With programs of such size, counts are usually done in kilo lines of code (KLOC). This method does not work as well in modern development programs because additional factors affect the overall cost, such as the complexity of the application/ program being written. This method determines cost solely on one factor—length of code. Considering that development packages can generate hundreds of line of code from only a few mouse clicks demonstrates that this model is not as useful as in years past. One solution to this is function point analysis (FPA).

FPA is a method that the ISO has approved as a standard to estimate the complexity of software. FPA can be used to budget application-development costs, estimate productivity after project completion, and determine annual maintenance costs. FPA is based on the number of inputs, outputs, interfaces, files, and queries. Figure 3.3 shows an overview of function point analysis.

Figure 3.3 Function point analysis.

Per ISACA, function points are first computed by completing a table, as demonstrated in Table 3.2. The purpose of the table is to determine whether the task is simple, average, or very complex. One way to determine this subjective weighting factor is to apply the Halstead Complexity Measures (http://www.sei.cmu.edu/str/descriptions/halstead.html). The five functional point values are the number of user inputs, number of user outputs, number of user inquiries, number of files, and number of external interfaces. Take a moment to review Table 3.2.

Table 3.2 Computing Metrics of Function Point Analysis

When the table is completed, the organization can use the computed totals to run through an algorithm that determines factors such as reliability, cost, and quality, such that:

Productivity = FP/person-month

Quality = defects/FP

Cost= $/FP

With these calculations completed, the project team can identify resources needed for each specific task.

Scheduling

With software size and cost determined, the project team can turn its attention to scheduling. Scheduling involves linking individual tasks. The relationship between these tasks is linked either by earliest start date or by latest expected finish date. The Gantt chart is one way to display these relationships.

The Gantt chart was developed in the early 1900s as a tool to schedule activities and monitor progress. Gantt charts show the start and finish dates of each element of a project, and also show the relationship between each activity in a calendar-like format. They are one of the primary tools used to communicate project schedule information. Gantt charts use a baseline to illustrate what will happen if a task is finished early or late.

Program Evaluation and Review Technique (PERT) is the preferred tool for estimating time when a degree of uncertainty exists. PERT uses a critical-path method that applies a weighed average duration estimate.

PERT uses probabilistic time estimates to estimate the best and worst time estimates. Basically, PERT uses a three-point time estimate to develop best, worst, and most likely time estimates. The PERT weighted average is calculated as follows:

PERT Weighted Average = Optimistic Time + 4 × Most Likely Time + Pessimistic Time ÷ 6

A PERT chart is used to depict this information. Each chart begins with the first task that branches out to a connecting line that contains three estimates:

The most optimistic time in which the task will be completed

The most likely time in which the task will be completed

The worst-case scenario or longest the task will take

This line is then terminated by another node, which signifies the start of another task. Figure 3.4 shows an example of a PERT chart.

Figure 3.4 PERT chart.

As an example, suppose that the project team has been assigned to design an online user-registration system for the organization and that one task is to develop the Java input screen the user will use to enter personal data. It might be reasonable to estimate that this task will take five work days to complete. With PERT, the best and worst completion time would also be factored in. To get a better example of how an estimate would be performed with PERT, consider the following step-by-step example.

---

Step By Step

3.1 Calculating Time Estimates with PERT

1. Determine the average amount of time to complete the task. In the previous example, this was estimated to be five work days.

2. Estimate the best possible completion time for the task. For this step by step, assume that it is three work days.

3. Estimate the worst possible completing time. For this step, assume 10 work days.

4. Plug these values into the PERT weighted average formula:

(3 + 10 +(4 × 5) ÷ 6)= 5.5 days

5. The value of 5.5 days would be recorded as the critical path time.

---

Critical Paths

Project management mirrors life, in that anything can go wrong. Things can happen to affect the time, cost, or success of a project. That is why all project-management techniques compute the critical path, or the sequence of tasks that must be completed on schedule for the project to be finished on time. We can expand on the house-building project we used in an earlier example. Having a foundation is a task on the critical path; it must be completed before the walls and roof can be built. Exterior painting is not on the critical path and can be done at any time once the foundation, walls, and roof are completed.

Critical path methodology (CPM) determines what activities are critical and what the dependencies are between the various tasks. CPM is accomplished by the following:

Compiling a list of each task required to complete the project

Determining the time that each task will take, from start to finish

Examining the dependencies between each task

Critical tasks have little flexibility in completion time. The critical path can be determined by examining all possible tasks and identifying the longest path. Even if completed on time, this path indicates the shortest amount of time in which the project can be completed. CPM offers real advantages over Gantt, in that it identifies this minimum time. If the total project time needs to be reduced, one of the tasks on the CPM path must be finished earlier. This is called crashing, in that the project sponsor must be prepared to pay a premium for early completion as a bonus or in overtime charges. The disadvantage to CPM is that the relation of tasks is not as easily seen as it is with Gantt charts.

Timebox Management

Timebox management is used in projects when time is the most critical aspect and software projects need to be delivered quickly. It is used to lock in specifications and prevent creep. As an example, if given time, engineers might overengineer a system or decide to add more functionality or options. Although users might appreciate the added functionality, these items add time to the build phase and slow progress. Timeboxing counteracts this tendency by placing a very rigid time limit on the build of the system. When using timeboxing, the project time must never slip or be extended. If the project manager foresees time problems, he should consider corrective action, such as reducing the scope of the project or adding manpower or resources.

Project Control and Execution

Project control requires the collection, measurement, and dissemination of information and project performance. The bulk of the budget will be spent during the execution of the project. It is entirely possible that project changes might be needed. If so, the changes to the project must be recorded. Changes typically result in additional funds, manpower, or time. Auditors must be aware of any changes and must examine how this could affect any existing controls and the overall project. The auditor must also be concerned with end-user training. When new software products are released to users, the users must be trained on how the application works, what type of authentication is required, and how overrides or dual controls work.

Closing a Project

The last step in the project-management process is to close the project. Projects are, after all, temporary endeavors. At the conclusion of the project, the project manager must transfer control to the appropriate individuals. The project closing includes the following tasks:

Administrative closure

Release of final product or service

Update of organizational assets

At the close of the project, surveys or post-project reviews might be performed. This is a chance to survey the project team and end users to gauge their satisfaction with the project and examine how things could have been done differently or what changes should be implemented next time. A postmortem review is similar but is usually held after the project has been in use for some time.

Business Application Development

Tasks

Evaluate the business case for the proposed system development/acquisition to ensure that it meets the organization’s business goals.

Evaluate proposed control mechanisms for systems and/or infrastructure during specification, development/acquisition, and testing to ensure that they will provide safeguards and comply with the organization’s policies and other requirements.

Evaluate the readiness of the system and/or infrastructure for implementation and migration into production.

Evaluate the process by which systems and/or infrastructure are disposed of to ensure that they comply with the organization’s policies and procedures.

Knowledge Statements

Knowledge of benefits management practices, (e.g., feasibility studies, business cases)

Knowledge of quality assurance methods

Knowledge of the management of testing processes (e.g., test strategies, test plans, test environments, entry and exit criteria)

Knowledge of data conversion tools, techniques, and procedures

Knowledge of system and/or infrastructure disposal procedures

Knowledge of software and hardware certification and accreditation practices

Knowledge of post-implementation review objectives and methods (e.g., project closure, benefits realization, performance measurement).

Knowledge of system migration and infrastructure deployment practices

Knowledge of requirements analysis and management practices (e.g., requirements verification, traceability, gap analysis)

Knowledge of acquisition and contract management processes (e.g., evaluation of vendors, preparation of contracts, vendor management, escrow)

Business application development is largely a product of the systems development lifecycle (SDLC). New applications are typically created when new opportunities are discovered, when companies want to take advantage of new technology or use technology to solve an existing problem. Organizations use a structure approach for these reasons:

To minimize risk

To maximize return

To establish controls so that the likelihood that the software meets user needs is high

As an auditor, you are not expected to be an expert programmer or understand the inner workings of a Java program. Instead, the auditor must know how to manage the development process so that adequate controls are developed and implemented. The auditor must be able to review information at each step of the process and provide input on the adequacy of controls being designed. Auditors are also responsible for reporting independently to management on the status of the project and the implementation of controls. Auditors might also become more deeply involved in the process, based on their individual skills and abilities. Now let’s look more closely at how the SDLC process works.

Systems-Development Methodology

The SDLC is designed to produce high-quality software in a structured way that minimizes risk. The traditional approach to SDLC is the waterfall model, illustrated in Figure 3.5. The name of the waterfall model comes from the fact that progress flows from the top to the bottom progressing through each phase. W.W. Royce originally described the model as having seven phases. Some variations show it as having five or six phases. ISACA uses a modified model that has five primary phases and the post implementation phase.

Figure 3.5 The waterfall SDLC.

As Figure 3.5 illustrates, the waterfall model starts with a feasibility study and progresses through implementation. The advantage of this model is that it is well known and extremely stable when requirements are not expected to change and the architecture is well known. Table 3.5 describes each phase of the SDLC.

Table 3.5 SDLC Overview

The National Institute of Standards and Technology (NIST) defines the SDLC in NIST SP 800-34 as “the scope of activities associated with a system, encompassing the system’s initiation, development and acquisition, implementation, operation and maintenance, and ultimately its disposal that instigates another system initiation.” Therefore, the goal of the SDLC is to control the development process and add security checks at each phase. The failure to adopt a structured development model will increase risk and the likelihood that the final product may not meet the customer’s needs. The following sections describe each phase of the SDLC in more detail.

Phase 1: Feasibility

In this step, the feasibility of the project is considered. The cost of the project must be discussed, as well as the potential benefits that it will bring to the system’s users. A payback analysis must be performed to determine how long the project will take to pay for itself. In other words, the payback analysis determines how much time will lapse before accrued benefits will overtake accrued and continuing costs. Figure 3.6 illustrates an example of this. If it is determined that the project will move forward, the team will want to develop a preliminary timeline. During the feasibility phase, everyone gets a chance to meet and understand the goals of the project.

Figure 3.6 Charting the payback period.

Phase 2: Requirements Definition

This phase entails fully defining the need and then mapping how the proposed solution meets the need. This requires the participation of management as well as users. Users should also be involved because they should have input on how the applications are designed.

At this phase, an entity relationship diagram (ERD) is often used. An ERD helps map the requirements and define the relationship between elements. The basic components of an ERD are an entity and a relationship. An entity is very much like a database, in that it is a grouping of like data elements. An entity has specific attributes, which are called the entity’s primary key. Entities are drawn as a rectangular box with an identifying name. Relationships describe how entities are related to each other and are defined as a diamond. ERDs can be used to help define a data dictionary. When a data dictionary is designed, the database schema can be developed. The database schema defines the database tables and fields, and the relationship between them. Figure 3.7 shows the basic design of an ERD. The completed ERD will be used in the design phase as the blueprint for the design.

Figure 3.7 Entity relationships diagram.

During the requirements phase, auditors must verify the requirements and determine whether adequate security controls are being defined. These controls should include the following mechanisms:

Preventive—Preventive controls can include user authentication and data encryption.

Detective—Detective controls can include embedded audit modules and audit trails.

Corrective—Corrective controls can include fault tolerance controls and data integrity mechanisms.

Build Versus Buy

Although this is not a step in the SDLC, an organization might decide to buy a product instead of building it. The decision typically comes down to time, cost, and availability of a predesigned substitute. Figure 3.8 shows how this affects the SDLC process.

Figure 3.8 Build versus buy.

Before moving forward with the option to buy, the project team should develop a request for proposal (RFP) to solicit bids from vendors. Vendor responses should be closely examined to find the vendor that best meets the project team’s requirements. Some of the questions that should be asked include these:

Does the vendor have a software product that will work as is?

Will the vendor have to modify the software product to meet our needs?

Will the vendor have to create a new, nonexistent software product for us?

The reputation of the vendor is also important. Is the vendor reliable, and do references demonstrate past commitment to service? When a vendor is chosen, the last step is to negotiate and sign a contract. Auditors will want to make sure that a sufficient level of security will be designed into the product and that risks are minimized.

Phase 3: Design

During the design phase, users might not be involved, but the auditor will still be working in an advisory role. The auditor must again check that security controls are still in the design and test documents. Test plans should detail how security controls will be tested. Tests should be performed to validate specific program units, subsystems, interfaces, and backup/recovery. Change-control procedures should be developed to prevent uncontrolled changes.

There are ways to decrease design and development time. Reverse engineering is one such technique. Reverse engineering converts executable code into human-readable format and can be performed with tools such as IDA Pro. This is a somewhat controversial subject because, although reverse engineering has legitimate uses, a company could use it to disassemble another company’s program. Most software licenses make this illegal. Reverse engineering is also sometimes used to bypass access-restriction mechanisms.

Phase 4: Development

During the development phase, programmers work to develop the application code. Programmers might use online programming facilities so that many programmers can access the code directly from their workstation. Although this typically increases productivity, it also increases risk because someone might gain unauthorized access to the program library. Programmers should strive to develop modules that have high cohesion and low coupling. Cohesion addresses the fact that a module is focused on a single task. Coupling is the measurement of the interconnection between modules. Low coupling means that a change to one module should not affect another. Figure 3.9 demonstrates this concept.

Figure 3.9 High and low coupling.

During development, auditors must verify that input and output controls, audit mechanisms, and file-protection schemes are used. Examples of input controls include dollar counts, transaction counts, error detection, and correction. Examples of output controls include validity checking and authorizing controls. Testing these controls and the functionality of the program is an important part of this phase. Testing can be done by using one of the following testing methods:

Top down—Top-down testing starts with a depth or breadth approach. Its advantage is that it gets programmers working with the program so that interface problems can be found sooner. It also allows for early testing of major functions.

Bottom up—Bottom-up testing works up from the code to modules, programs, and all the way to systems. The advantage of bottom-up testing is that it can be started as soon as modules are complete; work does not have to wait until the entire system is finished. This approach also allows errors in modules to be discovered early. Most application testing follows the bottom-up approach.

Regardless of the chosen approach, test classifications are divided into the following categories:

Unit testing—Examines an individual program or module

Interface testing—Examines hardware or software to evaluate how well data can be passed from one entity to another

System testing—A series of tests that can include recovery testing, security testing, stress testing, volume testing, and performance testing. Although unit and interface testing focus on individual objects, the objective of system testing is to assess how well the system functions as a whole.

Final acceptance testing—When the project staff is satisfied with all other tests, final acceptance testing, or user acceptance testing, must be performed. This occurs before the application is implemented into a production environment.

Table 3.6 lists some other types of tests that are used for requirement verification.

Table 3.6 Testing Types

Before coding can begin, programmers must decide what programming language they will use. To some measure, the organization will decide this. For example, if the company has used FORTRAN for engineering projects for the last 25 years, it might make sense to do so for the current project. Programming has evolved, and there are five generations of programming languages:

Generation Five (5GL)—Natural language

Generation Four (4GL)—Very high-level language

Generation Three (3GL)—High-level language

Generation Two (2GL)—Assembly language

Generation One (1GL)—Machine language

Figure 3.10 shows the five generations of languages and an example of each with their syntax.

Figure 3.10 Programming languages.

Phase 5: Implementation

In the implementation phase, the application is prepared for release into its intended environment. Final user acceptance is performed, as are certification and accreditation. This is typically the final step in accepting the application and agreeing that it is ready for use. Certification is the technical review of the system or application. Certification testing might include an audit of security controls, a risk assessment, or a security evaluation. Accreditation is management’s formal acceptance of a system or application. Typically, the results of the certification testing are compiled into a report, and management’s acceptance of the report is used for accreditation. Management might request additional testing, ask questions about the certification report, or accept the results as is. Once accepted, a formal acceptance statement is usually issued.

Now it is time to roll out the application to the users. Some support functions will need to be established. Items such as maintenance, support, and technical response must be addressed. Data conversion might also need to be considered. If an existing system is being replaced, data from the system might need to be migrated to the new one. Computer-aided software engineering (CASE) is used for program and data conversions.

Hopefully, some of the users have been involved throughout the process and can help in the training process. The training strategy can include classroom training, online training, practice sessions, and user manuals. The rollout of the application might be all at once or phased in over time. Changeover techniques include the following:

Parallel operation—Both the old and new systems are run at the same time. Results between the two systems can be compared. Fine-tuning can also be performed on the new system as needed. As confidence in the new system improves, the old system can be shut down. The primary disadvantage of this method is that both systems must be maintained for a period of time.

Phased changeover—If the system is large, a phased changeover might be possible. With this method, systems are upgraded one piece at a time.

Hard changeover—This method establishes a date at which users are forced to change over. The advantage of the hard changeover is that it forces all users to change at once. However, this introduces a level of risk into the environment because things can go wrong.

Phase 6: Post-Implementation

In the post-implementation phase, some might be ready to schedule a party and declare success. What really needs to be done is to assess the overall success of the project. Actual costs versus projected costs should be reviewed to see how well cost-estimating was done at the feasibility phase. Return on investment (ROI) and payback analysis should be reviewed. A gap analysis can determine whether there is a gap between requirements that were or were not met. An independent group might conduct performance measurement, such as an audit. If this is the case, it should not be done by auditors who were involved in the SDLC process. Overall, post-implementation should answer the following questions:

Is the system adequate?

What is the true ROI?

Were the chosen standards followed?

Were good project-management techniques used?

Disposal

Applications and systems don’t last forever. This means that, at some point, these systems must be decommissioned and disposed of. This step of the process is reached when the application or system is no longer needed. Those involved in the disposal process must consider the disposition of the application. Should it be destroyed or archived, or does the information need to be migrated into a new system? Disk sanitization and destruction are also important, to ensure confidentiality. This is an important step that is sometimes overlooked. Table 3.7 outlines a sample policy for data disposal.

Table 3.7 Sample Media Destruction Policy

Review Break

The CISA needs to understand business application development. Table 3.8 outlines each step in the SDLC and its defining attributes.

Table 3.8 SDLC Implementation Phases

Alternative Application-Development Techniques

Knowledge Statement

Knowledge of system development methodologies and tools and an understanding of their strengths and weaknesses (e.g., agile development practices, prototyping, rapid application development [RAD], object-oriented design techniques)

Globalization has increased the pace of change and reduced the amount of time that organizations have to respond to changes. The new system must be brought online quickly. The SDLC is not the only development methodology used today. As an auditor, you must be knowledgeable of other development methods and have a basic understanding of their operations. Some popular models include the following:

Incremental development—Defines an approach that develops systems in stages so that development is performed one step at a time. A minimal working system might be deployed while subsequent releases build on functionality or scope.

Spiral development—The spiral model was developed based on the experience of the waterfall model. The spiral model is based on the concept that software development is evolutionary. The spiral model begins by creating a series of prototypes to develop a solution. As the project continues, it spirals out, becoming more detailed. Each step passes through planning, requirements, risks, and development phases.

Prototyping—The prototyping model reduces the time required to deploy applications. Prototyping uses high-level code to quickly turn design requirements into application screens and reports that the users can review. User feedback can fine-tune the application and improve it. Top-down testing works well with prototyping. Although prototyping clarifies user requirements, it can result in overly optimistic project timelines. Also, when change happens quickly, it might not be properly documented, which is a real concern for the auditor.

Rapid application development (RAD)—RAD uses an evolving prototype and requires heavy user involvement. Per ISACA, RAD requires well-trained development teams that use integrated power tools for modeling and prototyping. With the RAD model, strict limits are placed on development time. RAD has four unique stages, which include concept, functional design, development, and deployment.

These models share a common element, in that they all have a predictive lifecycle. This means that when the project is laid out, costs are calculated and a schedule is defined.

A second category of application development can be defined as agile software development. With this development model, teams of programmers and business experts work closely together. Project requirements are developed using an iterative approach because the project is both mission driven and component based. The project manager becomes much more of a facilitator in these situations. Popular agile development models include the following:

Extreme programming (XP)—The XP development model requires that teams include business managers, programmers, and end users. These teams are responsible for developing useable applications in short periods of time. Issues with XP are that teams are responsible not only for coding, but also for writing the tests used to verify the code. Lack of documentation is also a concern. XP does not scale well for large projects.

Scrum—Scrum is an iterative development method in which repetitions are referred to as sprints and typically last 30 days. Scrum is typically used with object-oriented technology and requires strong leadership and a team meeting each day for a short time. The idea here is for more planning and directing tasks from the project manager to the team. The project manager’s main task is to work on removing any obstacles from the team’s path.

Application-Development Approaches

Applications are written in a programming language. Programming languages can be low level so that the system easily understands the language; they also can be high level, enabling humans to easily understand them but requiring translation for the system. The programs used to turn high-level programs into object- or machine-readable code are known as interpreters, compilers, or assemblers. Most applications are compiled. The information also can be grouped for the development process in various ways, including data-oriented system development, object-oriented systems development, component-based development, and web-based application development. Table 3.10 describes these methods in more detail.

Table 3.10 Application-Development Approaches

Information Systems Maintenance Practices

Task

Evaluate the process by which systems and/or infrastructure are maintained to ensure the continued support of the organization’s objectives and are subject to effective internal control.

Knowledge Statement

Knowledge of configuration, change, and release management in relation to development and maintenance of systems and/or infrastructure

When a system moves into production, the thought might be that the work is done. This is not the case. Changes need to be made and must be done in a controlled manner. The integrity of the application and source code must be ensured. Most organizations use a change-control board that includes a senior manager as the chairperson and individuals from various organizational groups. The change-control board is responsible for developing a change-control process and also for approving changes.

Although the types of changes vary, change control follows a predictable process:

1. Request the change.

2. Approve the change request.

3. Document the change request.

4. Test the proposed change.

5. Present the results to the change-control board.

6. Implement the change, if approved.

7. Document the new configuration.

Documentation is the key to a good change-control process. All system documents should be updated to indicate any changes that have been made to the system or environment. The system maintenance staff or department responsible for requesting the change should keep a copy of the change approval.

The auditor should ensure that backup copies of critical documents are created. These documents should be kept off-site in case of a disaster or other situation. The auditor should also watch for the possibility of unauthorized changes because of poor oversight or the lack of proper security controls. Items to look for include the following:

Changes are implemented directly by the software vendor, without internal control.

Programmers place code in an application that has not been tested or validated.

The changed source code has not been reviewed by the proper employee.

No formal change process is in place.

The change review board has not authorized the change.

The programmer has access to both the object code and the production library.

Finally, this does not mean that a situation will never arise in which a change does not have to go through the change-control process. Emergency changes might have to be made. These typically occur because of situations that endanger production or halt a critical process. In these situations, it is important to maintain the integrity of the process. These changes should be followed up by procedures to ensure that standard controls are applied retroactively. If programmers are given special access or an increased level of control, the accounts and mechanisms they use should be closely monitored.

Chapter Summary

This chapter covered the systems development lifecycle and project management. These topics are important to the auditor from the standpoint of governance. Throughout these processes, the auditor has a leadership role. This chapter’s goal was to introduce the auditor to the various steps, review the activity performed at each stage, and help the auditor become familiar with the terms and concepts used in software development.

Key Terms

Accreditation

Bottom-up testing

Certification

Critical path methodology (CPM)

Entity relationship diagram (ERD)

Function point analysis (FPA)

Kilo lines of code (KLOC)

Object breakdown structure (OBS)

Reverse engineering

Source lines of code (SLOC)

Top-down testing

Work breakdown structure (WBS)

Apply Your Knowledge

This chapter introduced you to the role an auditor plays in the development of new applications and systems. As such, it is important that you understand some common project-management techniques.

Exercises

3.1 Project Management

In this exercise, examine different project-management techniques.

Estimated Time: 30 Minutes

Part 1

1. Find someone who works as a project manager at your organization, and see if you can interview this person by phone, by email, or in person. Formulate several questions to ask about the project-management process. Ask what types of concerns he has when working on software projects. Ask what methods are used to calculate project time and whether PERT, CPM, RAD, prototyping, or other techniques are used.

2. Search the Web for function point analysis and source lines of code. Record the number of hits you get for each, and examine any interesting articles found.

Part 2

1. Examine Table 3.11 and calculate the totals with the values shown.

Table 3.11 Function Point Analysis

2. Next, go to http://www.effortestimator.com/ and download the demo version of FPApal.

3. Once downloaded, install the program, select Open Project from the File menu, and enter the same values, as shown in Table 3.11. Figure 3.11 shows the program.

Figure 3.11 FPApal.

4. Compare the results between the manual and automated methods. Were they the same?
Yes ___ No ____

Exam Questions

1. During which step of the SDLC is final user acceptance usually performed?

A. Design

B. Development

C. Implementation

D. Requirements

2. When planning to add time constraints to a project, which of the following should be examined most closely?

A. Budget

B. Critical path

C. Skills of the project team

D. Tasks that require the most time

3. During the post-implementation review, which of the following activities should be performed?

A. Perform an ROI

B. Design the audit trail

C. Complete an entity relationship diagram

D. Perform acceptance testing

4. Which of the following types of tests is used to verify that the proposed design will function in its intended environment?

A. Regression testing

B. Function testing

C. Pilot testing

D. Sociability testing

5. Which of the following development methods is known to not work well for large projects?

A. Spiral model

B. Rapid application development

C. Scrum

D. Extreme programming

6. Programming languages that most closely map to database management are found at what generational level?

A. 2GL

B. 3GL

C. 4GL

D. 5GL

7. Which of the following does the PERT weighted average consider?

A. High cost, low cost, best cost

B. Average cost plus 5%

C. Best time, worst time, average time

D. Average time plus 5%

8. This type of changeover process requires all users to get up to speed in advance so that a defined changeover can be set to a fixed date.

A. Fallback scenario

B. Abrupt changeover

C. Phased changeover

D. Parallel changeover

9. Entity relationship diagrams are built using two essential components. These include which of the following?

A. Processes and attributes

B. Processes and decision blocks

C. Entities and relationships

D. Nouns and adverbs

10. Which of the following development techniques uses short cycles, referred to as sprints, and is focused on object-oriented technology?

A. Spiral model

B. Rapid application development

C. Scrum

D. Extreme programming

Answers to Exam Questions

1. C. Implementation is the stage at which user acceptance is usually performed. Therefore, answers A, B, and D are incorrect.

2. B. The critical path is the sequence of activities that must be completed on time for the project to stay on schedule. Delays of any items on the critical path will slow the entire project. Answers A, C, and D are incorrect because, although the budget, team skills, and individual tasks are all items to consider, the critical path should be examined first because that will affect all other items.

3. A. Following implementation, a cost-benefit analysis or ROI should be performed. Answer B is incorrect because the audit trail should be designed during the design phase. Answer C is incorrect because an ERD should be performed during the requirements phase. Answer D is incorrect because final acceptance testing should be performed during the implementation phase.

4. D. Sociability testing is performed to confirm that a new or modified system will work in its intended environment. Answer A is incorrect because regression testing verifies that changes have not introduced errors. Answer B is incorrect because function testing verifies that systems meet specifications. Answer C is incorrect because pilot testing is used for limited evaluations.

5. D. Extreme programming does not work well for large project teams. Extreme programming requires that teams include business managers, programmers, and end users. These teams are responsible for developing useable applications in short periods of time. Answer A is incorrect because the spiral model is based on the concept that software development is evolutionary. The spiral model begins by creating a series of prototypes to develop a solution. As the project continues, it spirals out, becoming more detailed. Each step passes through planning, requirements, risks, and development phases. Answer B is incorrect because RAD requires well-trained development teams that use integrated power tools for modeling and prototyping. Answer C is incorrect because scrum uses short cycles referred to as sprints and is focused on object-oriented technology.

6. C. Fourth-generation languages (4GL) are most commonly used for databases. Examples of 4GLs include Focus, natural, and dBase. Answer A is incorrect because 2GL is assembly language. Answer B is incorrect because 3GL includes languages such as FORTRAN, Pascal, and C. Answer D is incorrect because 5GLs are very high-level languages such as LISP and PROLOG.

7. C. PERT is used to schedule, organize, and coordinate tasks. The PERT weighted average examines the shortest time, average time, and longest time a task is scheduled to be completed. Therefore answers A, B, and D are incorrect.

8. B. The abrupt changeover requires the establishment of a cut-off date so that all users must switch to the new system by then. Answer A is incorrect because a fallback scenario is used for data conversion and to ensure that the new application is ready for use. Answer C is incorrect because a phased changeover is gradual. Answer D is incorrect because a parallel changeover brings the new system online while the old is still in operation.

9. C. Entity relationship diagrams are built using two essential components that include entities and relationships. Therefore answers A, B, and D are incorrect.

10. C. Scrum uses short cycles referred to as sprints and is focused on object-oriented technology. Answer A is incorrect because the spiral model is based on the concept that software development is evolutionary. The spiral model begins by creating a series of prototypes to develop a solution. As the project continues, it spirals out, becoming more detailed. Each step passes through planning, requirements, risks, and development phases. Answer B is incorrect because RAD requires well-trained development teams that use integrated power tools for modeling and prototyping. Answer D is incorrect because extreme programming requires that teams include business managers, programmers, and end users. These teams are responsible for developing useable applications in short periods of time.

Need to Know More?

Which Development Method Is Right for Your Company: http://tinyurl.com/2do5ws

Prototyping: http://www.umsl.edu/~sauter/analysis/prototyping/proto.html

On Overview of RAD: http://csweb.cs.bgsu.edu/maner/domains/RAD.htm

Fundamentals of Function Point Analysis: http://www.ifpug.com/fpafund.htm

PERT, CPM, and Gantt: http://studentweb.tulane.edu/~mtruill/dev-pert.html

Building Security Controls into the SDLC: http://tinyurl.com/27hurl

Auditing the SDLC: http://tinyurl.com/24x7ah