This chapter covers CISA Domain 1, “The Process of Auditing Information Systems.” The topics in this chapter represent 21 percent of the CISA examination.

The IS audit process is the procedural structure used by auditors to assess and evaluate the effectiveness of the IT organization and how well it supports the organization’s overall goals and objectives. The audit process is backed up by the Information Technology Assurance Framework (ITAF) and the ISACA code of ethics. The ITAF is used to ensure that auditors will take a consistent approach from one audit to the next throughout the entire industry. This will help to advance the entire audit profession and facilitate its gradual improvement over time.

Audit Management

An organization’s audit function should be managed so that an audit charter, strategy, and program can be established; audits performed; recommendations enacted; and auditor independence assured throughout. The audit function should align with the organization’s mission and goals, and work well alongside IT governance and operations.

The Audit Charter

As with any formal, managed function in the organization, the audit function should be defined and described in a charter document. The charter should clearly define roles and responsibilities that are consistent with ISACA audit standards and guidelines, including but not limited to ethics, integrity, and independence. The audit function should have sufficient authority that its recommendations will be respected and implemented, but not so much power that the audit tail will wag the IS dog.

The Audit Program

Audit program is the term used to describe the audit strategy and audit plans that include scope, objectives, resources, and procedures used to evaluate a set of controls and deliver an audit opinion. You could say that an audit program is the plan for conducting audits over a given period.

The term program in audit program is intended to evoke a similar “big picture” point of view as the term program manager does. A program manager is responsible for the performance of several related projects in an organization. Similarly, an audit program is the plan for conducting several audits, types of audits, or audits of varying scope in an organization.

Strategic Audit Planning

The purpose of audit planning is to determine the audit activities that need to take place in the future, including an estimate of the resources (tools, budget, and manpower) required to support those activities.

Factors That Affect an Audit

Like security planning, audit planning must take into account several factors:

•  Organization’s strategic goals and objectives   The organization’s overall goals and objectives should flow down to individual departments and their support of these goals and objectives. These goals and objectives will translate into business processes, technology to support business processes, controls for both the business processes and technologies, and audits of those controls. This is depicted in Figure 3-1.

Figure 3-1   The organization’s goals and objectives translate down into audit activities.

•  New organization initiatives   Closely related to goals and objectives, organizations often embark on new initiatives, whether new products, new services, or new ways of delivering existing products and services.

•  Market conditions   Changes in the product or service market may have an impact on auditing. For instance, in a product or services market where security is becoming more important, market competitors could decide to voluntarily undergo audits in order to show that their products or services are safer or better than those from competing organizations. Other market players may need to follow suit for competitive parity. Changes in the supply or demand of supply-chain goods or services can also affect audits.

•  Changes in technology   Enhancements in the technologies that support business processes may affect business or technical controls, which in turn may affect audit procedures for those controls. Organizations moving their applications or services from on-premise to the cloud is a good example of this.

•  Changes in regulatory requirements   Changes in technologies, markets, or security-related events can result in new or changed regulations. Maintaining compliance may require changes to the audit program. In the 20-year period preceding the publication of this book, many new information security–related regulations have been passed or updated, including the Gramm-Leach-Bliley Act (GLBA), the Sarbanes-Oxley Act (SOX), and the Health Insurance Portability and Accountability Act (HIPAA), the European Data Protection Directive, as well as national and state laws on privacy.

All of the changes listed here usually translate into new business processes or changes in existing business process. Often, this also involves changes to information systems and changes to the controls supporting systems and processes.

Changes in Audit Activities

These external factors may affect auditing in the following ways:

•  New internal audits   Business and regulatory changes sometimes compel organizations to audit more systems or processes. For instance, after passage of the Sarbanes-Oxley Act of 2002, U.S. publicly traded companies had to begin conducting internal audits of those IT systems that support financial business processes.

•  New external audits   New regulations or competitive pressures could introduce new external audits. For example, virtually all banks and many merchants had to begin undergoing external Payment Card Industry Data Security Standard (PCI-DSS) audits when that standard was established.

•  Market competition   In certain industries such as financial services, service providers are voluntarily undertaking new audits such as SOC 1 (SSAE 16 in the United States and ISAE 3402 elsewhere), SOC 2, TRUSTe, and ISO/IEC 27001 certification, partly to support marketing claims that their security is superior to that of their competitors’.

•  Increase in audit scope   The scope of existing internal or external audits could increase to include more processes or systems.

•  Impacts on business processes   This could take the form of additional steps in processes or procedures, or additions/changes in recordkeeping or record retention.

•  Changes in audit standards   Also undergoing continuous improvement, general and specific audit rules occasionally change, which might alter sampling methodologies as well as audit procedures. For example, the PCI-DSS 3.0 update requires penetration tests to include network segmentation validation, which can result in significant increases in the time required for penetration testing.

Resource Planning

At least once per year, management needs to consider all of the internal and external factors that affect auditing to determine the resources required to support these activities. Primarily, resources will consist of the budget for external audits and manpower for internal audits.

Additional external audits usually require additional man-hours to meet with external auditors; discuss scope; coordinate meetings with process owners and managers; discuss audits with process owners and managers; discuss audit findings with auditors, process owners, and management; and organize remediation work.

Internal and external audits usually require information systems to track audit activities and store evidence. Taking on additional audit activities may require additional capacity on these systems or new systems altogether.

Additional internal audits require all of the previously mentioned factors, plus time for performing the internal audits themselves. All of these details are discussed in this chapter, and in the rest of this book.

Audit and Technology

ISACA auditing standards require that the auditor retain technical competence. With the continuation of technology and business process innovation, auditors need to continue learning about new technologies, how they support business processes, and how they are controlled. Like many professions, IS auditing requires continuing education to stay current with changes in technology.

Some of the ways that an IS auditor can update their knowledge and skills include

•  ISACA training and conferences   As the developer of the CISA certification, ISACA offers many valuable training and conference events, including:

•  Computer Audit, Control, and Security (CACS) Conference

•  Governance, Risk, and Control (GRC) Conference

•  Cybersecurity Nexus (CSX) Conference

•  ISACA Training Week

•  University courses   This can include both for-credit and noncredit classes on new technologies. Some universities offer certificate programs on many new technologies; this can give an auditor a real boost of knowledge, skills, and confidence.

•  Vo-tech (vocational-technical) training   Many organizations offer training in information technologies, including MIS Training Institute, SANS Institute, and ISACA.

•  Training webinars   These events are usually focused on a single topic and last from one to three hours. ISACA and many other organizations offer training webinars, which are especially convenient since they require no travel and many are offered at no cost.

•  ISACA chapter training   Many ISACA chapters offer training events so that local members can acquire new knowledge and skills close to where they live.

•  Other security association training   Many other security-related trade associations offer training, including ISSA (International Systems Security Association), SANS Institute, and IIA (The Institute of Internal Auditors). Training sessions are offered online, in classrooms, and at conferences.

•  Security conferences   Several security-related conferences include lectures and training. These conferences include those hosted by RSA, SANS, ISSA, Gartner, and SecureWorld Expo. Many local ISACA and ISSA chapters organize local conferences that include training.

Audit Laws and Regulations

Laws and regulations are one of the primary reasons why organizations perform internal and external audits. Regulations on industries generally translate into additional effort on the part of target companies to track their compliance. This tracking takes on the form of internal auditing, and new regulations sometimes also require external audits. And while other factors such as competitive pressures can compel an organization to begin or increase auditing activities, this section discusses laws and regulations that require auditing.

Almost every industry sector is subject to laws and regulations that affect organizations’ use of information and information systems. These laws are concerned primarily with one or more of the following characteristics and uses of information and information systems:

•  Security   Some information in information systems is valuable and/or sensitive, such as financial and medical records. Many laws and regulations require such information to be protected so that it cannot be accessed by unauthorized parties and require that information systems be free of defects, vulnerabilities, malware, and other threats.

•  Integrity   Some regulations are focused on the integrity of information to ensure that it is correct and that the systems it resides on are free of vulnerabilities and defects that could make or allow improper changes.

•  Privacy   Many information systems store information that is considered private. This includes financial records, medical records, and other information about people that they feel should be protected.

Computer Security and Privacy Regulations

This section contains several computer security and privacy laws in the United States, Canada, Europe, and elsewhere. The laws here fall into one or more of the following categories:

•  Computer trespass   Some of these laws bring the concept of trespass forward into the realm of computers and networks, making it illegal to enter a computer or network unless there is explicit authorization.

•  Protection of sensitive information   Many laws require that sensitive information be protected, and some include required public disclosures in the event of a breach of security.

•  Collection and use of information   Several laws define the boundaries regarding the collection and acceptable use of information, particularly private information.

•  Offshore data flow   Some security and privacy laws place restrictions or conditions on the flow of sensitive data (usually about citizens) out of a country.

•  Law enforcement investigative powers   Some laws clarify and expand the search and investigative powers of law enforcement.

The consequences of the failure to comply with these laws vary. Some laws have penalties written in as a part of the law; however, the absence of an explicit penalty doesn’t mean there aren’t any! Some of the results of failing to comply include

•  Loss of reputation   Failure to comply with some laws can make front-page news, with a resulting reduction in reputation and loss of business. For example, if an organization suffers a security breach and is forced to notify customers, word may spread quickly and be picked up by news media outlets, which will help spread the bad news further.

•  Loss of competitive advantage   An organization that has a reputation for sloppy security may begin to see its business diminish and move to its competitors. A record of noncompliance may also result in difficulty wining new business contracts.

•  Government sanctions   Breaking many federal laws may result in sanctions from local, regional, or national governments, including losing the right to conduct business.

•  Lawsuits   Civil lawsuits from competitors, customers, suppliers, and government agencies may be the result of breaking some laws. Plaintiffs may file lawsuits against an organization even if there were other consequences.

•  Fines   Monetary consequences are frequently the result of breaking laws.

•  Prosecution   Many laws have criminalized behavior such as computer trespass, stealing information, or filing falsified reports to government agencies. Breaking some laws may result in imprisonment.

Knowledge of these consequences provides an incentive to organizations to develop management strategies to comply with the laws that apply to their business activities. These strategies often result in the development of controls that define required activities and events, plus analysis and internal audit to determine if the controls are effectively keeping the organization in compliance with those laws. While organizations often initially resist undertaking these additional activities, they usually accept them as a requirement for doing business and seek ways of making them more cost-efficient in the long term.

Determining Applicability of Regulations   An organization should take a systematic approach to determine the applicability of regulations as well as the steps required to attain compliance with applicable regulations and remain in a compliant state.

Determination of applicability often requires the assistance of legal counsel who is an expert on government regulations, as well as experts in the organization who are familiar with the organization’s practices.

Next, the language in the applicable law or regulation needs to be analyzed and a list of compliant and noncompliant practices identified. These are then compared with the organization’s practices to determine which practices are compliant and which are not. Those practices that are not compliant need to be corrected; one or more accountable individuals need to be appointed to determine what is required to achieve and maintain compliance.

Another approach is to outline the required (or forbidden) practices specified in the law or regulation and then “map” the organization’s relevant existing activities into the outline. Where gaps are found, processes or procedures will need to be developed to bring the organization into compliance.

U.S. Regulations   Selected security and privacy laws and standards applicable in the United States include

•  Privacy Act of 1974

•  Access Device Fraud, 1984

•  Computer Fraud and Abuse Act of 1984

•  Electronic Communications Privacy Act (ECPA) of 1986

•  Computer Matching and Privacy Protection Act of 1988

•  Communications Assistance for Law Enforcement Act (CALEA) of 1994

•  Economic and Protection of Proprietary Information Act of 1996

•  Health Insurance Portability and Accountability Act (HIPAA) of 1996

•  Economic Espionage Act (EEA), 1996

•  No Electronic Theft (NET) Act, 1997

•  Digital Millennium Copyright Act (DMCA), 1998

•  Children’s Online Privacy Protection Act (COPPA) of 1998

•  Identity Theft and Assumption Deterrence Act of 1998

•  Gramm-Leach-Bliley Act (GLBA) of 1999

•  Federal Energy Regulatory Commission (FERC), an agency with legally binding standards

•  Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism (USA PATRIOT) Act of 2001 (expired in 2015, succeeded by the USA Freedom Act)

•  Sarbanes-Oxley Act of 2002

•  Federal Information Security Management Act (FISMA) of 2002

•  Controlling the Assault of Non-Solicited Pornography and Marketing (CAN-SPAM) Act of 2003

•  California privacy law SB 1386 of 2003

•  Identity Theft and Assumption Deterrence Act of 2003

•  Basel II, 2004, an international accord

•  Payment Card Industry Data Security Standard (PCI-DSS), 2004; updated 2016

•  North American Electric Reliability Corporation (NERC), 1968/2006, an agency with legally binding standards

•  Massachusetts security breach law, 2007

•  Red Flags Rule, 2008

•  Health Information Technology for Economic and Clinical Health Act (HITECH) of 2009

•  USA Freedom Act, 2015

Canadian Regulations   Selected security and privacy laws and standards in Canada include

•  Interception of Communications (Section 184 of the Canada Criminal Code)

•  Unauthorized Use of Computer (Section 342.1 of the Canada Criminal Code)

•  Privacy Act, 1983

•  Personal Information Protection and Electronic Documents Act (PIPEDA), 2000

•  Digital Privacy Act, 2015

European Regulations   Selected security and privacy laws and standards from Europe include

•  Convention for the Protection of Individuals with Regard to Automatic Processing of Personal Data, 1981, Council of Europe

•  Computer Misuse Act (CMA), 1990, UK

•  Directive on the Protection of Personal Data (95/46/EC), 2003, European Union

•  Data Protection Act (DPA) 1998, UK

•  Regulation of Investigatory Powers Act 2000, UK

•  Anti-Terrorism, Crime, and Security Act 2001, UK

•  Privacy and Electronic Communications Regulations 2003, UK

•  Fraud Act 2006, UK

•  Police and Justice Act 2006, UK

•  European Data Protection Directive, 2016

Other Regulations   Selected security and privacy laws and standards from the rest of the world include

•  Cybercrime Act, 2001, Australia

•  Information Technology Act, 2000, India

ISACA Auditing Standards

(ISACA has published its Information Technology Assurance Framework in the ITAF: A Professional Practices Framework for IS Audit/Assurance (currently in its third edition and available free of charge at www.isaca.org/ITAF). ITAF consists of the ISACA Code of Professional Ethics, IS audit and assurance standards, IS audit and assurance guidelines, and IS audit and assurance tools and techniques. This section discusses the Code of Professional Ethics, standards, and guidelines. The relationship between these is illustrated in Figure 3-2.

Figure 3-2   Relationship between ISACA audit standards, audit guidelines, and Code of Professional Ethics

ISACA Code of Professional Ethics

Like many professional associations, ISACA has published a Code of Professional Ethics. The purpose of the code is to define principles of professional behavior that are based on the support of standards, compliance with laws and standards, and the identification and defense of the truth.

Audit and IT professionals who earn the CISA certification are required to sign a statement that declares their support of the ISACA Code of Professional Ethics. If someone who holds the CISA certification is found to be in violation of the code, he or she may be disciplined and possibly lose his or her certification. The full text of the ISACA Code of Professional Ethics can be found at www.isaca.org/ethics.

ISACA Audit and Assurance Standards

The ISACA audit and assurance standards framework, known as the Information Technology Assurance Framework (ITAF) defines minimum standards of performance related to security, audits, and the actions that result from audits. This section lists the standards and paraphrases each.

The full text of these standards is available at www.isaca.org/standards.

1001, Audit Charter

Audit activities in an organization should be formally defined in an audit charter. This should include statements of scope, responsibility, and authority for conducting audits. Senior management should support the audit charter through direct signature or by linking the audit charter to corporate policy.

1002, Organizational Independence

The IS auditor’s placement in the command-and-control structure of the organization should ensure that the IS auditor can act independently.

1003, Professional Independence

Behavior of the IS auditor should be independent of the auditee. The IS auditor should take care to avoid even the appearance of impropriety.

1004, Reasonable Expectation

IS auditors and assurance professionals shall have a reasonable expectation that an audit engagement can be completed according to ISACA and other audit standards, that the audit scope enables completion of the audit, and that management understands its obligations and responsibilities.

1005, Due Professional Care

IS auditors and assurance professionals shall exercise due professional care, including but not limited to conformance with applicable audit standards.

1006, Proficiency

IS auditors and assurance professionals shall possess adequate skills and knowledge on the performance of IS audits and of the subject matter being audited, and shall continue in their proficiency through regular continuing professional education and training.

1007, Assertions

IS auditors and assurance professionals shall review audit assertions to determine whether they are capable of being audited, and whether the assertions are valid and reasonable.

1008, Criteria

IS auditors and assurance professionals shall select objective, measurable, and reasonable audit criteria.

1201, Engagement Planning

IS auditors shall perform audit planning work to ensure that the scope and breadth of an audit is sufficient to meet the organization’s needs, that it is in compliance with applicable laws, and that it is risk based.

1202, Risk Assessment in Planning

The IS auditor should use a risk-based approach when making decisions about which controls and activities should be audited and the level of effort expended in each audit. These decisions should be documented in detail to avoid any appearance of partiality.

A risk-based approach looks not only at security risks, but at overall business risk. This will probably include operational risk and may include aspects of financial risk.

1203, Performance and Supervision

IS auditors shall conduct an audit according to the plan and on schedule; shall supervise audit staff; shall accept and perform audit tasks only within their competency; and shall collect appropriate evidence, document the audit process, and document findings.

1204, Materiality

The IS auditor should consider materiality when prioritizing audit activities and allocating audit resources. During audit planning, the auditor should consider whether ineffective controls or an absence of controls could result in a significant deficiency or material weakness.

In addition to auditing individual controls, the auditor should consider the effectiveness of groups of controls and determine if a failure across a group of controls would constitute a significant deficiency or material weakness. For example, if an organization has several controls regarding the management and control of third-party service organizations, failures in many of those controls could represent a significant deficiency or material weakness overall.

1205, Evidence

The IS auditor should gather sufficient evidence to develop reasonable conclusions about the effectiveness of controls and procedures. The IS auditor should evaluate the sufficiency and integrity of audit evidence, and this evaluation should be included in the audit report.

Audit evidence includes the procedures performed by the auditor during the audit, the results of those procedures, source documents and records, and corroborating information. Audit evidence also includes the audit report.

1206, Using the Work of Other Experts

An IS auditor should consider using the work of other auditors when and where appropriate. Whether an auditor can use the work of other auditors depends on several factors, including:

•  The relevance of the other auditors’ work

•  The qualifications and independence of the other auditors

•  Whether the other auditors’ work is adequate (this will require an evaluation of at least some of the other auditors’ work)

•  Whether the IS auditor should develop additional test procedures to supplement the work of another auditor(s)

If an IS auditor uses another auditor’s work, his report should document which portion of the audit work was performed by the other auditor, as well as an evaluation of that work.

1207, Irregularity and Illegal Acts

IS auditors should have a healthy but balanced skepticism with regard to irregularities and illegal acts: The auditor should recognize that irregularities and/or illegal acts could be ongoing in one or more of the processes that he is auditing. He should recognize that management may or may not be aware of any irregularities or illegal acts.

The IS auditor should obtain written attestations from management that state management’s responsibilities for the proper operation of controls. Management should disclose to the auditor any knowledge of irregularities or illegal acts.

If the IS auditor encounters material irregularities or illegal acts, he should document every conversation and retain all evidence of correspondence. The IS auditor should report any matter of material irregularities or illegal acts to management. If material findings or irregularities prevent the auditor from continuing the audit, the auditor should carefully weigh his options and consider withdrawing from the audit. The IS auditor should determine if he is required to report material findings to regulators or other outside authorities. If the auditor is unable to report material findings to management, he should consider withdrawing from the audit engagement.

1401, Reporting

The IS auditor should develop an audit report that documents the process followed, inquiries, observations, evidence, findings, conclusions, and recommendations from the audit. The audit report should follow an established format that includes a statement of scope, period of coverage, recipient organization, controls or standards that were audited, and any limitations or qualifications. The report should contain sufficient evidence to support the findings of the audit.

1402, Follow-up Activities

After the completion of an audit, the IS auditor should follow up at a later time to determine if management has taken steps to make any recommended changes or apply remedies to any audit findings.

ISACA Audit and Assurance Guidelines

ISACA audit and assurance guidelines contain information that helps the auditor understand how to apply ISACA audit standards. These guidelines are a series of articles that clarify the meaning of the audit standards. They cite specific ISACA IS audit standards and COBIT controls, and provide specific guidance on various audit activities. ISACA audit guidelines also provide insight into why each guideline was developed and published.

The full text of these guidelines is available at www.isaca.org/guidelines.

2001, Audit Charter

This guideline provides information on the following IS audit standards topics:

•  Mandate

•  Contents of the audit charter

2002, Organizational Independence

This guideline provides information on the following IS audit standards topics:

•  Position in the enterprise

•  Reporting level

•  Non-audit services

•  Assessing independence

•  Audit charter and audit plan

2003, Professional Independence

This guideline provides information on the following IS audit standards topics:

•  Conceptual framework

•  Threats and safeguards

•  Managing threats

•  Non-audit services or roles

•  Non-audit services or roles that do not impair independence

•  Non-audit services or roles that do impair independence

•  Relevance of independence when providing non-audit services or roles

•  Governance of the admissibility of non-audit services or roles

•  Reporting

2004, Reasonable Expectation

This guideline provides information on the following IS audit standards topics:

•  Standards and regulations

•  Scope

•  Scope limitations

•  Information

•  Acceptance of a change in engagement terms

2005, Due Professional Care

This guideline provides information on the following IS audit standards topics:

•  Professional skepticism and competency

•  Application

•  Life cycle of the engagement

•  Communication

•  Managing information

2006, Proficiency

This guideline provides information on the following IS audit standards topics:

•  Professional competence

•  Evaluation

•  Reaching the desired level of competence

2007, Assertions

This guideline provides information on the following IS audit standards topics:

•  Assertions

•  Subject matter and criteria

•  Assertions developed by third parties

•  Conclusion and report

2008, Criteria

This guideline provides information on the following IS audit standards topics:

•  Selection and use of criteria

•  Suitability

•  Acceptability

•  Source

•  Change in criteria during the audit engagement

2201, Engagement Planning

This guideline provides information on the following IS audit standards topics:

•  IS audit plan

•  Objectives

•  Scope and business knowledge

•  Risk-based approach

•  Documenting the audit engagement project plan

•  Changes during the course of the audit

2202, Risk Assessment in Planning

This guideline provides information on the following IS audit standards topics:

•  Risk assessment of the IS audit plan

•  Risk assessment methodology

•  Risk assessment of individual audit engagements

•  Audit risk

•  Inherent risk

•  Control risk

•  Detection risk

2203, Performance and Supervision

This guideline provides information on the following IS audit standards topics:

•  Performing the work

•  Roles and responsibilities, knowledge and skills

•  Supervision

•  Evidence

•  Documenting

•  Findings and conclusions

2204, Materiality

This guideline provides information on the following IS audit standards topics:

•  IS vs. financial audit engagements

•  Assessing materiality of the subject matter

•  Materiality and controls

•  Materiality and reportable issues

2205, Evidence

This guideline provides information on the following IS audit standards topics:

•  Types of evidence

•  Obtaining evidence

•  Evaluating evidence

•  Preparing audit documentation

2206, Using the Work of Other Experts

This guideline provides information on the following IS audit standards topics:

•  Considering the use of work of other experts

•  Assessing the adequacy of other experts

•  Planning and reviewing the work of other experts

•  Evaluating the work of other experts who are not part of the audit engagement team

•  Additional test procedures

•  Audit opinion or conclusion

2207, Irregularity and Illegal Acts

This guideline provides information on the following IS audit standards topics:

•  Irregularities and illegal acts

•  Responsibilities of management

•  Responsibilities of the professionals

•  Irregularities and illegal acts during engagement planning

•  Designing and reviewing engagement procedures

•  Responding to irregularities and illegal acts

•  Internal reporting

•  External reporting

2208, Audit Sampling

This guideline provides information on the following IS audit standards topics:

•  Sampling

•  Design of the sample

•  Selection of the sample

•  Evaluation of sample results

•  Documentation

2401, Reporting

This guideline provides information on the following IS audit standards topics:

•  Types of engagements

•  Required contents of the audit engagement report

•  Subsequent events

•  Additional communication

2402, Follow-up Activities

This guideline provides information on the following IS audit standards topics:

•  Follow-up process

•  Management’s proposed actions

•  Assuming the risk of not taking corrective action

•  Follow-up procedures

•  Timing and scheduling of follow-up activities

•  Nature and extent of follow-up activities

•  Deferring follow-up activities

•  Form of follow-up responses

•  Follow-up by professionals on external audit recommendations

•  Reporting of follow-up activities

Risk Analysis

In the context of an audit, risk analysis is the activity that is used to determine the areas that warrant additional examination and analysis.

In the absence of a risk analysis, an IS auditor is likely to follow his or her “gut instinct” and apply additional scrutiny in areas where they feel risks are higher. Or, an IS auditor might give all areas of an audit equal weighting, putting equal resources into low-risk areas and high-risk areas. Either way, the result is that an IS auditor’s focus is not necessarily on the areas where risks really are higher. This results in a disservice to the audit client.

Auditors’ Risk Analysis and the Corporate Risk Management Program

A risk analysis that is carried out by IS auditors is distinct and separate from risk analysis that is performed as part of the corporate risk management program. Often, these risk analyses are carried out by different personnel and for somewhat differing reasons. A comparison of IS auditor and IS management risk analysis is shown in Table 3-1.

Table 3-1   Comparison of IS Audit and IS Management Risk Analysis

In Table 3-1, I am not attempting to show a polarity of focus and results, but instead a tendency for focus based on the differing missions and objectives of IS audit and IS management.

The ISACA Risk IT Framework

Auditors’ risk analysis and corporate risk management can both be performed using the ISACA Risk IT Framework. This framework, which is depicted in Figure 3-3, approaches risk from the enterprise perspective, encompassing all types of business risk, including IT risk.

Figure 3-3   The ISACA Risk IT Framework high-level components

The Risk IT Framework consists of three major activities:

•  Risk governance   The purpose of risk governance is to ensure that IT risk is integrated into the organization’s enterprise risk management (ERM) program.

•  Risk evaluation   The purpose of risk evaluation is to provide a framework of processes for performing risk assessments against business assets and explained in business terms.

•  Risk response   The purpose of risk response is to provide a framework of processes for responding to identified risks through reporting and risk treatment.

Like other business frameworks, the Risk IT Framework contains detailed top-down explanations of business processes and includes references to COBIT, ISACA audit standards, and Val IT—another ISACA framework concerned with achieving business value from IT investments.

The Risk IT Framework is available from ISACA at www.isaca.org/riskit.

Evaluating Business Processes

The first phase of a risk analysis is an evaluation of business processes. The purpose of evaluating business processes is to determine the purpose, importance, and effectiveness of business activities. While parts of a risk analysis may focus on technology, remember that technology exists to support business processes, not the other way around.

When a risk analysis starts with a focus on business processes, it is appropriate to consider the entire process and not just the technology that supports it. When examining business processes, it is important to obtain all available business process documentation, including:

•  Charter or mission statement   Often, an organization will develop and publish a high-level document that describes the process in its most basic terms. This usually includes the reason that the process exists and how it contributes to the organization’s overall goals and objectives.

•  Process architecture   A complex process may have several procedures, flows of information (whether in electronic form or otherwise), internal and external parties that perform functions, assets that support the process, and the locations and nature of records. In a strictly IT-centric perspective, this would be a data flow diagram or an entity-relationship diagram, but starting with either of those would be too narrow a focus. Instead, it is necessary to look at the entire process, with the widest view of its functions and connections with other processes and parties.

•  Procedures   Looking closer at the process will reveal individual procedures—documents that describe the individual steps taken to perform activities that are part of the overall process. Procedure documents usually describe who (if not by name, then by title or department) performs what functions with what tools or systems. Procedures will cite business records that might be faxes, reports, databases, phone records, application transactions, and so on.

•  Records   Business records contain the events that take place within a business process. Records will take many forms, including faxes, computer reports, electronic worksheets, database transactions, receipts, canceled checks, and e-mail messages.

•  Information system support   When processes are supported by information systems, it is necessary to examine all available documents that describe information systems that support business processes. Examples of documentation are architecture diagrams, requirements documents (which were used to build, acquire, or configure the system), computer-run procedures, network diagrams, database schemas, and so on.

In addition to reviewing documentation, it is essential that personnel involved with each process be interviewed, so that they can describe their understanding of the process, its procedures, and other relevant details. The auditor can then compare descriptions from individuals with details in process and procedure documents. This will help the auditor understand the degree to which processes and procedures are performed consistently and in harmony with documentation.

Once the IS auditor has obtained business documents and interviewed personnel, she can begin to identify and understand any risk areas that may exist in the process.

Identifying Business Risks

The process of identifying business risks is partly analytical and partly based on the auditor’s experience and judgment. An auditor will usually consider both within the single activity of risk identification.

An auditor will usually perform a threat analysis to identify and catalog risks. A threat analysis is an activity whereby the auditor considers a large body of possible threats and selects those that have some reasonable possibility of occurrence, however small. In a threat analysis, the auditor will consider each threat and document a number of facts about each, including:

•  Probability of occurrence   This may be expressed in qualitative (high, medium, low) or quantitative (percentage or number of times per year) terms. The probability should be as realistic as possible, recognizing the fact that actuarial data on business risk is difficult to obtain and more difficult to interpret. Here, an experienced auditor’s judgment is required to establish a reasonable probability.

•  Impact   This is a short description of the results if the threat is actually realized. This is usually a short description, from a few words to a couple of sentences.

•  Loss   This is usually a quantified and estimated loss should the threat actually occur. This figure might be a loss of revenue per day (or week or month) or the replacement cost for an asset, for example.

•  Possible mitigating controls   This is a list of one or more countermeasures that can reduce the probability or the impact of the threat, or both.

•  Countermeasure cost and effort   The cost and effort to implement each countermeasure should be identified, either with a high-medium-low qualitative figure or a quantitative estimate.

•  Updated probability of occurrence   With each mitigating control, a new probability of occurrence should be cited. A different probability, one for each mitigating control, should be specified.

•  Updated impact   With each mitigating control, a new impact of occurrence should be described. For certain threats and countermeasures, the impact may be the same, but for some threats, it may be different. For example, for a threat of fire, a mitigating control may be an inert gas fire suppression system. The new impact (probably just downtime and cleanup) will be much different from the original impact (probably water damage from a sprinkler system).

The auditor will put all of this information into a chart (or electronic spreadsheet) to permit further analysis and the establishment of conclusions—primarily, which threats are most likely to occur and which ones have the greatest potential impact on the organization.

Since it is not the role of auditors to suggest solutions to risks, auditors sometimes forego the analysis of countermeasures. A more likely outcome is the identification of high-risk controls that warrant additional audit scrutiny.

Risk Mitigation

The actual mitigation of risks addressed in the risk assessment is the implementation of one or more of the countermeasures identified in the risk assessment. In simple terms, mitigation could be as easy as a small adjustment in a process or procedure, or a major project to introduce new controls in the form of system upgrades, new components, or new procedures.

When the IS auditor is conducting a risk analysis prior to an audit, risk mitigation may take the form of additional audit scrutiny on certain activities during the audit. Such subsequent analysis will give the IS auditor additional insight about the effectiveness of high-risk controls: a control that the auditor identified as high risk could end up performing well, while other, lower-risk activities could actually be the cause of control failures.

Additional audit scrutiny could take several forms, including one or more of the following:

•  More time spent in inquiry and observation

•  More personnel interviews

•  Higher sampling rates

•  Additional tests

•  Re-performance of some control activities to confirm accuracy or completeness

•  Corroboration interviews

Countermeasures Assessment

Depending upon the severity of the risk, mitigation could also take the form of additional (or improved) controls, even prior to (or despite the results of) the audit itself. The new or changed control may be major or minor, and the time and effort required to implement it could range from almost trivial to a major project.

The cost and effort required to implement a new control (or whatever the countermeasure is that is designed to reduce the probability or impact of a threat) should be determined before it is implemented. It probably does not make sense to spend $10,000 to protect an asset worth $100—unless, of course, there was considerable organizational reputation also associated with that $100 asset.

Monitoring

After countermeasures are implemented, the IS auditor will need to reassess the controls through additional testing. If the control includes any self-monitoring or measuring, the IS auditor should examine those records to see if there is any visible effect of the countermeasures.

The auditor may need to repeat audit activities to determine the effectiveness of countermeasures. For example, additional samples selected after the countermeasure is implemented can be examined and the rate of exceptions compared to periods prior to the countermeasure’s implementation.

Controls

The policies, procedures, mechanisms, systems, and other measures designed to reduce risk are known as controls. An organization develops controls to ensure that its business objectives will be met, risks will be reduced, and errors will be prevented or corrected.

Controls are used in two primary ways in an organization: they are created to ensure desired events, and they are created to avoid unwanted events.

Control Classification

Several types, classes, and categories of controls are discussed in this section. Figure 3-4 depicts this control classification.

Figure 3-4   Control classification shows types, classes, and categories of controls.

Types of Controls

The three types of controls are physical, technical, and administrative:

•  Physical   These types of controls exist in the tangible, physical world. Examples of physical controls are video surveillance, bollards, and fences.

•  Technical   These controls are implemented in the form of information systems and are usually intangible. Examples of technical controls include encryption, computer access controls, and audit logs. These are also referred to sometimes as logical controls.

•  Administrative   These controls are the policies and procedures that require or forbid certain activities. An example administrative control is a policy that forbids personal use of information systems. These are also referred to sometimes as managerial controls.

Classes of Controls

There are six classes of controls:

•  Preventive   This type of control is used to prevent the occurrence of an unwanted event. Examples of preventive controls are computer login screens (which prevent unauthorized persons from accessing information), keycard systems (which prevent unauthorized persons from entering a building or workspace), and encryption (which prevent persons lacking an encryption key from reading encrypted data).

•  Detective   This type of control is used to record both wanted and unwanted events. A detective control cannot enforce an activity (whether it is desired or undesired), but instead it can only make sure that it is known whether, and how, the event occurred. Examples of detective controls include video surveillance and audit logs.

•  Deterrent   This type of control exists in order to convince someone that they should not perform some unwanted activity. Examples of deterrent controls include guard dogs, warning signs, and visible video surveillance cameras and monitors.

•  Corrective   This type of control is activated (manually or automatically) after some unwanted event has occurred. An example corrective control is the act of improving a process when it was found to be defective.

•  Compensating   This type of control is enacted because some other direct control cannot be used. For example, a video surveillance system can be a compensating control when it is implemented to compensate for the lack of a stronger detective control, such as a keycard access system. A compensating control addresses the risk related to the original control.

•  Recovery   This type of control is used to restore the state of a system or asset to its pre-incident state. An example recovery control is the use of a tool to remove a virus from a computer.

Auditors need to understand one key difference between preventive and deterrent controls. A deterrent control requires knowledge of the control by the potential violator—it only works if they know it exists. A preventive control works regardless of whether or not the violator is aware of it.

Categories of Controls

There are two categories of controls:

•  Automatic   This type of control performs its function with little or no human judgment or decision-making. Examples of automatic controls include a login page on an application that cannot be circumvented, and a security door that automatically locks after someone walks through the doorway.

•  Manual   This type of control requires a human to operate it. A manual control may be subject to a higher rate of errors than an automatic control. An example of a manual control is a monthly review of computer users.

Internal Control Objectives

Internal control objectives are statements of desired states or outcomes from business operations. Example control objectives include

•  Protection of IT assets

•  Accuracy of transactions

•  Confidentiality and privacy

•  Availability of IT systems

•  Controlled changes to IT systems

•  Compliance with corporate policies

Control objectives are the foundation for controls. For each control objective, one or more controls will exist to ensure the realization of the control objective. For example, the “Availability of IT Systems” control objective will be met with several controls, including

•  IT systems will be continuously monitored, and any interruptions in availability will result in alerts sent to appropriate personnel.

•  IT systems will have resource-measuring capabilities.

•  IT management will review capacity reports monthly and adjust resources accordingly.

•  IT systems will have anti-malware controls that are monitored by appropriate staff.

Together, these four (or more) controls contribute to the overall control objective on IT system availability. Similarly, the other control objectives will have one or more controls that will ensure their realization.

IS Control Objectives

IS control objectives resemble ordinary control objectives but are set in the context of information systems. Examples of IS control objectives include

•  Protection of information from unauthorized personnel

•  Protection of information from unauthorized modification

•  Integrity of operating systems

•  Controlled and managed changes to information systems

•  Controlled and managed development of application software

An organization will probably have several additional IS control objectives on other basic topics such as malware, availability, and resource management.

Like ordinary control objectives, IS control objectives will be supported by one or more controls.

General Computing Controls

An IS organization supporting many applications and services will generally have some controls that are specific to each individual application. However, IS will also have a set of controls that apply across all of its applications and services. These are usually called its general computing controls, or GCCs.

An organization’s GCCs are general in nature and are often implemented in different ways on different information systems, based upon their individual capabilities and limitations, as well as applicability. Examples of GCCs include

•  Applications require unique user IDs and strong passwords.

•  Passwords are encrypted while stored and transmitted, and are not displayed.

•  Highly sensitive information, such as bank account numbers, is encrypted when stored and transmitted.

•  All administrative actions are logged, and logs are protected from tampering.

Readers who are familiar with information systems technology will quickly realize that these GCCs will be implemented differently across different types of information systems. Specific capabilities and limitations, for example, will result in somewhat different capabilities for password complexity and data encryption. Unless an organization is using really old information systems, the four GCCs shown here can probably be implemented everywhere in an IS environment. How they are implemented is the subject of the next section.

IS Controls

The GCCs discussed in the previous section are implemented across a variety of information technologies. Each general computing control is mapped to a specific IS control on each system type, where it is implemented. In other words, IS controls describe the implementation details for GCCs.

For example, a GCC for password management can be implemented through several IS controls—one for each type of technology platform in use in the organization: one for Windows servers, one for Linux servers, and one for each application that performs its own access management. Those specific IS controls would describe implementation details that reflect the capabilities and limitations of each respective platform.

Performing an Audit

An audit is a systematic and repeatable process whereby a competent and independent professional evaluates one or more controls, interviews personnel, obtains and analyzes evidence, and develops a written opinion on the effectiveness of the control(s).

An IS audit, then, is an audit of information systems and the processes that support them. An IS auditor interviews personnel, gathers and analyzes evidence, and delivers a written opinion on the effectiveness of controls implemented in information systems.

An auditor cannot just begin an audit. Rather, audits need to be planned events. Formal planning is required that includes

•  Purpose   The IS auditor and the auditee must establish a reason why an audit is to be performed. The purpose for a particular audit could be to determine the level of compliance to a particular law, regulation, standard, or contract. Another reason could be to determine whether specific control deficiencies identified in past audits have been remediated. Still another reason is to determine the level of compliance to a new law or standard that the organization may be subject to in the future.

•  Scope   The auditor and the auditee must also establish the scope of the audit. Often, the audit’s purpose will make the scope evident, but not always. Scope may be multidimensional: it could be a given period, meaning records spanning a start date and end date may comprise the body of evidence, geography (systems in a particular region or locale), technology (systems using a specific operating system, database, application, or other aspect), business process (systems that support specific processes such as accounting, order entry, or customer support), or segment of the organization.

•  Risk analysis   To know which areas require the greatest amount of attention, the IS auditor needs to be familiar with the levels of risk associated with the domain being audited. Two different perspectives of risk may be needed: First, the IS auditor needs to know the relative levels of risk among the different aspects of the domain being audited so that audit resources can be allocated accordingly. For example, if the audit is on an enterprise resource planning (ERP) system and the auditor knows that the accounts receivable function has been problematic in the past, the IS auditor will probably want to devote more resources and time on the accounts receivable function than on others. Second, the IS auditor needs to know about the absolute level of risk across the entire domain being audited. For example, if this is an audit to determine compliance to new legislation, the overall risk could be very high if the consequences of noncompliance are high. Both aspects of risk enable the IS auditor to plan accordingly.

•  Audit procedures   The purpose and scope of the audit may help to define the procedures that will be required to perform the audit. For a compliance audit, for example, there may be specific rules on sample sizes and sampling techniques, or it may require auditors with specific qualifications to perform the audit. A compliance audit may also specify criteria for determining if a particular finding constitutes a deficiency or not. There may also be rules for materiality.

•  Resources   The IS auditor must determine what resources are needed and available for the audit. In an external audit, the auditee (which is a client organization) may have a maximum budget figure available. For an external or internal audit, the IS auditor needs to determine the number of man-hours that will be required in the audit and the various skills required. Other resources that may be needed include specialized tools to gather or analyze information obtained from information systems—for example, an analysis program to process the roles and permissions in a database management system in order to identify high-risk areas. To a great degree, the purpose and scope of the audit will determine which resources are required to complete it.

•  Schedule   The IS auditor needs to develop an audit schedule that will give enough time for interviews, data collection and analysis, and report generation. However, the schedule could also come in the form of a constraint, meaning the audit must be complete by a certain date. If the IS auditor is given a deadline, he will need to see how the audit activities can be made to fit within that period. If the date is too aggressive, the IS auditor will need to discuss the matter with the auditee to make required adjustments in scope, resources, or schedule.

Appendix A is devoted to a pragmatic approach to conducting professional audits.

Audit Objectives

The term audit objectives refers to the specific goals for an audit. Generally, the objective of an audit is to determine if controls exist and are effective in some specific aspect of business operations in an organization. Generally an audit is performed as a part of regulations, compliance, or legal obligations. It may also be performed as the result of a serious incident or event.

Depending on the subject and nature of the audit, the auditor may examine the controls and related evidence herself, or the auditor may instead focus on the business content that is processed by the controls. In other words, if the focus of an audit is an organization’s accounting system, the auditor may focus on financial transactions in the system to see how they affect financial bookkeeping. Or, the auditor could focus on the IS processes that support the operation of the financial accounting system. Formal audit objectives should make such a distinction so that the auditor has a sound understanding of the objectives. This tells the auditor what to look at during the audit. Of course, the type of audit being undertaken helps too; this is covered in the next section.

Types of Audits

The scope, purpose, and objectives of an audit will determine the type of audit that will be performed. IS auditors need to understand each type of audit, including the procedures that are used for each:

•  Operational audit   This type of audit is an examination of IS controls, security controls, or business controls to determine control existence and effectiveness. The focus of an operational audit is usually the operation of one or more controls, and it could concentrate on the IS management of a business process or on the business process itself. The scope of an operational audit is shaped to meet audit objectives.

•  Financial audit   This type of audit is an examination of the organization’s accounting system, including accounting department processes and procedures. The typical objective of a financial audit is to determine if business controls are sufficient to ensure the integrity of financial statements.

•  Integrated audit   This type of audit combines an operational audit and a financial audit in order for the auditor to gain a complete understanding of the entire environment’s integrity. Such an audit will closely examine accounting department processes, procedures, and records, as well as the IS applications that support the accounting department. Virtually every organization uses a computerized accounting system for management of its financial records; the computerized accounting system and all of the supporting infrastructure (database management system, operating system, networks, workstations, and so on) will be examined to see if the IS department has the entire environment under adequate control.

•  IS audit   This type of audit is a detailed examination of most or all of an IS department’s operations. An IS audit looks at IT governance to determine if IS is aligned with overall organization goals and objectives. The audit also looks closely at all of the major IS processes, including service delivery, change and configuration management, security management, systems development life cycle, business relationship and supplier management, and incident and problem management. This audit will determine if each control objective and control is effective and operating properly.

•  Administrative audit   This type of audit is an examination of operational efficiency within some segment of the organization.

•  Compliance audit   This type of audit is performed to determine the level and degree of compliance to a law, regulation, standard, or internal control. If a particular law or standard requires an external audit, the compliance audit may have to be performed by approved or licensed external auditors; for example, a U.S. public company financial audit must be performed by a public accounting firm, and a PCI audit must be performed by a licensed QSA (qualified security assessor). If, however, the law or standard does not explicitly require audits, the organization may still wish to perform one-time or regular audits to determine the level of compliance to the law or standard. This type of audit may be performed by internal or external auditors, and typically is performed so that management has a better understanding of the level of compliance risk.

•  Forensic audit   This type of audit is usually performed by an IS auditor or a forensic specialist in support of an anticipated or active legal proceeding. In order to withstand cross-examination and to avoid having evidence being ruled inadmissible, strict procedures must be followed in a forensic audit, including the preservation of evidence and a chain of custody of evidence.

•  Service provider audit   Because many organizations outsource critical activities to third parties, often these third-party service organizations will undergo one or more external audits in order to increase customer confidence in the integrity of the third-party organization’s services. In the United States, a Statement on Standards for Attestation Engagements No. 16, Reporting on Controls at a Service Organization (SSAE 16) can be performed on a service provider’s operations and the audit report transmitted to customers of the service provider. SSAE 16 superseded the Statement of Accounting Standards No. 70 (SAS 70) audit in 2011. The SSAE 16 standard was developed by the American Institute of Certified Public Accountants (AICPA) for the purpose of auditing third-party service organizations that perform financial services on behalf of their customers.

•  Pre-audit   While not technically an audit, a pre-audit is an examination of business processes, IS systems, and business records in anticipation of an upcoming external audit. Usually, an organization will undergo a pre-audit in order to get a better idea of its compliance to a law, regulation, or standard prior to an actual compliance audit. An organization can use the results of a pre-audit to implement corrective measures, thereby improving the outcome of the real audit.

Compliance vs. Substantive Testing

It is important for IS auditors to understand the distinction between compliance testing and substantive testing. These two types of testing are defined here:

•  Compliance testing   This type of testing is used to determine if control procedures have been properly designed and implemented, and are operating properly. For example, an IS auditor might examine business processes, such as the systems development life cycle, change management, or configuration management, to determine if information systems environments are properly managed.

•  Substantive testing   This type of testing is used to determine the accuracy and integrity of transactions that flow through processes and information systems. For instance, an IS auditor may create test transactions and trace them through the environment, examining them at each stage until their completion.

IS audits sometimes involve both compliance testing and substantive testing. The audit objectives that are established will determine if compliance testing, substantive testing, or both will be required.

Audit Methodology

An audit methodology is the set of audit procedures that is used to accomplish a set of audit objectives. An organization that regularly performs audits should develop formal methodologies so that those audits are performed consistently, even when carried out by different personnel.

The phases of a typical audit methodology are described in the remainder of this section.

Audit Subject

Determine the business process, information system, or other domain to be audited. For instance, an IS auditor might be auditing an IT change control process, an IT service desk ticketing system, or the activities performed by a software development department.

Audit Objective

Identify the purpose of the audit. For example, this may be an audit that is required by a law, regulation, standard, or business contract. Or this may be an audit to determine compliance with internal control objectives to measure control effectiveness.

Type of Audit

Identify the type of audit that is to be performed. This may be an operational audit, financial audit, integrated audit, administrative audit, compliance audit, forensic audit, or a security provider audit.

Audit Scope

The business process, department, or application that is the subject of the audit needs to be identified. Usually, a span of time needs to be identified as well so that activities or transactions during that period can be examined.

Pre-Audit Planning

Here, the auditor needs to obtain information about the audit that will enable her to establish the audit plan. Information needed includes

•  Location or locations that need to be visited

•  A list of the applications to be examined

•  The technologies supporting each application

•  Policies, standards, and diagrams that describe the environment

This and other information will enable the IS auditor to determine the skills required to examine and evaluate processes and information systems. The IS auditor will be able to establish an audit schedule and will have a good idea of the types of evidence that are needed. The IS audit may be able to make advance requests for certain other types of evidence even before the on-site phase of the audit begins.

For an audit with a risk-based approach, the auditor has a couple of options:

•  Precede the audit itself with a risk assessment, to determine which processes or controls warrant additional audit scrutiny.

•  Gather information about the organization and historic events to discover risks that warrant additional audit scrutiny.

Audit Statement of Work

For an external audit, the IS auditor may need to develop a statement of work or engagement letter that describes the audit purpose, scope, duration, and costs. The auditor may require a written approval from the client before audit work can officially begin.

Establish Audit Procedures

Using information obtained regarding audit objectives and scope, the IS auditor can now develop procedures for this audit. For each objective and control to be tested, the IS auditor can specify

•  A list of people to interview

•  Inquiries to make during each interview

•  Documentation (policies, procedures, and other documents) to request during each interview

•  Audit tools to use

•  Sampling rates and methodologies

•  How and where evidence will be archived

•  How evidence will be evaluated

Communication Plan

The IS auditor will develop a communication plan in order to keep the IS auditor’s management, as well as the auditee’s management, informed throughout the audit project. The communication plan may contain one or more of the following:

•  A list of evidence requested, usually in the form of a PBC (provided by client) list, which is typically a worksheet that lists specific documents or records and the names of personnel who can provide them (or who provided them in a prior audit).

•  Regular written status reports that include activities performed since the last status report, upcoming activities, and any significant findings that may require immediate attention.

•  Regular status meetings where audit progress, issues, and other matters may be discussed in person or via conference call.

•  Contact information for both IS auditor and auditee so that both parties can contact each other quickly if needed.

Report Preparation

The IS auditor needs to develop a plan that describes how the audit report will be prepared. This will include the format and the content of the report, as well as the manner in which findings will be established and documented.

The IS auditor will need to ensure that the audit report complies with all applicable audit standards, including ISACA IS audit standards.

If the audit report requires internal review, the IS auditor will need to identify the parties that will perform the review, and make sure that they will be available at the time when the IS auditor expects to complete the final draft of the audit report.

Wrap-up

The IS auditor needs to perform a number of tasks at the conclusion of the audit, including the following:

•  Deliver the report to the auditee.

•  Schedule a closing meeting so that the results of the audit can be discussed with the auditee and so that the IS auditor can collect feedback.

•  For external audits, send an invoice to the auditee.

•  Collect and archive all work papers. Enter their existence in a document management system so that they can be retrieved later if needed, and to ensure their destruction when they have reached the end of their retention life.

•  Update PBC documents if the IS auditor anticipates that the audit will be performed again in the future.

•  Collect feedback from the auditee and convey to any audit staff as needed.

Post-Audit Follow-up

After a given period (which could range from days to months), the IS auditor should contact the auditee to determine what progress the auditee has made on the remediation of any audit findings. There are several good reasons for doing this:

•  It establishes a tone of concern for the auditee organization (and an interest in its success) and demonstrates that the auditee is taking the audit process seriously.

•  It helps to establish a dialogue whereby the auditor can help auditee management work through any needed process or technology changes as a result of the audit.

•  It helps the auditor better understand management’s commitment to the audit process and to continuous improvement.

•  For an external auditor, it improves goodwill and the prospect for repeat business.

Audit Evidence

Evidence is the information collected by the auditor during the course of the audit project. The contents and reliability of the evidence obtained are used by the IS auditor to reach conclusions on the effectiveness of controls and control objectives. The IS auditor needs to understand how to evaluate various types of evidence and how (and if) it can be used to support audit findings.

The auditor will collect many kinds of evidence during an audit, including:

•  Observations

•  Written notes

•  Correspondence

•  Independent confirmations from other auditors

•  Internal process and procedure documentation

•  Business records

When the IS auditor examines evidence, he needs to consider several characteristics about the evidence, which will contribute to its weight and reliability. These characteristics include the following:

•  Independence of the evidence provider   The IS auditor needs to determine the independence of the party providing evidence. She will place more weight on evidence provided by an independent party than upon evidence provided by the party being audited. For instance, phone and banking records obtained directly from those respective organizations will be given more credence than an organization’s own records (unless original statements are also provided).

•  Qualifications of the evidence provider   The IS auditor needs to take into account the qualifications of the person providing evidence. This is particularly true when evidence is in the form of highly technical information, such as source code, system configuration settings, or database extracts. The quality of the evidence will rest partly upon the evidence provider’s ability to explain the source of the evidence, how it was produced, and how it is used. Similarly, the qualifications of the auditor come into play, as he will similarly need to be able to thoroughly understand the nature of the evidence and be familiar enough with the technology to be able to determine its veracity.

•  Objectivity   Objective evidence may be considerably more reliable than subjective evidence. An audit log, for instance, is quite objective, whereas an auditee’s opinion of the audit log is less objective.

•  Timing   The IS auditor needs to understand the availability of evidence in the systems being audited. Certain log files, extract files, debug files, and temporary files that may be of value during the examination of the system may be available only for short periods before they are recycled or removed. Often, intermediate files are not backed up or retained for long periods. When an IS auditor is tracing transactions through a system during substantive testing, she will need to understand early on what files or intermediate data should be retrieved so that she can later analyze the data after those intermediate files have been cycled out.

Gathering Evidence

The IS auditor must understand and be experienced in the methods and techniques used to gather evidence during an audit. The methods and techniques used most often in audits include

•  Organization chart review   The IS auditor should request a current org chart, as well as the job descriptions of key personnel. This will help the auditor to understand management, control, and reporting structures within the organization.

•  Review of department and project charters   These documents describe the roles and responsibilities of the IS organization overall, as well as specific departments within IS. The charters for any recent major projects should be requested as well in order to understand newer objectives that could represent adjustments in organizational behavior. If the audit is going to focus on applications used by other departments, the auditor should request those departments’ charters and descriptions, which will help him better understand those departments’ functions, roles, and responsibilities.

•  Review of third-party contracts and service level agreements (SLAs)   Even if they are not a focus of the audit, certain third-party contracts and SLAs may provide additional insight into the workings and culture of the IS organization.

•  Review of IS policies and procedures   The auditor should obtain and review IS policies as well as process and procedure documents that are related to the audit. This will help the auditor to better understand the tone and direction set by management, as well as give the auditor a better idea of how well organized the IS organization really is.

•  Review of risk register (also known as a risk ledger)   The auditor should obtain the organization’s risk register. This will give the auditor insight into the kinds of risks identified by the organization.

•  Review of incident log   The auditor should obtain the organization’s security incident log. This will help the auditor understand the types of security incidents that have occurred, including those involving in-scope processes and systems.

•  Review of IS standards   The auditor should obtain any IS standards documents to understand current policy on the vendors, products, methods, languages, and protocols in use. She should review process and documentation standards as well to see how consistently the organization follows them; this will give the auditor valuable insight into the discipline in the organization.

•  Review of IS system documentation   If the subject of the audit (directly or indirectly) is an IS application, the auditor should obtain much of the project documentation that chronicles the development or acquisition of the system. This may include the following:

•  Feasibility study

•  Functional, technical, and security requirements

•  Requests for proposals (RFPs)/requests for information (RFIs)

•  Responses from vendors (at least the one chosen)

•  Evaluation of vendor responses

•  System design documentation, including data flow diagrams, entity-relationship diagrams, database schema, and so on

•  Test plans and results

•  Implementation guides and results

•  User manuals

•  Operations manuals

•  Business continuity plans

•  Changes made since initial release

•  Reports of system stability, capacity, and availability

•  Personnel interviews   The IS auditor should conduct walkthrough interviews with key personnel who can describe the function, design, use, and operation of the system. Rather than assume that all acquired documentation is absolutely complete and accurate, the auditor should ask open-ended questions to gain additional insight into how well the system really operates. He should develop questions in advance in order to keep the interview on track and to make sure that all topics are covered. The auditor should carefully select key questions and ask them of more than one individual to compare answers, which will give him more insight.

•  Passive observation   When an IS auditor is embedded in an organization, people will “let their guard down” after they are accustomed to his presence. The auditor may be able to observe people being themselves and possibly will hear or see clues that will give better insight into the culture and tone of the organization.

Observing Personnel

It is rarely sufficient for an auditor to obtain and understand process documentation and be able to make judgments about the effectiveness of the process. Usually, the auditor will need to collect evidence in the form of observations to see how consistently a system’s process documentation is actually followed. Some of the techniques in observing personnel include

•  Real tasks   The auditor should request to see some IS functions actually being carried out. For example, if an auditor is examining user access management processes, he should request to observe the persons who manage user accounts to see how they perform their tasks. The auditor should compare the steps taken against procedure documentation; he can also observe the configuration settings that the interviewee has made to see if they are being done according to procedure documents.

•  Skills and experience   The auditor should ask each interviewee about his or her career background to determine the interviewee’s level of experience and career maturity. This will help the auditor to understand whether key responsibilities are in the hands of personnel who can really handle them.

•  Security awareness   The IS auditor should observe personnel to see if they are following security policies and procedures. She can casually ask the interviewee what they know about security procedures to see if the security awareness program is effective. This should implicitly be a part of every audit, even if not explicitly included in scope. Major deviations from policy or common sense could well constitute deficiencies.

•  Segregation of duties   The IS auditor should observe personnel to see if adequate segregation of duties (SOD) is in place. Lapses could include a user account administrator creating or changing a user account without official approval, or a systems engineer making a quick change on a system without going through the change management process.

An experienced IS auditor will have a well-developed “sixth sense,” an intuition about people that can be used to better understand the people who execute procedures.

Sampling

Sampling refers to the technique that is used when it is not feasible to test an entire population of transactions. The objective of sampling is to select a portion of a population so that the characteristics observed will reflect the characteristics of the entire population.

There are several methods for sampling:

•  Statistical sampling   The IS auditor uses a technique of random selection that will statistically reflect the entire population. The auditor will need to determine the size of the sample (usually expressed as a percentage of the entire population) so that the results obtained through testing will statistically reflect the entire population, where each event in the population has an equal chance of being selected.

•  Judgmental sampling   (also known as nonstatistical sampling) The IS auditor judgmentally and subjectively selects samples based on established criteria such as risk or materiality. For instance, when reviewing a list of user accounts to examine, the auditor can purposely select those users whose accounts represent higher risk than the others in the population.

•  Attribute sampling   This technique is used to study the characteristics of a given population to answer the question “how many?”. After the auditor has selected a statistical sample, she then examines the samples. A specific attribute is chosen, and the samples examined to see how many items have the characteristic and how many do not. For example, an auditor may test a list of terminated user accounts to see how many were terminated within 24 hours and how many were not. This is used to statistically determine the rate at which terminations are performed within 24 hours among the entire population.

•  Variable sampling   This technique is used to statistically determine the characteristic of a given population to answer the question “how much?”. For example, an auditor who wishes to know the total value of an inventory can select a sample and then statistically determine the total value in the entire population based on the total value of the sample.

•  Stop-or-go sampling   This technique is used to permit sampling to stop at the earliest possible time. The IS auditor will use this technique when he feels that there are a low risk and rate of exceptions in the overall population.

•  Discovery sampling   This technique is used when an IS auditor is trying to find at least one exception in a population. When he is examining a population where even a single exception would represent a high-risk situation (such as embezzlement or fraud), the auditor will recommend a more intensive investigation to determine if additional exceptions exist.

•  Stratified sampling   Here, the event population will be divided into classes, or strata, based upon the value of one of the attributes. Then, samples are selected from each class, and results are developed from each class or combined into a single result. An example of where this could be used is a selection of purchase orders (POs), where the IS auditor wants to make sure that some of the extremely high-value and low-value POs will be selected to see if there is any statistical difference in the results in different classes.

When performing sampling, the IS auditor needs to understand several terms related to aspects of statistical sampling techniques, including:

•  Confidence coefficient   Sometimes known as the reliability factor or confidence level, this is expressed as a percentage, as the probability that the sample selected actually represents the entire population. A confidence coefficient of 95 percent is considered high.

•  Sampling risk   This is equal to one minus the confidence coefficient percentage. For example, if a given sample has a 93 percent confidence coefficient, the risk level is 7 percent (100 percent minus 93 percent equals 7 percent).

•  Precision   This represents how closely the sample represents the entire population. A low precision figure means high accuracy, and a high precision figure means low accuracy. A smaller sample makes the precision higher, and the risk of exceptions in the entire population is higher.

•  Expected error rate   This is an estimate that expresses the percentage of errors that may exist in the entire population. When the expected error rate is higher, the sample needs to be higher (because a population with a high rate of errors requires greater scrutiny). If the expected error rate is low, the sample can be smaller.

•  Sample mean   This is the sum of all samples divided by the number of samples. This equals the average value of the sample.

•  Sample standard deviation   This is a computation of the variance of sample values from the sample mean. This is a measurement of the “spread” of values in the sample.

•  Population standard deviation   Within the entire evidence population, this is a computation of the variance of values from the mean. All other factors being equal, a larger population standard deviation means the auditor should select a larger set of samples.

•  Tolerable error rate   This is the highest number of errors that can exist without a result being materially misstated.

Reliance Upon the Work of Other Auditors

Audit departments and external auditors, like other IT service organizations, are challenged to find qualified audit professionals who understand all aspects of organizations’ technologies in use. Increased specialization in IT is resulting in auditors with increased technical knowledge in certain areas, and fewer auditors with all of the knowledge required to perform an audit. This and other factors are putting increasing pressure on organizations to outsource some auditing tasks (or entire audits) to third-party organizations.

When considering reliance upon other auditors, many potential issues must be examined, including:

•  Laws, regulations, standards, or contracts that may place restrictions on the use of third-party auditors

•  Impact on risk

•  Costs and the overhead required to manage external auditors

•  Impact on audit schedule and reporting

•  Impact on general and professional liability

•  Audit standards and methodologies used by the third-party auditor

•  Competence and experience of the third-party auditor

•  Independence and objectivity of internal versus external auditors

•  Methods for communication of audit issues and results

•  Access by third-party auditor to internal records and systems

•  Protection and privacy of information made available to external auditors

•  Background checks, nondisclosure, and other agreements for third-party audit personnel

•  Audit management controls used to manage external audit activities

•  Compliance and/or compatibility with audit standards, regulations, and stipulations

Computer-Assisted Audit and Automated Work Papers

When auditing complex information systems, IS auditors often need to obtain sample data from systems with a variety of operating systems, database management systems, record layouts, and processing methods. Auditors are turning to computer-assisted audit techniques (CAATs) to help them examine and evaluate data across these complex environments.

Protecting Automated Work Papers

CAATs help IS auditors by making sampling easier and by capturing data that has varying degrees of persistence in an organization’s application environment. That captured data will often be considered part of the body of work papers. Controls need to be enacted to protect automated work papers, including:

•  Access control   Only authorized persons (ideally only the auditor) should have read access to automated work papers.

•  Protection from tampering and damage   Automated work papers must be protected so that no person is able to change them, remove them, or threaten their integrity.

•  Backup   Automated work papers should be included in data backups.

•  Encryption   Automated work papers should be encrypted if they contain sensitive information. Effective key management procedures and controls must be in place.

When using CAATs, auditors need to document the evidence they obtain from systems and be able to link it to business transactions. Often, auditors will have to obtain several other items, including:

•  Application source code

•  Online reports that correlate captured data to transactions and results

•  Database schemas

•  Data flow diagrams and flow charts

•  Sample reports

•  Operations procedures

Auditors will need to be able to stitch all these pieces back together to show a complete picture of the details behind business transactions.

Generalized Audit Software

IS auditors can use generalized audit software (GAS) to directly read and access data from database platforms and flat files. They can independently and directly acquire sample data from databases, which they can then analyze on a separate system. Generalized audit software has the ability to select samples, select data, and perform analysis on data. This can help the auditor better understand key data sets in a system, enabling him to determine the integrity and accuracy of a system and the business transactions it supports.

Continuous Auditing

CAATs can also be used as part of a continuous audit approach, where samples are obtained automatically over long periods instead of just during audit engagements. This represents a paradigm shift from the traditional model of periodic sampling and reporting. To be truly effective, continuous auditing should include the following:

•  Frequent notifications to auditors and control owners on audit results

•  Triggers to notify auditors and control owners of control failures and other exceptions

Additional guidance on the use of CAATs and continuous auditing is found in ISACA auditing guideline 2205, Evidence.

Reporting Audit Results

The work product of an audit project is the audit report, a written report that describes the entire audit project, including audit objectives, scope, controls evaluated, opinions on the effectiveness and integrity of those controls, and recommendations for improvement.

While an IS auditor or audit firm will generally use a standard format for an audit report, some laws and standards require that an audit report regarding those laws or standards contain specific information or be presented in a particular format. Still, there will be some variance in the structure and appearance of audit reports created by different audit organizations.

The auditor is typically asked to present findings in a closing meeting, where he can explain the audit and its results and be available to answer questions about the audit. The auditor may include an electronic presentation to guide discussion of the audit.

Structure and Contents

While there are often different styles for presenting audit findings, as well as regulations and standards that require specific content, an audit report will generally include several elements:

•  Cover letter   Audit reports frequently include a cover letter that describes the audit, its scope and purpose, and findings. Often, this letter is used as evidence to other organizations that the audit took place.

•  Introduction   The report should contain an introduction that describes the contents of the audit report.

•  Summary   The report should include an executive summary that briefly describes the audit, its purpose and scope, and findings and recommendations.

•  Description of the audit   The report should include a high-level description of the audit, its purpose, and its objectives.

•  Listing of systems and processes examined   The report should contain a list of systems, applications, and business processes that were examined.

•  Listing of interviewees   The report should contain a complete list of interviewees, when they were interviewed, and topics discussed.

•  Listing of evidence obtained   The report should contain a detailed list of all evidence obtained, from whom, and when it was obtained. Electronic evidence should be described, including the time it was acquired, the system it was obtained from, and the method used to obtain it. The names of any staff members who assisted should be included.

•  Explanation of sampling techniques   Each time the auditor performed any sampling, the techniques used should be described.

•  Description of findings and recommendations   Here, detailed explanations describe the effectiveness of each control, based on evidence and the auditor’s judgment. Exceptions are described in detail to demonstrate that they actually occurred. Information in this section may be organized according to criticality, technology in use, or business function, or more than one of these.

When the IS auditor is creating the report, he must make sure that it is balanced, reasonable, and fair. The report should not just be a list of everything that was bad; it should also include a list of controls that were found to be operating effectively.

The IS auditor also needs to take care when describing recommendations, realizing that any organization is capable of only so much change in a given period. If the audit report contains many findings, the auditor needs to realize that the organization may not be able to remediate all of them in an elegant manner. Instead, the organization will need to understand which findings should be remediated first—the audit report should provide this guidance.

Evaluating Control Effectiveness

When developing an audit report, the auditor needs to communicate the effectiveness of controls to the auditee. Often, this reporting is needed at several layers; for instance, the auditor may provide more detailed findings and recommendations to control owners, while the report for senior management may contain only the significant findings.

One method that auditors frequently use is the development of a matrix of all audit findings, where each audit finding is scored on a criticality scale. This helps the auditor to illustrate the audit findings that are the most important and those that are less important, in the auditor’s opinion. The auditor can also report on cases where an ineffective control is mitigated (fully or partially) by one or more compensating controls. For example, a system may not have the ability to enforce password complexity (e.g., requiring upper- and lowercase letters, plus numbers and special characters), but this can be compensated through the use of longer-than-usual passwords and perhaps even more frequent expiration.

Other Audit Topics

This section includes important discussions related to IS audits.

Detecting Fraud

Fraud is defined as the intentional deception made for personal gain or damage to another party. In the context of corporate information systems and IS auditing, fraud is an act whereby a person discovers and exploits a weakness in a process or system for personal gain.

Management is responsible for implementing controls designed to prevent, deter, and detect fraud. However, no system or process is without weaknesses—worse yet, if two or more employees agree to a conspiracy to defraud the organization, it is possible for the conspirators to, at least temporarily, steal from the organization.

While detecting fraud is certainly not the IS auditor’s primary responsibility, an IS auditor surely has many opportunities to discover exploitable weaknesses in processes and systems that could be used to defraud the organization. Occasionally, the IS auditor will discover evidence of fraud while examining transaction samples during substantive testing.

When the auditor detects signs of fraud, he should carefully evaluate what he has found and then communicate his findings to the appropriate authorities. Precisely whom he contacts will depend on the nature and structure of the organization and whether there is regulatory oversight of the organization and/or the auditor. The auditor needs to be extremely careful when reporting his findings within the organization because the person he reports his findings to could be the perpetrator—or associated with the perpetrator. This logic may prompt the auditor to report this finding directly to the audit committee, thereby bypassing all potential perpetrators in the organization (usually, members of the audit committee are not employees in the organization, have no role in the organization’s operations, and hence are probably not among the culprits).

If the organization has no audit committee or similar overseeing body, the auditor may be compelled to report his findings to industry regulators or law enforcement.

When the auditor is representing an external audit organization, often the auditor will discuss the weakness or actual fraud with selected members of the audit organization in order to confirm his observations and agree on a plan for notifying the auditee or outside authorities.

Communication of the weakness or actual fraud typically begins with a phone call or face-to-face conversation, which will be followed up by a formal written letter. This will satisfy the need to inform the auditee quickly (via the phone call) and formally (via the written letter).

Audit Risk and Materiality

What if there are material errors in business processes that remain undetected by the IS auditor? There are a number of ways in which this can occur, including:

•  Control risk   This is the risk that a material error exists that will not be prevented or detected by the organization’s control framework. For example, a manual control that is designed to detect unauthorized changes in an information system may fail if the personnel who review logs overlook significant errors or fraud.

•  Detection risk   This is the risk that an IS auditor will overlook errors or exceptions during an audit. Detection risk should be a part of the IS auditor’s risk assessment that is carried out at the beginning of an audit; this would help the auditor focus on those controls that require additional scrutiny (meaning higher sampling rates) and thereby improve the chances of detecting errors.

•  Inherent risk   This is the risk that there are material weaknesses in existing business processes and there are no compensating controls to aid in their detection or prevention. Inherent risks exist independent of the audit.

•  Overall audit risk   This is the summation of all of the residual risks discussed in this section.

•  Sampling risk   This is the risk that the sampling technique used will not detect transactions that are not in compliance with controls.

Materiality   In financial audits, materiality is established as a dollar amount threshold that is calculated in one of several possible ways, including a percentage of pretax income, a percentage of gross profit, a percentage of total assets, a percentage of total revenue, a percentage of equity, or blended methods using two or more of these.

Then, when an auditor is examining transactions and controls during an audit, a finding can be classified as a material weakness if the dollar amount of the exceptions exceeds the materiality threshold. There is, however, some latitude (more in some cases and less in others) in the auditor’s judgment as to whether a finding is material.

In an IS audit, the controls being examined usually do not have dollar figures associated with them and deficiencies are not measured against materiality thresholds in the same way. Instead, materiality in an IS audit occurs when a control deficiency (or combination of related control deficiencies) makes it possible for serious errors, omissions, irregularities, or illegal acts to occur as a result of the deficiency or deficiencies). Here more than in a financial audit, the judgment of the IS auditor is very important in determining if a finding is material.

Auditing and Risk Assessment

When assessing the effectiveness of controls in an organization, the IS auditor should take the time to understand how the organization approaches risk assessment and risk treatment.

Risk Assessment   Organizations should periodically undertake risk assessments in order to identify areas of risk that warrant remediation. A risk assessment should identify, prioritize, and rank risks. The subject of risk assessment should be those business processes and supporting information systems and infrastructures that are central to the organization’s mission.

After identifying risks, the risk assessment should include one or more potential remedies, each with an analysis of the cost and effort to implement and the estimated reduction in risk. When these remedies and their impact (in terms of risk reduction) are then ranked, the result should be a list of the most effective initiatives for reducing risk in the organization.

There are two types of risk assessment: qualitative and quantitative. A qualitative risk assessment rates risks as high-medium-low, whereas a quantitative risk analysis rates risks in terms of actual probabilities and costs. A quantitative risk assessment is considerably more difficult and time-consuming to perform, since it can be difficult to ascertain reasonable probabilities of threats and their financial impact. However, when seriously considering measures to reduce risk on the highest-risk areas in the assessment, sometimes it makes sense to perform some quantitative risk assessment in order to verify which investments are the ones that will make the most difference.

Risk Treatment   Once risks have been identified, risk treatment is the term that describes the action taken to address them. There are four possible avenues for risk treatment:

•  Risk reduction   This involves making changes to processes, procedures, systems, or controls that will reduce either the probability of a threat or its impact. For example, if the risk assessment identifies a threat of a SQL injection attack on an application, the organization can reduce risk by implementing an application firewall that will block such attacks.

•  Risk transfer   This typically involves the use of insurance, which is used to compensate the organization for the financial losses or damages that would occur if the threat were realized. For example, the organization can transfer the risk of a denial of service attack by purchasing a cyber-insurance policy that would compensate the organization if such an attack were to occur.

•  Risk avoidance   Here, the organization will cease the activity associated with the risk. For instance, if the risk assessment identifies risks associated with the implementation of an e-commerce capability, the organization might choose to abandon this idea, thereby avoiding e-commerce–related risk.

•  Risk acceptance   In this case, the organization feels that the risk is acceptable and that no measures need to be taken to reduce the risk further.

Rarely does an organization make a decision that fits entirely within a single risk treatment category. Rather, risk treatment is usually a blended approach, where, for instance, measures are taken to reduce risk; however, even a combination of measures rarely eliminates all risk—there is usually some risk left over after some risk treatment is performed. This leftover risk is known as residual risk. And like the remaining dirt that can’t be picked up with a broom and dustpan, the leftover risk is usually accepted.

Control Self-Assessment

Control self-assessment (CSA) is a methodology used by an organization to review key business objectives, risks related to achieving these objectives, and the key controls designed to manage those risks. The primary characteristic of a CSA is that the organization takes initiative to self-regulate rather than engage outsiders, who may be experts in auditing but not in the organization’s mission, goals, and culture.

CSA Advantages and Disadvantages

Like almost any business activity, control self-assessments have a number of advantages and disadvantages that the IS auditor and others should be familiar with. This will help the organization make the most of this process and avoid some common problems.

The advantages of a control self-assessment include

•  Risks can be detected earlier, since subject matter experts are involved earlier.

•  Internal controls can be improved in a timely manner.

•  It leads to greater ownership of controls through involvement in their improvement.

•  It leads to improved employee awareness of controls through involvement in their improvement.

•  It may help improve relationships between departments and auditors.

Some of the disadvantages of a control self-assessment include the following:

•  It could be mistaken by employees or management as a substitute for an internal audit.

•  It may be considered extra work and dismissed as unnecessary.

•  Employees may attempt to cover up shoddy work and misdeeds.

•  It may be considered an attempt by the auditor to shrug off responsibilities.

•  Lack of employee involvement would translate to little or no process improvement.

The Control Self-Assessment Life Cycle

Like most continuous-improvement processes, the control self-assessment process is an iterative life cycle. The phases in the control self-assessment are

•  Identify and assess risks   Operational risks are identified and analyzed.

•  Identify and assess controls   Controls to manage risks are identified and assessed. If any controls are missing, new controls are designed.

•  Develop questionnaire or conduct workshop   An interactive session is conducted, if possible, for discussion of risks and controls. If personnel are distributed across several locations, a conference call can be convened, or a questionnaire may be developed and sent to them.

•  Analyze completed questionnaires or assess workshop   If a workshop was held, the workshop results are assessed to see what good ideas for remediation emerged. If a questionnaire was distributed, the results are analyzed to see what good ideas for risk remediation were identified.

•  Control remediation   Using the best ideas from the workshop or questionnaire, controls are designed or altered to better manage risk.

•  Awareness training   This activity is carried out through every phase of the life cycle to keep personnel informed about the activities in the various phases.

The control self-assessment life cycle is illustrated in Figure 3-5.

Figure 3-5   The control self-assessment life cycle

Self-Assessment Objectives

The primary objective of a control self-assessment is to transfer some of the responsibility for oversight of control performance and monitoring to the control owners. The IS auditor’s role is not diminished, as the IS audit still needs to periodically test control effectiveness, but control owners will play a more active role in the audit of their controls.

Another objective of control self-assessment is the long-term reduction in exceptions. As control owners assume more responsibility for the performance of their controls, they will strive to avoid situations where IS auditors identify exceptions. The CSA gives control owners an opportunity and a process for cleaning house and improving audit results.

Auditors and Self-Assessment

IS auditors should be involved in control self-assessments that various departments conduct. The role of an IS auditor should be that of an objective subject matter expert who can guide discussions in the right direction so that controls will receive the right kind of development over time.

IS auditors should resist too large a role in control self-assessments. Responsibility for control development and maturation should lie within the department that owns the CSA. However, if a department is new at conducting a CSA, it may take some time before they are confident and competent enough to take full ownership and responsibility for the process.

Implementation of Audit Recommendations

The purpose of internal and external audits is to identify potential opportunities for making improvements in control objectives and control activities. The handoff point between the completion of the audit and the auditee’s assumption of control is in the portion of the audit report that contains findings and recommendations. These recommendations are the imperatives that the auditor recommends the auditee perform to improve the control environment.

Implementation of audit recommendations is the responsibility of the auditee. However, there is some sense of shared responsibility with the auditor, as the auditor seeks to understand the auditee’s business so that she can develop recommendations that can reasonably be undertaken and completed. In a productive auditor-auditee relationship, the auditor will develop recommendations using the fullest possible understanding of the auditee’s business environment, capabilities, and limitations—in essence saying, “Here are my recommendations to you for reducing risk and improving controls.” And the auditee, having worked with the auditor to understand his methodology and conclusions, and who has been understood by the auditor, will accept the recommendations and take full responsibility for them—in essence saying, “I accept your recommendations and will implement them.” This is the spirit and intent of the auditor-auditee partnership.

Summary

The audit function in an organization should be defined and described in a charter. The audit program and audit strategy should support the organization’s mission and objectives and facilitate business development and growth.

Auditors need to establish and maintain technical competence so that they can effectively evaluate technical controls and identify technical control risks. They will need to attend periodic training in the technologies in use by the organization, as well as in emerging technologies that the organization may use in the future.

The ISACA Code of Professional Ethics defines the standards of behavior and conduct for IS auditors. The ISACA auditing standards framework defines mandatory audit standards, guidelines that contain suggestions for implementing the standards. All persons who hold the CISA designation are required to uphold the ISACA Code of Professional Ethics; violations will result in investigations and possible disciplinary actions, including expulsion.

ISACA audit standards, guidelines, and procedures provide a framework that can be used to guide the development of an IT assurance program. The Information Technology Assurance Framework (ITAF) is a complete IT assurance and audit framework that borrows elements from COBIT and ISACA audit standards, guidelines, and procedures.

IS auditors may need to perform a risk analysis as an integral part of an audit project in order to identify risk areas that require additional audit resources. The result of the risk analysis will help the auditor to build a complete audit plan that includes the right level of activities to be carried out during the audit.

The ISACA Risk IT Framework provides a set of processes for performing risk governance, risk evaluation, and risk response.

Internal controls are the policies, procedures, mechanisms, systems, and other means designed to reduce risk and facilitate the achievement of business objectives. Controls are classified in several different ways that describe how they are designed to control behaviors and outcomes.

Internal control objectives are statements of desired states and outcomes in the organization. They are supported by one or more controls that ensure the realization of control objectives. Controls are measurable and can be defined and enforced with processes, procedures, or automatic mechanisms within information systems. IS control objectives resemble internal control objectives, but are focused on the desired states and outcomes within the context of information systems.

General computing controls are controls that are applied across an entire IS environment. An organization will likely have additional controls that are applied to individual applications or components in the environment.

An audit is the planned, methodical evaluation of controls and control objectives. A key activity in an audit is the identification and acquisition of evidence that supports the operation of controls and helps the auditor reach a conclusion about the effectiveness of a control.

IS auditors generally develop and follow an audit methodology, which is a process that ensures consistent, repeatable audits from start to finish.

Evidence is the information collected by the auditor during the course of the audit. The reliability and relevance of evidence helps the auditor reach sound conclusions on the effectiveness of controls and control objectives.

Sampling is the technique used when it is not feasible to test an entire population of transactions. Sampling techniques need to be carefully considered so that they accurately represent the entire population.

Computer-assisted audit techniques (CAATs) are used to automate sampling and analysis of information in complex application environments. CAATs can help to analyze and correlate data in ways that would be too difficult to perform manually. Continuous auditing consists of samples that are automatically gathered over long periods.

The audit report is the work product of the audit project. It contains a summary, a description of evidence gathered, and findings and conclusions.

In IS audits, materiality is the threshold where control deficiencies make it possible for serious errors, omissions, irregularities, or illegal acts to occur.

A control self-assessment is an activity used by an organization to take ownership of controls and make improvements in the implementation of its controls through workshops and other activities.

Notes

•  An audit program is the audit strategy and the plans that include scope, objectives, resources, and procedures used to evaluate controls and processes.

•  IS auditors need to stay current with technology through training courses, webinars, ISACA chapter training events, and industry conferences.

•  Several laws, regulations, and standards require internal or external audits to ensure that organizations achieve and maintain compliance.

•  The types of controls are physical, technical, and administrative.

•  The classes of controls are preventive, detective, deterrent, corrective, compensating, and recovery.

•  The categories of controls are automatic and manual.

•  The types of audits are operational audits, financial audits, integrated audits, IS audits, administrative audits, compliance audits, forensic audits, and service provider audits. Pre-audits can be performed to help an organization prepare for an upcoming audit. Internal audits and external audits refer to the relationship of auditor to auditee.

•  Compliance testing is used to determine if control procedures have proper design and are operating properly. Substantive testing is used to verify the accuracy and integrity of transactions as they flow through a system.

•  Audit methodologies define an audit subject, audit objective, type of audit, audit scope, pre-audit planning, audit statement of work, audit procedures, communication plan, report preparation, wrap-up, and post-audit follow-up.

•  The types of evidence that the auditor will collect during an audit include observations, written notes, correspondence, process and procedure documentation, and business records.

•  During an audit, the auditor should obtain org charts, department charters, third-party contracts, policies and procedures, risk ledgers, incident logs, standards, and system documentation. She should conduct several interviews with pre-written questions and carefully observe personnel to better understand their discipline, as well as organizational culture and maturity.

•  The types of sampling include statistical sampling, judgmental sampling, attribute sampling, variable sampling, stop-or-go sampling, discovery sampling, and stratified sampling. The IS auditor needs to understand the meaning of confidence coefficient, sampling risk, precision, expected error rate, sample mean, sample standard deviation, and tolerable error rate.

•  An audit report usually includes a cover letter, introduction, summary, audit description, list of systems examined, interviewees, evidence, explanation of sampling techniques, findings, and recommendations.

•  The types of risks that are related to audits include control risk, detection risk, inherent risk, overall audit risk, and sampling risk.

•  Computer-assisted audit techniques, generalized audit software, and continuous auditing present many challenges and opportunities that may result in more frequent and rapid reporting, which can help to reduce the time needed to mitigate control failures and exceptions.

•  External auditors may be needed when the organization lacks specific expertise or resources to conduct an internal audit. However, some regulations and standards require external, independent audits.

Questions

1.  An IS auditor is planning an audit project and needs to know which areas are the highest risk. What is the best approach for identifying these risk areas?

A.  Perform the audit; control failures will identify the areas of highest risk.

B.  Perform the audit and then perform a risk assessment.

C.  Perform a risk assessment first and then concentrate control tests in high-risk areas identified in the risk assessment.

D.  Increase sampling rates in high-risk areas.

2.  An auditor has detected potential fraud while testing a control objective. What should the auditor do next?

A.  Notify the audit committee.

B.  Conduct a formal investigation.

C.  Report the fraud to law enforcement.

D.  Report the suspected fraud to management.

3.  The possibility that a process or procedure will be unable to prevent or detect serious errors and wrongdoing is known as:

A.  Detection risk

B.  Inherent risk

C.  Sampling risk

D.  Control risk

4.  The categories of risk treatment are

A.  Risk avoidance, risk transfer, risk mitigation, and risk acceptance

B.  Risk avoidance, risk transfer, and risk mitigation

C.  Risk avoidance, risk reduction, risk transfer, risk mitigation, and risk acceptance

D.  Risk avoidance, risk treatment, risk mitigation, and risk acceptance

5.  An IS auditor needs to perform an audit of a financial system and needs to trace individual transactions through the system. What type of testing should the auditor perform?

A.  Discovery testing

B.  Statistical testing

C.  Compliance testing

D.  Substantive testing

6.  An IS auditor is auditing the change management process for a financial application. The auditor has two primary pieces of evidence: change logs and a written analysis of the change logs performed by a business analyst. Which evidence is best and why?

A.  The change log is best because it is subjective.

B.  The written analysis is best because it interprets the change log.

C.  The change log is best because it is objective and unbiased.

D.  The written analysis is best because it is objective.

7.  Under which circumstances should an auditor use subjective sampling?

A.  When the population size is low

B.  When the auditor feels that specific transactions represent higher risk than most others

C.  When the risk of exceptions is low

D.  When statistical sampling cannot be performed

8.  An IS auditor has discovered a high-risk exception during control testing. What is the best course of action for the IS auditor to take?

A.  Immediately perform mitigation.

B.  Include the exception in the report and mark the test as a control failure.

C.  Immediately inform the auditee of the situation.

D.  Immediately inform the audit committee of the situation.

9.  What is the appropriate role of an IS auditor in a control self-assessment?

A.  The IS auditor should participate as a subject matter expert.

B.  The IS auditor should act as facilitator.

C.  The IS auditor should not be involved.

D.  The IS auditor should design the control self-assessment.

10.  Which of the following would NOT be useful evidence in an IS audit?

A.  Personnel handbook

B.  Organization mission statement and objectives

C.  Organization chart

D.  Organization history

11.  An auditor has discovered that automated work papers were configured with read/write permissions for database administrators. What actions should the auditor take?

A.  Simply continue to rely on the automated work papers.

B.  Note an exception and continue to rely on these automated work papers.

C.  Recommend that permissions on automated work papers be changed so that no personnel have write access so that this data may be relied upon in the future.

D.  Notify the board of directors or the audit committee.

12.  During an audit, an auditor has discovered a process that is being performed consistently and effectively, but the process lacks procedure documentation. What action should the auditor take?

A.  Document the process.

B.  Find that the process is effective but recommend that it be documented.

C.  Write the procedure document for the auditee and include it in audit evidence.

D.  Find that the process is ineffective.

13.  During audit planning, an auditor has discovered that a key business process in the auditee organization has been outsourced to an external service provider. Which option should the auditor consider?

A.  Audit the external service provider or rely on an SSAE 16 audit report if one is available.

B.  Audit the external service provider.

C.  Determine that the business process is not effective.

D.  Request that the external service provider submit its internal audit work papers.

14.  Why should an auditor prefer bank statements over a department’s own business records that list bank transactions?

A.  Bank statements can be provided in electronic format.

B.  Bank statements contain data not found in internal records.

C.  Bank statements are usually easier to obtain.

D.  Bank statements are independent and objective.

15.  Which of the following statements is true about ISACA audit standards and guidelines?

A.  ISACA audit standards are mandatory, while ISACA audit guidelines are optional.

B.  ISACA audit standards are optional, while ISACA audit guidelines are mandatory.

C.  ISACA audit standards and guidelines are mandatory.

D.  ISACA audit standards and guidelines are optional.

Answers

1.  C. The IS auditor should conduct a risk assessment first to determine which areas have highest risk. She should devote more testing resources to those high-risk areas.

2.  A. When the IS auditor suspects fraud, he should conduct a careful evaluation of the matter and notify the audit committee. Because audit committee members are generally not involved in business operations, they will be sufficiently removed from the matter, and they will have the authority to involve others as needed.

3.  D. Control risk is the term that signifies the possibility that a control will fail to prevent or detect unwanted actions.

4.  A. The four categories of risk treatment are risk mitigation (where risks are reduced through a control or process change), risk transfer (where risks are transferred to an external party such as an insurance company), risk avoidance (where the risk-bearing activity is discontinued), and risk acceptance (where management chooses to accept the risk).

5.  D. The auditor should perform substantive testing, which is a test of transaction integrity.

6.  C. The change log is the best evidence because it is objective and not subject to human judgment.

7.  B. Subjective sampling is used when the auditor wants to concentrate on samples known to represent higher risk.

8.  C. The IS auditor should immediately inform the auditee when any high-risk situation is discovered.

9.  B. The IS auditor should act as a facilitator of a control self-assessment, but allow management to make any decisions regarding changes to controls.

10.  D. Of the choices given, the organization history would be the least useful. The others will provide insight into the organization’s mission and goals, and how it sets out to achieve them.

11.  C. If an auditor has discovered that automated work papers could be updated by any personnel, then they should not be trusted to contain complete and accurate information. Once this is fixed by eliminating write access, the auditor can rely on this data going forward, but should not rely on the information prior to that point in time.

12.  B. The auditor should determine that the process is effective and recommend that it be documented. However, some regulations may find that such a process is ineffective based on the principle that it lacks documentation.

13.  A. An auditor who has determined that a key business process has been outsourced needs to determine effectiveness of that process by auditing that process or by relying on a separate audit report of that process.

14.  D. An auditor would prefer bank statements over internal records because bank statements are produced by a bank, which is independent and objective. A bank is unlikely to alter its records to improve the audit outcome of one of its customers.

15.  A. ISACA audit standards are mandatory for all ISACA certification holders, including those with the CISA certification. ISACA audit guidelines are optional.