4 EMBEDDING PENETRATION TESTING WITHIN ORGANISATIONAL SECURITY POLICIES AND PROCEDURES
Ceri Charlton
An important part of the strategy of utilising penetration tests is identifying when they are to be used. This chapter discusses the way in which the activities relating to penetration testing can be built into the Information Security Management System (ISMS) of an organisation and the broader risk management framework. This chapter aims to explore some of the drivers, approaches and obstacles to embedding penetration testing (however it may be conducted) within an organisation.
ADDING PENETRATION TESTING TO AN EXISTING ENTERPRISE INFORMATION SECURITY STRATEGY
Increasingly, regardless of any additional industry-specific or regulatory requirements, penetration testing is legally becoming an expectation: the legal view is moving towards a stance that penetration testing is not just a ‘nice to have’ or something that conscientious, ‘best in class’ organisations undertake. The absence of evidence of penetration testing having been performed is increasingly viewed as negligence.
As an example, the investigation conducted by the ICO in 2017 specifically cited the failure of Boomerang Video Ltd to conduct regular penetration testing on its website, as a factor in its determination that Boomerang took inadequate steps to protect the data that it held.1
In this case, Boomerang was a company which provided rentals of videogames to customers, in exchange for payment, made via an e-Commerce portal. The portal was originally developed almost a decade prior to the breach, but since then, had not been subjected to regular penetration testing. In investigating the breach, the ICO determined that: (i) customers’ payment card information had been accessed by the attacker; and (ii) this had been possible due to vulnerabilities existing within this platform.
One of the particularly inexcusable vulnerabilities cited was SQL Injection, which existed within the login screen. Exploiting this flaw, the attacker was able to cause the web page to return details of both usernames and the hashes of the passwords associated with these accounts. At the time of the breach occurring, SQL Injection specifically had been a disclosed class of vulnerability for over a decade and a half and OWASP (the Open Web Application Security Project) had included input validation and injection vulnerabilities among its ‘Top Ten’ common vulnerabilities for around a decade. The existence of these vulnerabilities is trivially easy to test for and can even be discovered using automated tools. It is unlikely that vulnerabilities as obvious and easy to detect as this would have been overlooked during penetration tests performed by any competent tester.
It is interesting to note that even though the insecure portal was developed by a third party, Boomerang was found to be at fault for failing to detect the vulnerabilities, due to this lack of penetration testing.
Not all penetration tests are created equal and indeed, there is a great deal of variation in terms of how regularly organisations conduct penetration tests. In the Payment Card Industry, Data Security Standard (PCI DSS) and its related standards, we see that a failure to have performed a penetration test for a new system ahead of deployment to production is considered a compliance violation in itself, as is failure to re-test any system, at least once a year after the last test.
By building penetration testing into an organisation’s ISMS, an organisation achieves several things:
• It is clearly presentable to any external parties reading formal security documentation, such as prospective customers, regulators or legislators, that there is an expectation that penetration testing is performed.
• It eliminates the excuse of individuals or projects within the organisation in the position to be able to deploy new software that they had not realised that it would be necessary.
• It formalises the expectation of penetration testing, which in turn allows internal or external auditors, who base the scope of their own assessment on the stated requirements of the organisations’ governance, to validate that such testing has occurred.
• It may assist those procuring software to also secure the budget to have penetration testing performed.
When attempting to build penetration testing into existing security requirements and testing procedures, IT management should consider:
• Which systems require penetration testing? All? Those where the risk warrants it? Only where mandated by regulation?
• How frequently are tests and re-tests required? Is there a requirement to periodically re-test even unchanged legacy systems, in case newly disclosed vulnerabilities have emerged since the last test?
• How significant must a release be, before it requires a re-test? Is it only necessary for major releases, or after every patch? In order to meaningfully answer this question, it is also necessary to formally and explicitly state the means by which a release will be categorised. For example, does changing a specific number of screens or forms constitute a major release, or is anything larger than a ‘bug fix’, that is anything introducing new functionality, a release which necessitates a re-test?
• Is it acceptable to constrain the scope of the penetration test to only those elements or modules of an application that have changed?
• Where will penetration testing occur? Is the organisation highly security-oriented and is penetration testing in production justifiable in order to minimise threats introduced in deployment? Is the organisation’s availability so critical that tests can only ever be sanctioned in pre-production environments?
• Can the organisation afford the desired rate and thoroughness of testing (costs consisting not only of that of the testing itself, but the subsequent remediation of findings and retesting, as may be necessary)? Do lower sensitivity systems really warrant the cost of a lengthy third-party test, relative to the risks they pose?
• What would be the consequence of a decision not to penetration test?
• What commitments have been made (or are likely to need to be made in the foreseeable future) regarding the performance of penetration testing? Do any customers have an expectation, or contractual permission to see the results? Do any regulatory standards which apply require penetration testing?
Once made, these decisions should be formally recorded, along with narrative justifying these choices. Doing so will not only make these choices more defensible in future, but it will also help drive good decision-making in the first instance. Furthermore, perhaps less obviously, it will assist when periodically reviewing these decisions in future. Specifically, if the historical reason for a given stance is formally recorded, it will be much more readily apparent if this reason no longer applies and hence the stance should be reconsidered.
As well as building these requirements into documents such as the Information Security Policy, consideration should be given to other existing documentation which ought to be updated to reflect the decided-upon strategy. For example, Software Development Life Cycle (SDLC) documentation needs to have its ‘testing’ section and diagrams updated to explain the location(s) in which penetration testing is expected to occur.
Even once formalised and captured in documentation, it should not be assumed that this will translate easily or quickly to these requirements being followed as a matter of course within the organisation. There should be an exercise of communicating this to all parties involved in the implementation of new systems. As well as the (perhaps obvious) candidates of technical IT staff, individuals in project management and programme management, there are also key stakeholders who need to understand these requirements. The attribution of responsibility for ensuring that penetration tests are undertaken needs to be formally defined and allocated to a specific party, or class of parties; otherwise there is a danger that it will be missed. One traditionally popular approach is to make the information security team responsible for performing the testing, or ensuring that the testing is performed.
Given the growing tendency for ‘Shadow IT’,2 however, this increasingly includes individuals outside the IT department. Ideally, anyone in a position to purchase software or systems (i.e. any budget holder) would be aware of these requirements. Such responsibilities usually need to not only be communicated, but to be documented, in order to create the accountability that may otherwise be missing, without this clarity. As an example, less knowledgeable system owners, especially those outside the IT department, may incorrectly presume that IT, or information security, are automatically aware of the existence of all systems associated with the organisation and will facilitate penetration testing on their behalf. This also presumes that such parties will have some degree of understanding of what a penetration test is and that it is something necessary to perform: an assumption which is not always correct.
Should any changes be made to the organisation’s stance on penetration testing, this should not only be re-communicated, but the appropriate documentation updated. An example of why this might occur would be changing regulatory requirements, or revised risk appetites following an information security incident within the organisation.
ALIGNMENT OF POLICIES AND PROCEDURES WITH THE CHANGING NATURE OF THREATS
IT is a fast-moving discipline in general and information security even more so. A key part of maturity with regards to a penetration testing strategy is recognition that the strategy itself should be reviewed regularly and, where appropriate, adjusted. Ideally, when a review of documentation which defines an approach to penetration testing is performed, it should not just be reviewed from the perspective of ensuring it reflects the current reality within the organisation. Rather, the appropriateness of the strategy itself should be considered and re-evaluated.
It is generally expected that those parties who are determining and reviewing the strategy remain up to date in their knowledge of threats and controls which are expected to mitigate them, and how the validation of the effectiveness of these controls can be built into the scope of penetration tests. Failure to do so will result in a strategy that quickly falls out of date: for example, by failing to consider fundamentally new threats such as changes in the way in which network connectivity is provided or how a user’s identity is validated.
Following any information security incident which involved a breach of a system, it would be a sensible approach to review the strategy which determines when penetration testing is necessary. For example, by asking questions such as:
• Would a penetration test have detected the vulnerability which was used to gain access to the system?
• Would a penetration test have pre-emptively identified that the control that failed was not operational?
• Would a penetration test have identified that there was no control in place?
Updating the organisation’s strategy (and associated policies and procedures) based on these answers means a more appropriate position on the use of penetration testing can be reached. Where obstacles to penetration testing (budget, prioritisation of new features over security fixes and so on) have previously been met, the tangible and close proximity of a security incident within an organisation can often have a sobering effect that subsequently removes these obstacles.
Another consideration in larger organisations is what other parts of the same group or corporation are doing and the services they offer. For example, one part of a software house might be producing marketing web pages, where a breach may have lower material impact; whereas another area of the organisation might make software for the financial services. The reputation of the latter will likely be harmed by association with a breach occurring in the former. This risk can mean that in some organisations it is prudent to adopt a ‘blanket’ requirement to penetration test all systems; even ones with a marginal degree of material risk.
Even in smaller organisations, the future strategy will play a part. Embedding penetration testing and developing maturity takes time. It can be a wise strategy to initiate it before it becomes a firm requirement for customers or regulatory drivers force it. This way, the inevitable teething problems can be overcome before the stakes are high enough that failing a penetration test and its re-test has a high consequence, or affects the decision to release a highly anticipated new version of a piece of software.
Budgetary and costing issues
No matter how efficient or leanly run, all forms of penetration testing are, in the short term, more expensive than not testing. At a high level a decision not to perform a penetration test can be seen as short-term gain (in terms of cheaper new software and system) at the cost of long-term risk.
When explaining to C-Level management, aside from providing the cost breakdown of the penetration tests themselves, it is important to highlight other factors that impact the cost to benefit to be considered. These include:
• Scale of coverage. A modest penetration testing budget can still achieve something meaningful when sensibly targeted at the most important system, or the one containing the most high-risk information, even if nothing else on the estate is tested.
• Customer expectation. Increasingly, customers are asking vendors for assurances that they perform penetration testing, before procuring their services. This is becoming increasingly true, even in markets such as retail, travel and hospitality, which have not historically been seen as security-conscious. This point can often be used to ‘sell’ some of the benefits of investing money in penetration testing, particularly if an organisation has encountered such questions from prospective customers.
• Regulatory. Generally, regulatory pressures are shifting to try to incentivise the desired behaviour of performing penetration testing; or, at least, to make the cost:benefit ratio of performing such tests appear more desirable. A particularly prominent example at present is GDPR, with fines of up to 4 per cent of group turnover for any organisation that has had a breach of personal data.
• Brand preservation. Commonly, people only consider the direct monetary loss of a breach, or any resultant fines when making a cost:benefit analysis of penetration testing. While this is somewhat understandable, given the good cost transparency that exists, it is far from the only cost. Notoriously difficult to calculate, brand damage and a loss of trust (and consequently, custom) are very real risks of a breach. What should be clearly communicated is that even when it is unsuccessful in preventing a breach, the performance of penetration testing has some benefit in reputational damage limitation. An organisation that took all reasonable steps and was hacked anyway is far more likely to be quickly forgiven than one which was deemed to be negligent and skimping on basic security controls such as penetration testing, which have now become an expectation.
After-effects of a breach
Earlier in this chapter, we discussed Boomerang Video Ltd. Even though the breach is over and, presumably, the vulnerabilities concerned have long since been addressed, it is still mentioned when the organisation’s name comes up. Looking at discussions and reviews of the company online, it was still possible in 2018 to find both message-board discussions criticising handling of the breach and ratings site posts of the company, which reference the breach and which continue to directly impact contemporary ratings of the company. The ease of searching for reviews online means that many consumers’ personal ‘due-diligence’ consists of researching a company in this way and critical comments such as these continue to influence those considering using the service. Although it is next to impossible to determine precisely how many potential new customers are still being lost, it is safe to say that senior management and the marketing department within the organisation would prefer not to have this persistent reputational damage.
There is some discussion by commentators that have performed analysis on the post-breach share price of organisations and surveyed the general perception of the (current) trustworthiness of a company after a breach, which suggest the effects do not last forever. It appears that the negative effect is most prominent immediately after the breach but over a period (estimates vary, but appear to be a year, possibly two), the organisation tends to return to the former position (author’s generalisation of research findings).
Cost:benefit of penetration testing
Penetration testing, regardless of whether it is performed internally or externally, invariably requires resources to perform. In organisations that have not previously performed penetration testing, or have done so only sporadically, it is likely that doing it regularly will be perceived as a fundamentally new or additional cost by budget holders and, as such, may be met with resistance.
Some of the benefits that can be presented, which may help offset the negative perception of this cost are:
• less need to invest in marketing activities to offset reputational damage following a breach;
• lessening (or even total avoidance) of regulatory fines;
• increased market share due to increased faith in solution by customers, as testing occurs;
• avoidance of potential loss of customers following a breach;
• early detection of security defects mean they can be remediated sooner in the process, ultimately more cheaply.
Who will conduct penetration tests going forward?
In many cases, rather than defining who will perform the test, it makes sense instead to define who is responsible for ensuring that the test is performed. For example, an organisation may determine that the chief information security officer (CISO) is ultimately responsible for ensuring that a penetration test is performed, without expecting them to perform it themselves. Such a choice of stance also provides flexibility as to how tests are carried out. If there are other stakeholders who require penetration testing to be performed, these too are to be considered and expressly stated.
For example, an organisation may have a variety of systems and may make the strategic decision that only certain key systems warrant the additional assurance of a penetration test. At the same time, this may be governed by regulatory drivers for particular data, or for systems connected to related organisations with high security requirements (for example, banks directly connected to a financial institution).
Where this is the case, a statement of which systems must be tested and by whom may be appropriate. For example, appropriate text might read:
The budget holder who funds the purchase and support of a given IT system is identified by default as the Owner of the system. Owners are responsible for determining whether they consider the risk posed by the data in a given system warrants the additional protection and assurance provided by a penetration test. [Company] recognises that for its systems which are directly connecting to the Government Connect Secure Extranet (GCSX) network, it is a requirement to utilise CREST- or CHECK-certified external penetration testers.
Where broader internal risk management frameworks exist, it can be useful to also introduce a requirement to formally capture (perhaps as a part of a project initiation document) the basis on which the decision whether to test or not was made. By insisting upon such a justification, it encourages consistency with the organisation’s stated stance on when penetration tests are to be used. If there are genuine reasons why a penetration test is not necessary, it allows them to be captured, so that these are more readily apparent to any parties scrutinising this decision subsequently. Similarly, it is clear that there was a conscious decision not to test, rather than simply that an oversight to test is trying to be dishonestly justified after the fact. Such decisions must of course take into consideration the requirement of any applicable legislation, such as PCI DSS or the Health Insurance Portability and Accountability Act (HIPAA), which explicitly requires a penetration test to be performed.
AWARENESS RAISING AND NOTIFICATION
Dependent upon the scope and nature of the testing, there may be no need to notify the broader community; for example, a test solely designed as an application test, which seeks only to discover vulnerabilities in code and explicitly excludes vulnerabilities in infrastructure and platforms used to host the software. In such a test, it may be possible to provide testers with the code away from your production environment and no one outside the development team needs to be aware that it is being carried out.
As soon as the scope of the test touches any infrastructure which is shared with production systems, at a minimum, this question needs to be considered: Is there a significant enough chance of affecting production to warrant notifying users, or at least system owners?
Depending upon the answer to this question, decisions need to be made as part of the change management process as to who does need to be notified. For example, unless it has expressly been requested, penetration tests typically omit actually validating the existence of a suspected DoS (Denial of Service)3 vulnerability, for fear of causing genuine disruption to live services. Therefore, depending upon the level of confidence that the test can be conducted in a manner that is highly unlikely to affect production systems, it may be decided to narrow the number of parties who are notified.
At the same time, even the most careful penetration tester may accidentally affect production systems. Some of the reasons for this include:
• poor scoping by the party commissioning the test;
• changes made to the environment during the window of time between the definition of the scope and the performance of the test;
• automated tools, particularly ‘spidering’ discovery tools, straying beyond the intended boundaries;
• omissions by the penetration tester of clauses that limit the range of a given test, or other syntactic errors of limit conditions;
• human operator error, for example, typing the wrong IP address of the system to test.
Given this potential for such mistakes, consideration needs to be given to the consequences of a mistake and what this could mean for the environment at large. As a general rule, in lieu of any other agreement, the level of seniority within an organisation, appropriate for providing ‘sign-off’ authorisation of the test needs to be at least as high as that of the ownership of the system being tested. A test of a small system used by only the human resources (HR) department could reasonably be sanctioned by the HR manager responsible for running it. Conversely, it could reasonably be argued that any test of the main production system be signed off by the COO (chief operations officer), or at least an individual the COO has explicitly empowered to do so on their behalf.
OTHER FACTORS FOR CONSIDERATION
Given the growing trend towards adopting cloud-based, or software as a service (SaaS), applications, consideration must be given to the need to adequately treat the risks of vulnerabilities, where penetration testing cannot necessarily be performed by the customer consuming the services. Unlike ‘on-premise’ solutions, SaaS frequently has services shared between multiple customers. Vendors sometimes argue (sometimes legitimately and sometimes from a position of cynicism and excuse-making) that they cannot permit a customer to perform a penetration test for fear that it will cause disruption to other customers, or even result in the disclosure of other customers’ data.
One approach to deal with this increasingly popular model of software is to ensure that responsibilities for some equivalent form of testing being carried out are clearly allocated. The most common means of doing so are to either: (i) reserve the right to perform (or for a third party of the client’s choice to perform) a test of the system; or (ii) require the organisation providing the solutions to offer evidence that they have arranged for this to be performed on the customers’ behalf.
The general trend is towards organisations favouring option 2 (ii) for SaaS which is general, rather than a solution specifically produced for a single customer. Typically, this model requires that it is a suitably qualified, independent third-party penetration tester that performs the test. A legitimate reason for using a single third party is that it would be impractical to permit a thousand customers on a shared environment to each conduct their own penetration tests, according to their own schedules.
What remains contentious, however, is the level of detail of information which vendors are expected to share with customers regarding the tests. This is of course open to negotiation, between client and vendor, but factors which generally play a significant part in the outcome include the following:
• Regulatory model governing the service – Does the customer, or the customers’ auditor, have an explicit need to see the results of the penetration test?
• Customer’s size, relative to that of the vendor – An especially large or important customer may be able to convince a vendor to disclose a greater level of detail than a smaller one might. Similarly, blue-chip ‘Mega Vendors’ will often get away with citing that the sheer quantity of customers they have prevents them from deviating from a ‘one size fits all’ model, which they implement for all customers and effectively refuse to enter into negotiations with individual customers in this respect.
• Level of security maturity of the vendor – A vendor with a high degree of confidence both in their current security posture and their ability to remediate any future penetration test findings is likely to be more willing to be transparent with the results of tests than a vendor who is not.
Although far from standard or formalised, there are some generally emerging conventions with regards to which information specifically is shared. It is generally difficult for a vendor to argue against sharing:
• the fact that penetration testing occurs (or not) for a given system and the frequency with which this is performed;
• whether the testing is performed by a third party or internal resource;
• whether any certifications or accreditations are necessary for the testers to be eligible to perform tests (for example, CREST or CHECK certified);
• the scope of the test and, crucially, confirming that the system or service(s) under discussion are included within the scope of this test.
Again, negotiation is possible, but vendors tend to be more reluctant to formally agree to the following:
• The detailed results of previous tests, including the specific vulnerabilities and where in the system that they exist.
• To notify the client as soon as a vulnerability is discovered by any future penetration test.
• The time frames in which they will fix any vulnerabilities detected by a test and that they will provide evidence of this remediation by sharing details of the results of a re-test covering the specific vulnerabilities which have been discovered.
Another factor to be considered that was still developing at the time of writing is the move by many organisations towards an ever-quicker release cycle. Historically, many monolithic applications only have a handful of releases per year. With Agile came an expectation that greater numbers of smaller releases provided a better way of working and this trend continues with DevOps4 and the move towards ‘continuous integration’ (CI)5 and ‘continuous delivery’ (CD).6
In many regards, these forms of more regularly delivering software are seen as advantageous. Even information security can generally be in favour of such an approach, as it hopefully means that time to implement fixes for vulnerabilities is decreased. With penetration testing specifically, however, the challenge becomes how to keep up a rate of testing that matches the ever more rapid form of deployment.
It appears clear that increasing amounts of automation of security testing will form at least part of the solution. Wholly automated testing, however, cannot really be considered penetration testing. In terms of means to address this, first, some vendors are starting to offer hybrid services where there are ongoing automated tests, augmented by periodic human intervention to provide additional assurance. Second, more frequent releases typically mean that individual releases are smaller and, as a result, it is possible that only a small number of areas of a system have been changed. It therefore becomes valuable to adopt means of identifying which areas of a system have changed. One such approach is to implement extensive file integrity monitoring (FIM),7 and to make the output available to testers. By being able to highlight where changes have been made, the testing can be more targeted, rather than having to perform a (more time-consuming and costly) complete re-test across areas which definitely have not been altered. Other mechanisms by which this means of demonstrating consistency has been enforced include: the use of containers that are read-only and through the use of CMDB (a configuration management database).
We have discussed how penetration testing is increasingly becoming seen as a ‘must do’, as opposed to a ‘nice to have’ activity. Some of the impacts of tests have been explored, along with some considerations to be made with regards to notifying users of systems that testing is occurring and notifying stakeholders and system owners of the results of the test, their implications and follow up. We have also discussed the need to ‘evolve’ penetration testing strategies on an ongoing basis perhaps after discovering vulnerabilities and especially after breaches. Perhaps most importantly, it is necessary to ensure that penetration testing is integrated into the broader processes and workflows of the organisation, if it is to be successful, especially as software lifecycles tend to become quicker.