Chapter 2. Bridging the InfoSec and DevOps Cultures

A common misconception about DevSecOps is that InfoSec and DevOps have irreconcilable goals due to the absence of conventional security controls and processes. On the one hand, we have the zero-trust principle of never trust, always verify, while on the other, trust is by far the most critical element of DevOps success. InfoSec professionals are commonly divided into color-coded groups with well-defined responsibilities such as red teams dedicated to offensive security, finding and exploiting possible points of attacks, and blue teams, responsible for defensive security, protecting the systems against real and perceived threats. Yet DevOps culture seeks to actively break such barriers and silos. And in a DevSecOps setup, security is everyone’s responsibility.

This doesn’t mean that organizations that use DevOps don’t have adequate controls and processes. Instead, security activities are performed at every stage of software development and delivery, improving its quality and compliance adherence, in a process of continuous security.

This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps cultures and processes, shifting from a reactive approach to a proactive one. It examines key capabilities that are essential to creating a successful DevSecOps program and to scale security as digital transformation becomes a reality for every business.

Accelerate: Capabilities to Drive Continuous Security

Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting research on measuring and improving software delivery. Their book Accelerate: The Science of DevOps (IT Revolution Press, 2018) presents the findings and the science behind their work: 24 key capabilities that influence the outcome of DevOps adoption and that leaders should invest in to seek higher performance. This section explores ten such capabilities that I believe every IT leader needs to promote continuous security.

Leadership

The first step for leaders heading the creation of a DevSecOps program is to understand that its success depends highly on your organization’s technical transformation maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be introduced to the InfoSec team.

Next, be comfortable with the fact that it’s your role to lead the efforts to integrate security into DevOps and generate a culture of high trust between the teams. Your leadership skills can have powerful results, so invest the time to review lean management practices on software delivery performance. IT leaders usually create a dedicated transformation team, selecting some champions to accelerate change and then scaling up their ideas and techniques for the entire team. Still, the challenge is ultimately up to you.

As a leader, your message should be clear: security teams alone cannot avoid incidents. Instead, their effort must be concentrated on improving the organization’s ability to respond to these events. I understand that this is the most challenging part for InfoSec professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a distributed and software-defined infrastructure that makes it easy for anyone inside the team to recreate entire IT environments in multiple regions turned our assets into ephemeral objects. At the same time, the number of security attacks published daily announces that this is a war we are clearly losing. We need more soldiers. As James Wickett once pointed out, there are 100 developers available for every 10 operators, and only one security professional. Once we do the math, we can see that only by genuinely engaging development and operations will we take our defenses back.

Value Stream

In high-performance organizations, teams have a superior understanding of the flow of work value, from the business all the way through to customers. This visibility creates a sense of awareness about which products and features really matter to the company. If you already have a DevOps program running, you can recognise the primary object that we secure: products, platforms, and data that create value to the company goals.

Does it mean that other assets will be managed traditionally and organizations will have a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing organizations. Software-defined networking and its security are becoming popular technologies, augmented by the use of artificial intelligence. Cloud-computing and distributed offices are also breaking the perimeter concept, and for high-performance companies, there are no more clearly defined lines between what is inside of the network or somewhere in the cloud.

A good example for InfoSec professionals is endpoint security. Once upon a time, the number of antivirus updates was one of the key metrics in our SOC dashboards. The Covid-19 crisis and the sudden necessity of home office work globally reduced the level of concerns high-performance organizations had with notebooks and who are using them. Instead, the core of a DevSecOps strategy became how to make our products and data resilient to any attack, be it by an internal or external device.

Collaboration

A successful DevSecOps program is all about better collaboration between teams. Leaders must promote trust between the people involved, and a starting point is to work on the InfoSec communication style. As a craft that in the mind of many is associated with the military, InfoSec sees its share of manipulative and coercive language that sometimes induces fear and blame. On the other hand, DevOps embraces a more blameless culture, where everyone feels safe to make mistakes. In a psychologically secure environment, professionals feel confident enough to express their ideas and take risks, resulting in innovative changes.

Another DevOps best practice revolves around the idea of shared ownership and common goals. Once it is established that security is now everyone’s responsibility, the teams must work together to achieve such a goal. They can create and share mechanisms or tools that help ensure the security of applications and platforms, such as libraries and authentication and encryption services. With everything available and easily searchable, it is easier for developers and operators to reuse secure code.

Post-mortem analysis, conducted after a security event happens, must also involve developers and operators. After identifying the source of the attack, together they can think about the necessary training and consider actions to prevent it from happening again in the future, including the action plan on the team’s roadmap.

Shift Left

Shift left means more than making development and operations teams responsible for securing what they built. It also means moving secure thinking to the earliest possible point in the development and delivery process.

Security can be included in the product design phases, which can reduce later security costs. Likewise, scans for insecure code and configuration mistakes can begin inside the integrated development environment (IDE) software -- an application that provides comprehensive facilities to computer programmers -- providing fast feedback and improving code’s quality.

When a security issue is found, it should be centralized and tracked inside a unique software issue platform. Some issues that may not need an InfoSec specialist to be fixed can often be quickly solved by a senior DevOps professional, for example. This lightweight change process accelerates and improves vulnerability fixes.

CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps between a developer submitting their code and the release of that code into production, run vulnerabilities assessments in the merge request so developers can fix them immediately, as shown in Figure 2-1.

Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.

Empowered Teams

To give Dev and Ops ownership for security and thereby achieve better results, we need to train them to make informed decisions and also make clear that the security professionals are there to support them to make such choices effectively. Developers and operations teams already are responsible for many tasks and may need to be inspired to contribute to security. Otherwise, they may be tempted to merely plug a scanner inside the CI/CD pipeline and wait for better results.

Tip

Are developers aware of secure development practices such as input validation and data cryptography? Are they able to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online community that produces freely available content and tools for web application security. It can be a valuable resource for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure Coding Practices" material and the Top 10 Web Application Security Risks list.

To show that security is there for DevOps, you can start by promoting an informal chat between the two teams. While working at Etsy, Zane Lackey made t-shirts and other swag and gave them to anyone who approached his team to ask a security question. “Whether that was they reported a phishing email or a lead architect asking, Hey, can you have a quick look at the service that we’re actually starting to think about doing?” says Lackey. In the following weeks, the swag was seen inside the office, and people kept asking security questions. This practice would basically do 90% of the branding and cultural awareness work for a security team internally.

Test Automation

It is possible to bring more security to a DevOps pipeline by automating vulnerability scanners. For code security, in the pre-build phase, we can use static application security testing (SAST) tools: this is a test we perform in a non-production environment that inspects an application’s code for coding flaws, back doors, and malware. Open source tools such as Brakeman and SonarQube are good examples.

We can also scan dependencies at this stage. It is a matter of inventorying all dependencies of binaries and executables and ensuring that these dependencies, over which we often have no control, are free from vulnerabilities or malicious binaries. The OWASP dependency check and the Snyk open source tool are the industry standard for this test.

After deployment, we resort to dynamic application security testing (DAST) tools: unlike static tests, DAST consists of assessments performed during execution, monitoring items such as system memory, functional behavior, response time, and overall system performance. This test is similar to tests that an attacker would do to gain improper access to the application. Examples include OWASP ZAP, Arachini, and AppScan. Some penetration tests can also be performed in an automated way using tools like Metasploit, Burp Suite, and Nikto.

Infrastructure security tests can also be automated: from container to cloud security, a tool can be included within the software delivery pipeline, as demonstrated in Figure 2-2 and explored in more detail in Part 2 of this book.

Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.

Working in Small Batches

In the same way that development and operations slice their work into small pieces, security tasks should be completed in a week or less. For example, instead of extensive vulnerability assessments, teams must aim for rapid and effective security fixes. This is precisely the idea behind the “start little and do often” principle of the Agile threat modelling: a continuous, timeboxed process to help teams talk about risk and build security within.

You can then create a user story in the team backlog. The report should be short and include a description of the possible threats identified. Start small, identifying the easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce small victories and allow your team to continue to add new security stories at the beginning of each new sprint.

Tip

The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor. Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and create a report with potential risks and mitigation advice.

Once the team has the muscle memory of threat modelling, you can move to find the highest value security tasks, the ones that will improve the resilience of your systems. In this way, we decentralize and automate such vulnerability assessments, giving our Dev and Ops teams fast feedback and the power to take the necessary measures in their daily routine.

Team Experimentation

Another important guiding principle while adopting DevSecOps is to move from a culture of fear to one of readiness. Teams must accept two facts: first, their software is likely to be used in ways they don’t expect and will be exploited by unrelenting bad actors or automated threats. Second, the security controls they put in place will eventually fail. Accepting that, DevSecOps teams can actively test their resilience against attacks to ruggedize their practices and be prepared to respond promptly to any security incident, improving their systems’ resilience.

This readiness is the main idea behind chaos engineering, a strategy made popular several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s infrastructure by continually – and randomly – terminating instances in production to check how the shutdown would impact business. Chaos engineering has now evolved into security chaos engineering, and the tools to automate such resilience tests are in growing development, including commercial products such as Verica and Gremlin. Chapter 8 discusses this subject in more detail and you can also check out Aaron Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).

Other fun activities to promote team experimentation include game days, with development and operations teams playing capture the flag (CTF)-style competitions. These can be important learning opportunities, especially if scenarios involving the actual business are outlined.

Finally, teams can experiment with automating incident response activities to reduce time to mitigate a critical incident, preventing further damage. Instead of using manual and procedural checklists, there are plenty of open-source tools available such as TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and Deep Instinct.

Visualizing Work and Proactive Notifications

Here, once again I recommend the use of an unique platform to manage tasks and make all the work visible. Many organizations create a single chat room for the IT team, rather than splitting chats into development, operation, and security. This strategy works against the creation of silos, enabling fast communication and collaboration while also reinforcing a culture of transparency. Notifications from your CI/CD pipeline can also be delivered to this same channel, becoming another tool to decentralize remediation, as shown in Figure 2-3.

Figure 2-3. Vulnerability scan’s results notifications are shown inside a chat room.

Monitoring

InfoSec loves a metric. That is why some vendors used to gauge how advanced a security tool was by the number of graphics and dashboards it displayed in security operations centers (SOCs), strategically placed on video walls. (Do a web search for “security operation centers” and you’ll see how much money was invested in this.)

There is a reasoning behind this behavior that can excuse us. Until recently, security was rarely a priority for organizations, and unless you worked in a financial institution, you could struggle with a lack of security resources. To show our work’s value, we displayed all possible metrics on that wall: number of vulnerabilities, outdated operating systems, threats by the level of risk, and so on. And it worked! Our SOCs used to be the first room the Board of Directors showed to visiting customers and partners, rising the InfoSec status quo.

However, if your organization is a high-performer evolving to DevSecOps, you are already managing your infrastructure using code, which makes it easier to automatically update operating systems and regularly scan and fix vulnerabilities. Added to other continuous security practices, the end result is that your team is increasingly reducing the number of successful attacks and also accepting that some will happen anyway.

Now It’s time to let go of our video walls and focus on the metric that really matters for our business: mean time to recovery (MTTR) from a failure. The 2020 State of DevOps Report argued that the level of security integration of an organization is strongly correlated with the ability to quickly remediate critical vulnerabilities. It concluded that those companies with full security integration could recover within one day, whereas those with no integration can take up to one week. In other words, the stronger your DevOps strategy is, the easier it will be to integrate security into the process. The more integrated security is, the easier it will be to recover from failures and contribute to business continuity and growth.

Summary

The adoption of DevSecOps culture through continuous security at every stage of the software development and delivery seeks to replace manual approvals and reduce the need for conventional controls, bridging the gap between InfoSec and DevOps culture.

Security teams alone can not avoid incidents. Any security controls they put in place will eventually fail. Instead, the core of a DevSecOps strategy becomes how to make our organization’s most valuable products and data truly resilient to attacks. In this manner, we move from a culture of fear to one of readiness.

The shift left notion is about moving secure thinking to the earliest possible point in the development and delivery process, making security a shared responsibility for Development, Operations, and Security. The more integrated security is, the easier it will be to recover from failures and contribute to business continuity and growth, so the key metric for our monitoring systems must be mean time to recover (MTTR).

To empower developers and operators to also take on security, we need to train them to make informed decisions. At the same time, security vulnerability scanners such as SAST and DAST can be automated inside a CI/CD platform. Likewise, infrastructure security tests can also be automated: from container to cloud security, a tool can be included within the software delivery pipeline, as we’ll see in the next section.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.88.254.50