Dealing with Security in Requirements

Traditional Waterfall or V-model software development assumes that all the requirements for a system can be captured, analyzed, and exhaustively defined up front, then handed off to the development team to design, build, and test. Any changes would be handled as exceptions.

Agile software development assumes that requirements or needs can only be understood in person, because many functional requirements are like art: “I’ll know it when I see it.”

Specifically, Agile software practitioners believe that requirements are difficult for users or customers to accurately specify, because language is a lossy communication mechanism, and because often what the users say they want is not what they actually want.

Agile requirements are therefore done iteratively and concretely, relying heavily on personas and prototypes, then delivered in small, frequent steps for demos and feedback.

Regardless of how you are specifying requirements, it is often hard to define the security attributes of the software that you are designing and building.

Users are able to explain their needs for software to act in certain ways, but no user is ever going to know that she needs secure tokens at the session layer for CSRF protection—nor should the user be expected to know this kind of thing.

Software development methodologies group requirements like this into sets of cross-functional or nonfunctional requirements, taking into account security, maintainability, performance, stability, and other aspects of a system that need to be accounted for as teams design, code, and test software.

But Agile methods have a difficult time dealing with nonfunctional and cross-functional requirements, but they are difficult to associate with concrete user needs, and difficult for a customer or customer representative to prioritize against delivering customer-facing features.

The security and reliability of a system often depends on fundamental, early decisions made in architecture and design, because security and reliability can’t be added later without having to throw away code and start over, which nobody wants to do.

People who object to the Agile way of working point to this as where Agile falls down. A lack of forward planning, up front requirements definition and design, and an emphasis on delivering features quickly can leave teams with important nonfunctional gaps in the system that might not be found until it’s too late.

In our experience, Agile doesn’t mean unplanned or unsafe. Agile means open to change and improvement, and as such we believe that it is possible to build software with intrinsic security requirements in an Agile manner.

Let’s start with explaining a bit about how requirements are done in Agile development—and why.

Agile Requirements: Telling Stories

Most requirements in Agile projects are captured as user stories: informal statements that describe what a user needs and why. Stories are concrete descriptions of a need, or a specific solution to a problem, stating clearly what the user needs to do and the goal the user wants to achieve, usually from the point of view of a user or type of user in the system. They are written in simple language that the team and users can all understand and share.

Most stories start off as an “epic”: a large, vague statement of a need for the system, which will be progressively elaborated into concrete stories, until the team members clearly understand what they actually need to build, closer to when they need to build it.

Stories are short and simple, providing just enough information for the team to start working, and encouraging the team to ask questions and engage the users of the system for details. This forces team members to work to understand what the user wants and why, and allows them to fill in blanks and make adjustments as they work on implementing the solution.

Unlike Waterfall projects, where the project manager tries to get the scope defined completely and exhaustively up front and deal with changes as exceptions, Agile teams recognize that change is inevitable and expect requirements to change in response to new information. They want to deliver working software quickly and often so that they can get useful feedback and respond to it.

This is critical, since it means that unlike planned Waterfall projects, Agile teams tend not to create interconnected requirements. Each user story or piece of functionality should stand on its own if the team decides to stop delivering at any point. This fixed-time-and-budget, variable-scope approach is common in Agile projects.

Tracking and Managing Stories: The Backlog

As stories are written, they are added to a product or project backlog. The backlog is a list of stories in prioritized order that defines all the features that need to be delivered, and changes or fixes that need to be made to the system at that point in time. Teams pull stories from the backlog based on priority, and schedule them to be worked on.

In Kanban or other continuous flow models, individual team members pull the highest priority story from the top of the backlog queue. In Scrum and XP, stories are selected from the overall product backlog based on priority and broken down into more detailed tasks for the team in its sprint backlog, which defines the set of work that the team will deliver in its next time box.

In some Agile environments, each story is written up on an index card or a sticky note. The backlog of stories is put up on a wall so that the work to be done is visible to everyone on the team.

Other teams, especially in larger shops, track stories electronically, using a system like Jira, Pivotal Tracker, Rally, or VersionOne. Using an electronic story tracking system offers a few advantages, especially from a compliance perspective:

1. An electronic backlog automatically records history on changes to requirements and design, providing an audit trail of when changes were made, who approved them, and when they were done.

2. Other workflows can automatically tie back into stories. For example, code check-ins can be tagged with the story ID, allowing you to easily trace all the work done on a story, including coding changes, reviews, automated testing, and even deployment of the feature through build pipelines.

3. You can easily search for security stories, compliance requirements, stories that deal with private information, or critical bug fixes, and tag them for review.

4. You can also tag security and compliance issues for bigger-picture analysis to understand what kinds of issues come up and how often across projects, and use this information to target security education or other proactive investments by your security team.

5. Information in online systems can be more easily shared across teams, especially in distributed work environments.

Stories in the product backlog are continuously reviewed, updated, elaborated on, re-prioritized, or sometimes deleted by the Product Owner or other members of the team, as part of what is called “grooming the backlog.”

Dealing with Bugs

Are bugs stories? How are bugs tracked? Some teams don’t track bugs at all. They fix them right away, or they don’t fix them at all. Some teams only track the bugs that they weren’t able to fix right away, adding them to the backlog as technical debt.

But what about security vulnerabilities? Are they tracked by the team as bugs? Or are they tracked by the security team separately, as part of its vulnerability management program? We’ll look at this in more detail in Chapter 6, Agile Vulnerability Management.

Getting Security into Requirements

For security teams, the speed that decisions are made in Agile development environments and the emphasis on “working software over documentation” means that they need to stay close to development teams to understand what the team is working on and recognize when requirements and priorities are changing.

As we discussed earlier in how to scale security across teams, you will need to work out how and when you can afford to get the security team involved—and when you can’t afford not to.

The security team should participate in release planning, sprint planning, and other planning meetings to help review and fill in security-related and compliance-related stories, and other high-risk stories, as they come up. Being part of the planning team gives security a better understanding of what is important to the organization, and a chance to help the Product Owner and the rest of the team understand and correctly prioritize security and compliance issues.

If possible, they also should participate in the development team’s daily stand-ups to help with blockers and to watch out for sudden changes in direction.

Security should also be involved in backlog reviews and updates (backlog grooming), and stay on the lookout for stories that have security, privacy, or compliance risks.

Security doesn’t always have to wait for the development team. They can write stories for security, privacy, and compliance requirements on their own and submit them to the backlog.

But the best way to scale your security capability is to train the team members on the ideas and techniques in this chapter and help them to create security personas and attack trees so that they can understand and deal with security risks and requirements on their own.

Security Personas and Anti-Personas

Personas are another tool that many Agile teams use when defining requirements and designing features. Personas are fictional descriptions of different types of people who will use the system. For each persona, the team writes a fictional biography, describing their background and experience, technical ability, goals, and preferences. These profiles could be built up from interviews, market research, or in brainstorming sessions.

Personas help the team to get into the mindset of users in concrete ways, to understand how someone would want to use a feature and why. They can be helpful in working out user experience models. Personas are also used in testing to come up with different kinds of test scenarios.

For teams that are already using personas, it can be useful to ask the team to also consider anti-personas: users of the system that won’t follow the normal rules.

Designers and teams look for ways to make system features as simple and intuitive to use as possible. However, security can require that a system put deliberate speed bumps or other design anti-patterns in place, because we recognize that adversaries are also going to use our system, and we need to make it difficult for them to achieve their goals.

When defining personas, the recommendation is to create a single persona for each “category” or “class” of user. There is very little to gain from creating too many or complex personas when a few simple ones will do.

For example, one system that an author worked on had 11 user personas, and only 5 anti-personas: Hacking Group Member, Fraudulent User, Organized Criminal Gang, Malware Author, and Compromised Sysadmin.

When detailing an anti-persona, the key things to consider are the motivations of the adversary, their capability, and their cutoff points. It’s important to remember that adversaries can include legitimate users who have an incentive to break the system. For example, an online insurance claim system might have to consider users who are encouraged to lie to claim more money.

Personas are used by the entire team to help design the entire system. They shouldn’t be constrained to the application, so understanding how those users might attack business processes, third parties, and physical premises can be important. It’s possible that the team is building a computer solution that is simply one component of a large business process, and the personas represent people who want to attack the process through the application.

Here are some examples of simple anti-personas:

• Brian is a semiprofessional fraudster

  • He looks for a return on investment of attacks of at least £10k

  • Brian doesn’t want to get caught, and won’t do anything that he believes will leave a trail

  • Brian has access to simple hacking tools but has little computer experience and cannot write code on his own

• Laura is a low-income claimant

  • Laura doesn’t consider lying to the welfare system immoral and wants to claim the maximum she can get away with

  • Laura has friends who are experts in the benefits system

  • Laura has no technical competence

• Greg is an amateur hacker in an online hacking group

  • Greg wants to deface the site or otherwise leave a calling card

  • Greg is after defacing as many sites as possible and seeks the easiest challenges

  • Greg has no financial acumen and is unaware of how to exploit security holes for profit

  • Greg is a reasonable programmer and is able to script and modify off-the-shelf tools

For more examples of anti-personas, and more on how to use attacker personas or anti-personas in security requirements and threat modeling, check out Appendix C of Adam Shostack’s book Threat Modeling: Designing for Security (Wiley).

Attacker Stories: Put Your Black Hat On

Another way to include security in requirements is through attacker stories or misuse cases (instead of use cases). In these stories the team spends some time thinking through how a feature could be misused by an attacker or by another malicious, or even careless, user. This forces the team to think about what specific actions it needs to defend against, as the following example shows:

As {some kind of adversary}

I want to {try to do something bad}

so that {I can steal or damage sensitive information
or get something without paying for it
or disable a key function of the system
or some other bad thing…}

These stories are more concrete, and more testable, than SAFECode’s security stories. Instead of acceptance criteria which prove out success scenarios, each attacker story has a list of specific “negation criteria” or “refutation criteria”: conditions or scenarios that you need to disprove for the story to be considered done.

Take a user story, and as part of elaborating the story and listing the scenarios, step back and look at the story through a security lens. Don’t just think of what the user wants to do and can do. Think about what you don’t want them to do. Get the same people who are working on the story to “put their black hats on” and think evil for a little while, to brainstorm and come up with negative cases.

Thinking like an attacker isn’t easy or natural for most developers, as we discuss in Chapter 8, Threat Assessments and Understanding Attacks. But it will get easier with practice. A good tester on your team should be able to come up with ideas and test cases, especially if he has experience in exploratory testing; or you could bring in a security expert to help the team to develop scenarios, especially for security features. You can also look at common attacks and requirements checklists like SAFECode’s security stories or the OWASP ASVS, which we will look at later in this chapter.

Anti-personas can come in very useful for misuse cases. The As A can be the name or anti-persona in question, and this can also help developers to build the so that.

Attacker stories can be tested in an automated fashion. This is particularly useful to teams that follow test-driven development (TDD) or behavior-driven development (BDD) practices, where developers write automated tests for each story before they write the code, and use these tests to drive their design thinking. By including tests for attacker stories, they can ensure that security features and controls cannot be disabled or bypassed.

Attack Trees

A relatively new methodology for understanding the ways that systems can be attacked is to use an attack tree.

This approach was first described by Bruce Schneier in 1999, where he proposed a structured method of outlining growing chains of attack.

To build an attack tree, you start by outlining the goals of an attacker. This might be to decrypt secrets, gain root access, or make a fraudulent claim.

We then map all the possible ways that someone can achieve the action. The canonical example from Schneier says that to open a safe, you might pick the lock, learn the combination, cut open the safe, or install it improperly.

We then iterate on the attack tree, covering any points where we think that we are involved; so for example, to learn the combination, one could find the combination written down, or one could get the combination from a target.

Modern usage of these attack trees can result in very broad trees and very deep trees at times. Once we have this tree, we can start looking at each node and determining properties such as likelihood, cost, ease of attack, repeatability, chance of being caught, and total reward to the attacker. The properties you will use will depend on your understanding of your adversaries and the amount of time and effort you want to go to.

For each node in the tree, you can easily identify which nodes are higher risk by calculating the cost-benefit ratio for the attacker.

Once we have identified the highest risk areas, we can consider countermeasures, such as staff training, patrolling guards, and alarms.

If done well, this can enable security to trace back controls and identify why they put a control in place, and justify its value. If you get extra investment, would you be better off with a more expensive firewall or installing a secrets management system? It can also help you identify replacement controls. So if a security control is unacceptable to the use of the system, you can understand what the impact is to removing it and what risks it opens up.

Advanced users of this attack tree method are able to build per-system attack trees as well as departmental, unit, and even organization-wide attack trees. One of us used this method to completely change the security spending of the organization toward real risks rather than the traditional security views of where to spend money (hint: it meant spending a lot less on fancy firewalls than before).

Infrastructure and Operations Requirements

Because of the speed at which today’s Agile—and especially DevOps—teams deliver systems to production, and the rate that they make changes to systems once they are being used, developers have to be much closer to operations and the infrastructure. There are no Waterfall handoffs to separate operations and maintenance as developers move on to the next project. Instead, these teams work in a service-based model, where they share or sometimes own the responsibility of running and supporting the system. They are in it for the life of the system.

This means that they need to think not just about the people who will use the system, but also the people who will run it and support it: the infrastructure and network engineers, operations, and customer service. All of them become customers and partners in deciding how the system will be designed and implemented.

For example, while working on auditing and logging in the system, the team must meet the needs of the following teams and requirements:

Business analytics

Tracking details on users and their activity to understand which features users find valuable, which features they don’t, how they are using the system, and where and how they spend their time. This information is used in A/B testing to drive future product design decisions and mined using big data systems to find trends and patterns in the business itself.

Compliance

Requirements for activity auditing to meet regulatory requirements, what information needs to be recorded, for how long, and who needs to see this information.

Infosec

What information is needed for attack monitoring and forensic analysis.

Ops

System monitoring and operational metrics needed for service-level management and planning.

Development

The development team’s own information needs for troubleshooting and debugging.

Teams need to understand and deal with operational requirements for confidentiality, integrity, and availability, whether these requirements come from the engineering teams, operations, or compliance. They need to understand the existing infrastructure and operations constraints, especially in enterprise environments, where the system needs to work as part of a much larger whole. They need answers to a lot of important questions:

• What will the runtime be: cloud or on-premises or hybrid, VMs or containers or bare metal servers?

• What OS?

• What database or backend data storage?

• How much storage and CPU and memory capacity?

How will the infrastructure be secured? What compliance constraints do we have to deal with?

Packaging and deployment

How will the application and its dependencies be packaged and built? How will you manage the build and deployment artifacts? What tools and procedures will be used to deploy the system? What operational windows do you have to make updates– when can you take the system down, and for how long?

Monitoring

What information (alerts, errors, metrics) does ops need for monitoring and troubleshooting? What are the logging requirements (format, time synchronization, rotation)? How will errors and failures be escalated?

Secrets management

What keys, passwords, and other credentials are needed for the system? Where will they be stored? Who needs to have access to them? How often do they need to be rotated, and how is this done?

Data archival

What information needs to be backed up and kept and for how long to meet compliance, business continuity or other requirements? Where do logs need to be stored, and for how long?

Availability

How is clustering handled -at the network, OS, database and application level? What are the Recovery Time and Recovery Point Objectives (RTO/RPO) for serious failures? How will DDOS attacks be defended against?

Separation of duties

Are developers permitted access to production for support purposes? Is testing in production allowed? What can developers see or do, what can’t they see or do? What changes (if any) are they allowed to make without explicit approval? What auditing needs to be done to prove all of this?

Logging

As we’ve already seen, logging needs to be done for many different purposes: for support and troubleshooting, for monitoring, for forensics, for analytics. What header information needs to be recorded to serve all of these purposes: date and timestamp (to what precision?), user ID, source IP, node, service identifier, what else? What type of information should be recorded at each level (DEBUG, INFO, WARN, ERROR)?

Should log messages be written for human readers, or be parsed by tools? How do you protect against tampering and poisoning of logs? How do you detect gaps in logging? What sensitive information needs to be masked or omitted in logs?

What system events and security events need to be logged? PCI DSS provides some good guidance for logging and auditing. To comply with PCI DSS, you need to record the following information:

• All access to critical or sensitive data

• All access by root/admin users

• All access to audit trails (audit the auditing system)

• Access control violations

• Authentication events and changes (logons, new users, password changes, etc.)

• Auditing and logging system events: start, shutdown, suspend, restart, and errors

Key Takeaways

Here are the key things that you need to think about to get security onto the backlog:

• Security happens in the thought processes and design of stories, not just during coding. Be involved early and educate everyone to think about security.

• Pay attention to user stories as they are being written and elaborated. Look for security risks and concerns.

• Consider building attack trees to help your team understand the ways that adversaries could attack your system, and what protections you need to put in place.

• Write attacker stories for high-risk user stories: a mirror of the story, written from an adversary’s point of view.

• Use OWASP’s ASVS and SAFECode’s Security Stories as resources to help in writing stories, and for writing conditions of satisfaction for security stories.

• If the team is modeling user personas, help them to create anti-personas to keep adversaries in mind when filling out requirements and test conditions.

• Think about operations and infrastructure, not just functional needs, when writing requirements, including security requirements.