Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Chapter 3. Rigor

The last rule was to make enumerations so complete, and reviews so comprehensive, that I should be certain of omitting nothing.

René Descartes, Discourse on the Method

The Principle: Specify and enforce the expected states, behaviors, and processes governing the relevant systems and actors.

Key Question: What is correct behavior, and how am I ensuring it?

Related Concepts: Governance, Requirements, Monitoring, Audits

Rigor is the principle of ensuring that our work is thorough, methodical, and robust. It is where we build and carry out processes that reduce confusion, enhance accountability, and improve upon themselves in light of new evidence.

Information security can’t be all fun and games; sometimes you must put in the work to ensure that you get the job done right. Rigor is the Principle about doing things right. It is where procedure, governance, accountability, and oversight have their day in the sun. It’s never good enough to rest on assumptions. Rigor makes you write those assumptions down, justify their existence, and spell out a plan for when they blow up in your face. When your manager, CEO, or board of directors asks you “how can I be sure?”, Rigor is your answer. It is the series of processes that you put in place and the steps you take to follow through to ensure that information security consistently, efficiently, and effectively accomplishes its goals.

In his book The Checklist Manifesto, Harvard professor and surgeon Atul Gawande paints a troubling picture of the state of surgical assurance. Surgeons, as Gawande describes them, tend to view themselves as rockstars in the operating room, expecting absolute deference to their skills and discretion to ensure patient safety. And to some degree this makes sense: surgery is complicated, and a lot of things can go wrong; surely you shouldn’t try to predetermine the surgeon’s actions when there are so many contingencies? Yet, Gawande goes on to prove just how wrong this assumption can be. Surgeons, it turns out, are just as fallible the rest of us, and relying purely on their instincts can lead to common, yet impactful, mistakes. To this end, Gawande shows how substantial gains in patient outcomes can be obtained through the implementation of a simple baseline of procedurality—a checklist.

The results Gawande describes from adherence to his checklist are staggering: 36% reduction in major complications; 47% reduction in patient deaths; 50% reduction in postoperative infections. All from adherence to a 19-step checklist. This list was not designed to encompass the vast array of surgical complications that might arise. Rather, it was designed to identify the most impactful and frequently skipped steps to ensure patient safety. The items on the list—appropriate timing of antibiotics, proper hygiene and cleanliness, building out communication, and teamwork among the surgical team—were not complex or unknown. In fact, they were among the most basic things doctors are taught. Yet by taking steps to ensure that these most basic procedures were followed, hospitals were able to substantially improve patient outcomes, and by extension, reduce costs.

The lessons Gawande describe are paramount to understanding Rigor. The role of the checklist is not to do the surgeon’s job for them. The checklist is a tool to aid surgeons in doing their job more effectively. Indeed, across a range of fields, from logistics to aviation to structural engineering, one of the most important traits Gawande identifies for an effective checklist is to make room for experts to do their job, and, perhaps more important, communicate with one another. By understanding this fundamental balance—between prescribing action and imbuing flexibility—security practitioners will be able to create the processes to ensure appropriate action, while allowing for the freedom to actually carry out those actions. For instance, when crafting incident response “playbooks,” those playbooks need to be clear about who has authority, what each actor’s responsibilities are, and what steps need to be taken in response to specific incidents. But playbooks can never predict every future scenario, and thus must ensure that people with expertise and experience are empowered to make critical decisions when a best path forward is not set out in advance.

The Principle in the Wild

Implementing rigor is a matter of specifying correct behavior and then ensuring that it is happening in practice.

Document the expected inputs, outputs, and behaviors of each element. Software, especially complex software, can be very difficult to get right. Writing things down makes design easier, testing faster, and reduces the need for meetings to try to keep everyone on the same page.
Put in place mechanisms that monitor security-sensitive activities and provide a steady flow of security critical data. Decisionmakers are only as good as the information that informs them, and in today’s information systems, that information must be acquired quickly and evaluated efficiently.
Monitor and audit systems and software to ensure their actual behavior reflects specified expectations. Assuming that systems and software are working without actually checking, (or simply burying your head in the sand), is a recurrent source of security failures.
Follow-up on security training and policy developments to make sure they are actually being implemented, that they are effective, and that they are striving towards continuous improvement.
Develop key performance indicators (KPIs) for security critical systems to aid in evaluating whether security strategies or controls are having a beneficial impact.

Rigor and Its Trade-offs

At its core, Rigor is a two-phase process. In phase one, you specify what your system needs for it to be rigorous. In phase two, you enforce what you specified in phase one. Phase one literally just asks you to write things down. Don’t rest on assumptions: make sure everyone is on the same page. Likewise, phase two is about going back and making sure that the things you wrote down are actually working, and stepping in to make adjustments where they aren’t.

Nevertheless, actually implementing rigorous processes requires understanding a few key trade-offs:

First, Rigor is designed to support, not supplant, the role of the security practitioner. We want to avoid falling into a checklist–compliance mindset. Although checklists can aide Rigor when known solutions exist, they are frequently misapplied, simply out of a desire to do something. Ultimately, the goal of Rigor is to ensure that practitioners do their job optimally. This means avoiding vague standards that are difficult to interpret, and avoiding overly strict requirements that are too burdensome to meet (and therefore ignored). Effective Rigor will always fall somewhere in the middle.

Second, rigorous processes should be built upon evidence of what works. By evidence, we don’t mean flat rejection of anything not supported by gold standard, randomized controlled trials. Practitioners must make pragmatic judgments based on the best evidence available. Decisions are made by weighing the totality of the evidence available—be it expert advice, scientific research, first-hand observation, or the experience of similar organizations—and drawing reasonable conclusions.

Third, no process will ever be perfect directly out of the box, so the goal of Rigor should be to build processes that evolve and improve. Each time you enforce your specifications, you are likely to learn a little more about what works, what doesn’t, and where you have room to improve. So, when doing Rigor, don’t view it as a static, one-and-done process. Truly rigorous security improves on itself.

Although Rigor means more work, you can view the structure that Rigor puts in place as an opportunity to thrive, innovate, and experiment. Too often, when the whole of the world is open to us, we feel overwhelmed, and stick to what we know best. Rigor is a chance for practitioners to play within the new bounds set in place. Although you’ll rarely get Rigor “right” on the first try, these failures will provide an opportunity to iterate and improve in the future.

Interactions

Due to its potential to be applied overzealously, Rigor is best understood in conjunction with Proportionality. As discussed earlier, Rigor must always leave room for practitioners to actually do their jobs, which often requires a fair amount of leeway. And more fundamentally, Rigor can simply take a lot of time, something we all seem to be short on, and which is crucial in security scenarios like incident response. Proportionality helps temper the aggressive middle-manager tendencies that Rigor can arouse in some people, by ensuring that the processes being put in place are actually helping the mission. We don’t enact policies and procedures just because we can; each should be serving a clear purpose to help better fulfill the organization’s mission.

Takeaways

Rigor is the Principle by which we take steps to ensure that security is done properly.
Rigor has two phases: specification and enforcement.
Rigor is an ongoing and continually updated process.
Rigor must make room for practitioners to do their jobs.
Rigor is important to understand in combination with Proportionality (Chapter 7).

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for 3. Rigor

Create new playlist

Sign In

Sign Up

Chapter 3. Rigor

The Principle in the Wild

Rigor and Its Trade-offs

Interactions

Takeaways

Table of Contents for
3. Rigor