Before we get to the details of conducting the actual interview, it’s important to review the preparation you need to have done to this point. Without having worked through Can and Trust, you won’t have properly filtered the initial candidate pool down to those whom it is worth your time (and theirs) to interview, and the interview will be ineffective because you won’t be interviewing the candidates most likely to succeed in your company. Interviewing people from a correctly refined candidate pool is crucial for a successful behavioral interview. In addition, without having clearly delineated what behavioral characteristics correlate to Success, Failure–Coachable, and Failure–Noncoachable, you won’t be able to construct the correct behavioral question sets and will fail to differentiate candidates based on willingness to perform the job you need done. Properly done, behavioral interviews take time, so they should not be broadly used for large candidate pools. They should only be used for final differentiation, as this is the final phase of the Can–Trust–Will model. Austin Berglas illustrates his role in the interview process:
Austin: I think the biggest hurdle for candidates is getting to the interview phase. For me, that’s where I separate the wheat from the chaff. I had an open posting for a cybersecurity analyst position, and it was directly underneath my lead penetration tester, a vulnerability specialist. So I posted it, and within say four days, I had about 140 candidates. And the difficulty is, I don’t have the time—and my team doesn’t have the time—to interview all 140 people. We’ll get through the ones that are completely unqualified, we’ll cut through those. For me, if you get to the interview stage, the way I’ll do it is I’ll have my lead person do the first interview, then I’ll have somebody else on the team do an interview who is completely separate from that job role, and then if that candidate passes muster, and it appears to be a super great candidate, I’ll interview him. And I’m not interviewing them for technical capabilities; I’m interviewing them for all of the other issues: can they speak properly, are they presentable, do they have the drive to want to learn, are they just looking to punch their ticket and move on?1
Not only does Austin describe a process which tracks the strategy of Can–Trust–Will by quickly and efficiently filtering the broad candidate pool into a refined and more manageable pool of initially qualified candidates, he has the discipline to act as the filter for the Will component—he takes over the behavioral interview aspect. He specifically rejects the temptation to interview for technical capability. Part of the reason is that he has built a trust culture, so he relies on his team to have done the Can work—he’s confident they won’t send him anyone who doesn’t have the correct technical skills. And he also recognizes that if he’s not disciplined in this way, the whole system breaks down. John Kolb has a similar strategy:
John: I think it’s surprising when people get to have an interview like that with me because they expect that I’m going to ask the really hard technical question that they don’t have an answer to, and I usually tell them, “Look, relax. Other people are going to make the decision on whether to hire you. I just want to know who you are.” And that’s incredibly important to me. And so is whether they bring different perspectives. Diversity is an important issue to me—so, are you bringing a different thought process or life experiences to the table, versus the people that are already here, so you can complement that. Those things are the types of questions I ask when I meet somebody. And I like them to talk about themselves—what they’ve done in their lives and what they’re passionate about and so on, and that tells me quite a lot about how they’ll fit into the team, and how they’ll be successful at Rensselaer.2
John’s point is crucial. Diversity is necessary for a variety of reasons. Most important: high-performing teams are made up of different perspectives, approaches, and ways of thinking. John specifically conducts his behavioral interviews to ensure he knows what each candidate’s life experiences and perspectives are so he can assemble a diverse, and therefore high-functioning, team. In addition, he avoids the legal issues which can follow from a poorly-designed hiring process. If you don’t adequately structure the behavioral interview to ensure it differentiates for the capabilities you actually need, it will become a generic personality assessment and may open you to lawsuits from failed candidates (still, remember to always speak with an attorney—be it your in-house or outside counsel, or otherwise—before finalizing your behavioral question sets).
Generic personality assessments are often implemented through testing and are subject to two high-impact flaws. First, behavioral tests only assess general personality traits and consequently miss behavior-based indicators which demonstrably correlate with job performance. In essence, it’s a judgment of whether a person can do the job but fails to assess whether the specific candidate being tested will do the job you need done. We addressed the importance of this distinction earlier. Second, since tests are provided by vendors, the generic personality traits they test for most often do not correlated with success in the particular job role at your company. The closest this type of test will get is to identify personality traits which their survey research has identified to be relevant to general notions of “success” or even survey-based data which proposes that particular traits are indicative of success in a given industry.
Developing Question Sets
There are several ways to get to know a candidate—depending on what you really want to know, of course. When it comes to actual behaviors, the interview can attempt to set up a real-time scenario where the candidate must confront a massive data breach, with PR reps running wild, and sirens blaring. Would that be helpful? Sure. Is it practical? No. So how can actual behaviors truly be identified without setting the candidate off into a customized virtual disaster zone? The next best option is tailored behavioral question sets, which are designed to reveal whether a candidate has the behavior characteristics you have identified as correlating to successful job performance. Again, this is based on the well-established principle that past behavior is the best indicator of future behavior. In addition, it recognizes that it is much easier to accurately analyze a story than it is to analyze a highly prepared answer. The key to behavioral interviews is to ask questions designed to draw stories of the past from each candidate. Andrea Markstrom’s approach is holistic:
Andrea: During interviews, I’ll always ask them a few questions generally focused on situational behavior. Tell me about a situation where this happened, or that happened—but what I’m looking for are individuals who, when faced with a crisis, how do they approach that? Do they go in and do they have their hair on fire? Or is it a calm, collected approach—what is their thinking pattern when they go into a critical situation? Because the most important thing—let’s just say hopefully this will never happen, but if we were in the midst of a breach situation—the worst thing that can happen is IT has their hair on fire. And it is about how do you maintain that cool, calm approach in a crisis situation. So I ask during the interview: explain an example and how did you handle that. So that’s one very important skill set. The other piece is tell me about a time where you made a mistake, and how did you handle that, and how did you communicate it?3
All of Andrea’s questions are behavioral in nature because they ask for descriptors and not statements. It is critical to understand that behavioral question sets are not standard interview questions. Standard questions fail. They fail because they do not reveal anything about the candidate’s willingness to do the job. Asking, “Why are you interested in working here?” will get you an answer, often a well-prepared and good-sounding answer. But it’s not useful in the hiring process because having a good reason for wanting the job does not correlate to good job performance. Adam Bricker asks pointed questions which highlight the candidates’ “mental models” and decision-making processes:
Adam: I would also ask candidates and staff questions specific to the really important cybersecurity positions (looking for what they know, and do they know what they don’t know; looking for collaboration examples). Finally, I ask them to define a situation where they faced a “cloud” where they have to progress through a tenuous situation where they cannot safely see the other side. I have them talk me through their mental model, their decision-making process, their fears, their joys. These situations include a start-up, climbing an unknown mountain in bad weather, dealing with a complex and unpleasant interpersonal situation, a complex incident response, etc. In each, I’m looking for that ethical dilemma and how they thought through it.4
Inherently, Adam’s questions seek insight into how each candidate processes ambiguity and how they troubleshoot and problem solve when they know they don’t have all the facts. In addition, his question elicits descriptions of how a candidate has exercised persistence and previously handled the emotional rollercoaster of working through a stressful experience. Nowhere in the answer will the candidate simply state what they are willing to do, but by listening carefully, Adam reaps a wealth of data about what the candidate most likely Will do in similar future situations.
To do this as effectively, as Adam does, requires the use of behavioral questions, those which generally begin with, “Tell us about a time when …” or “Describe for us a situation where you … .” By asking for stories about previous experiences, the candidate can’t prepare textbook answers, and you don’t fall into the trap of thinking that a candidate who “interviewed well” will perform well on the job. Rodney points out how government agencies are also looking at parts of the interview process as an opportunity to allow candidates to illustrate what they can accomplish:
Rodney: Even as we review applications for jobs, we don’t just look at the degree, and those tried and true credentials, but we are thinking through the interview process and the selection process to find a better way for learners to demonstrate their capabilities or what they can accomplish what they can do and what they can accomplish.5
Viewing the behavioral interview as a method to truly understand what a candidate who Can do the job actually Will do once in a job is critical. And Rodney’s focus on what the candidate can demonstrate, rather than what the candidate can say, is what keeps the interview on track. Only behavioral questions which trigger stories of previous events can demonstrate how the candidate responded and will reveal the deeper values which drive behavior and which otherwise would remain hidden. Once you have the behavior characteristics which correlate with success for your job properly categorized, and the behavioral question sets properly structured, you will be well on track to extracting the data you need to make better hiring decisions with confidence.
Developing question sets is a specific skill which usually requires expertise. To illustrate, we had a client who needed new hires to have a high level of both perseverance and initiative. Two of the five questions we delivered were as follows:
• Tell us about a time you set a goal that nobody thought you could achieve. How did you handle the situation? What was the outcome?
• Tell us about a time when you did more than was required on your own initiative. Specifically, how did you come to decide to do more, how did you decide what to do, and how did you deal with acting without permission?
These questions are difficult to develop because it requires expertise and practice to frame a question which will trigger a story that reveals what you need to know, or even better, will lead to a conversation which reveals deeper insight into the candidate’s character. Based on the four model job roles we developed and presented in Chapter 5, we have created corresponding Will question sets. These are available in Appendix A. Additionally, we have included behavioral question sets related to certain specific behaviors and included those in Appendix B for reference.
Starting the Interview: Setting the Stage for Something Completely Different
At this point, you’ve done the preparation, you’ve correctly identified the candidates to interview, and your behavioral question sets are prepared. What now? The first phase of the interview is preparing the candidate for what you need him or her to do—which is to dig deep for insightful and relevant stories. Behavioral interviews are not easy for either the interviewer or the candidate, but it’s important to emphasize this is not a game of “gotcha.” Remember, you are not looking for a candidate who interviews well. You are looking for a candidate who has a history of actually performing the behaviors you need for the job you are filling. Consequently, part of your job as the interviewer is to help the candidate dig deep for relevant stories. And that job starts with letting him or her know what the interview is about, what you will be doing during the interview, and most importantly, why.
An example of the introduction to the interview might run something like this:
Thank you for coming to interview with us. We are a high-quality team and we spend a lot of time and effort to find people we think can succeed here. We know it’s not for everyone, and that’s why we do this interview. We want to learn more about you.
To do that, we have some questions we would like you to talk about that will require some thought, but that help us understand whether or not this will be a good fit moving forward.
We know it’s difficult being interviewed like this, and we appreciate you being here and going through this process. And we are excited to get to learn more about you as a person. Are you ready to begin?
A more direct version of this introduction might run something like this:
Thank you for coming to interview with us. We are a high-quality team, and we spend a lot of time and effort to find people we think can succeed here. We know it’s not for everyone, and that’s why we do this interview. We want to learn more about you.
To do that, we have some questions we would like you to talk about. And while we want you to be comfortable, we also recognize that sometimes the best replies come when you’re stressed. In fact, sometimes we learn a lot just by seeing which question topics are stressful! Please be assured we’re not going to intentionally provoke you or anything like that, but some of this stuff is difficult and should make you feel uncomfortable. We just want you to know we’re not being intentionally rude. But we are trying to learn about you from how you respond.
So again, we know this is difficult, and we appreciate you being here and going through this process. And we are excited to get to learn more about you as a person. Are you ready to begin?
Some might object to having a written introduction for being so formal and formulaic to the point where it interferes with building the rapport necessary to conduct the interview. The criticism is fair, but there are important reasons for using a template. First, it keeps the interviewer on track. The most significant cause of failure is an interviewer who becomes enamored with a charismatic candidate. Notwithstanding all of the time and effort which goes into planning a behavioral interview, we are all human. And the reason The Big Mistake exists is because it is so very easy to default to it. Second, it also serves to pull the candidate away from their prepared answers. Most candidates will have done a good deal of preparation for the interview, so it’s only fair to let them know up front that the path to success is to trash their notes and their memorized answers, and to speak from the heart. Additionally, it’s helpful to offer a sentiment like: “This is also your opportunity to evaluate (company name) based on what we ask and how we engage with you. Since you’ll be working with me if hired, I encourage you to learn about me while I’m learning about you.”
Finally, using a script for the initial stage helps demonstrate that all candidates were treated equally. Litigation from failed candidates is growing across all industries and simply must be considered during the behavioral interview process, and to remind the interviewer this is a focused conversation and not a feel-good exercise.
There are two additional items worth mentioning at this point. As you begin asking the behavioral question sets, always keep in mind you are evaluating the data given in the responses under the Failure–Coachable, Failure–Noncoachable, and Success assessment structure. This will keep you on track to make a candidate assessment which correlates to job performance. Beware if your thoughts begin tending toward, “What a great answer!” or “I really like this candidate!” The second is that while you should have writing materials to jot down a reminder or a particularly impactful thought during the conversation, the interviewer should never also be the notetaker. One person cannot effectively listen, evaluate, and ask follow-up questions, while also recording the interview content. Record the interview for later transcription or have a notetaker present to record what was said. The last thing you want is to miss an opportunity to dig for a truly revealing story because your train of thought was interrupted by writing down what was just said.
Conducting the Interview
After the introductory statement is done, the interview itself begins. This is a sometimes lengthy process of asking the question sets and interacting with the candidate through follow-up questions and facilitating discussion to ensure the best and most relevant stories are revealed in the interview process. The key here is to listen closely to how the candidate responds and to evaluate all of the data revealed by the candidate: some answers will reveal unexpected depth, and some will reveal a lack of depth. And while the content of the answer will provide a wealth of behavioral information, there is a large volume of additional data available in the context of the answer as well.
Behavioral interviews are not simply interviews to extract basic information. When I ask you, “Tell me about a time of highest stress in your life, and what you did to deal with it,” evaluating the answer to that question is in layers. If you tell me about a time where the heel on your shoe broke and you missed a flight, well, I’m interested to know how you dealt with that stress—but I also need to be aware that breaking a heel and missing a flight is your concept of the highest-stress event in your life. This could be opposed to, “I was in my last semester of grad school, and my grandmother got hit by a car two weeks after my grandfather was diagnosed with Alzheimer’s, I was the only one available so I quit school and moved across the country, and took care of my grandparents in the last eighteen months of their lives.” That is a different level of stressful event. And so when you’re evaluating behavioral questions, you have to understand what is being revealed, not only in the content of the answers but also in how the applicant is choosing to answer the question. This can help decide how the candidate Will function in the job.
Some candidates might be reluctant to share the stories you need. They may reply that it’s too personal and they are uncomfortable sharing. While that reply itself may be valid, the key to differentiating candidates is to consider what the reply reveals. It may indicate a person who is not open or trusting, or perhaps someone who is insecure. It may indicate someone who cannot form relationships quickly or who may have difficulty trusting others. None of these possibilities are disqualifying, but they do give insight into the candidate’s personality. Not in an absolute respect (i.e., open people are better) but in a differential data respect: are you looking for someone who is very private, or someone who is very open? This provides a foundation for your assessment of what the candidate will do during the workday. Again, the key differentiator revealed in a correctly-executed behavioral interview is to extract the behaviors each candidate is most likely to actually perform (Will ) and to differentiate those behaviors from those the candidate is capable of performing, but not likely to actually engage in the ordinary course of the day (Can). And that helps you build an understanding of who the candidate is—which is what you need to know before a correct hiring decision can be made. Alexi remembers being asked questions that attempted to investigate her behaviors:
Alexi: My boss who interviewed me, from what I remember he kept asking me why I wanted to work at BlackBag and I said I got into this industry because I wanted to help people. And what better way to help people than teaching people on how to enhance their skills and understanding of forensics and our tools. He said the main reason he hired me was because of my personality, and the fact that I never changed my answer when he asked that question and he could tell I genuinely want to help people!6
The ability to successfully answer behavioral questions is usually tied to storytelling. Some candidates may not tell stories well, and hopefully, our response to this circumstance is not a surprise at this point—it’s not a disqualifier, it’s a differentiator, which depends on the behavioral characteristics which correlate to Success, Failure–Coachable, or Failure– Noncoachable for the job role in your company. Thus, everything the candidate does during the interview is behavior, and every behavior can be assessed and mapped against the behavioral characteristics you structured when preparing for the interview. It doesn’t matter where the behavior comes from. A story describing previous behavior is as revealing as actual behavior during the interview.
Spotlight: Gosh, I Don’t Know!
What if your interview subject simply doesn’t have an answer? A few years ago, we were interviewing candidates for an analytic job role in a very large company. During a day of interviews, we had two candidates who responded to the struggle to find stories in very different ways. Our first candidate was a young lady who came into the interview focused and highly prepared. She had a stack of reference materials and filled the room with energy—until we introduced the behavioral interview. Upon understanding none of her preparation would be useful, her despair was evident, but she also appeared to decide to soldier on. And then we asked the first question. She paused for a minute and then blurted out, “Gosh, I don’t know!”
Our second candidate could not have been more different. An obviously experienced middle-aged man came into the interview with a calm and practiced ease. He used very effective technique to build rapport and showed little concern when we introduced the behavioral interview. But over the course of nearly an hour, he simply refused to share any stories or reveal any behavioral data about himself. And he did so with a high level of skill. To every behavioral question, he replied with some version of “Well, it’s not really a story, but …” and then would recite a wonderful sounding statement of his capabilities. After he left the room when the interview was over, the trainee who was observing stated how impressive the candidate was, “What a great interview! He had perfect answers for everything!”
It wasn’t until the trainee also listened to our analysis that he understood that the candidate had merely confirmed everything recited in his resumé but that we knew virtually nothing about him as a person. We knew all about his Can but nothing about his Will. He simply did not provide behavioral answers and he did it with great skill. The only thing we were certain he would actually do if he was hired was to speak eloquently about absolutely nothing at all. Our trainee had made The Big Mistake! We evaluated him as Failure–Noncoachable and recommended that he not be hired.
In contrast, our first candidate received a recommendation to hire with an evaluation of Failure–Coachable and a draft onboarding plan based on how she responded after her initial, “Gosh, I don’t know!” As experienced interviewers, we took it as a signal to start exploring and to start helping her dig back through her memory. Starting with “emotional first aid” (a technique you’ll learn in any quality interviewing course), we reassured her that her response was actually a great start and we went straight to the next question in the set. We prompted her to think back to events and activities outside what she might consider to be “work-related” experience; we reiterated that part of what makes stories come to the surface is taking a minute to think and that there was no penalty for sitting in silence beyond what would otherwise be normal in an interview to collect her thoughts. After about ten minutes of conversation and thinking, she began to brighten up and the stories flowed. “Oh, you know what? That happened to me in summer camp!” and, “That reminds me of when I was riding my bike across the country.” She would never have made the connections or found the stories without facilitation, but with facilitation she was able to reveal to us key behavioral characteristics upon which we were able to evaluate her for success (or, in this case, Failure–Coachable) in our client’s job role. This interaction gave us two things: one, her stories, and two, insight into how she problem-solved and how she responded to something unexpected and consequential. The man we gave a no-hire recommendation was unwilling to change in the face of something new. He was Failure–Noncoachable.
The process of facilitating the search for an answer to a behavioral question shows how the candidate is likely to interact with others when employed. A candidate’s willingness to engage in the interview conversation with you to find stories also demonstrates how the candidate responds to guidance and suggestions. And in some ways, it’s a window into how the candidate handles a stressful situation, even if it’s just trying to answer an interview question. Even if it’s not an “I don’t know” answer, but it’s a story which doesn’t really provide much, and in some cases, the interviewer must press the candidate for more detail or for a more significant example. How the candidate responds to being pressed and whether more relevant examples are revealed are additional components of the final analysis.
A key component of the interview process to continuously keep in mind is this is not a game of “gotcha.” Scrupulously avoid finding some kind of success in stumping the candidate during the behavioral interview. If a candidate cannot think of a story, that’s the time to assist the candidate. You are not evaluating the candidate for his skill in having or finding stories. And you shouldn’t really be interested in whether he tells stories well (unless that’s a behavior which correlates to success in the job). You should focus on discovering her track record of behavior. When a candidate struggles to recall a story which answers the question you’ve posed, help her dig.
At the other end of the spectrum, some candidates will have no hesitation in launching into a story. That’s neither good nor bad, it’s just another factor to consider. The type of story the candidate shares and how they characterize it can also reveal a good deal about the candidate’s personality. For example, a candidate whose “most stressful experience” was missing a connecting flight is likely to be a very different person from the candidate who shares the story of moving across the country for a year to give end-of-life care to a grandparent. Again, there is no correct answer. Rather, it’s the type of story and how it’s told which reveal characteristics of the candidate’s personality. Once you have an understanding of who the candidate is, correlating the personality to the success/failure factors of the job role your company needs to have done becomes more straightforward. Amanda has learned how her team functions, how she functions within it, and why certain behaviors benefit her team, in all of their roles:
Amanda: For information security, we can’t just hide in our corner and review logs all day. A really vital component for our success is being able to communicate, especially under high pressure situations. As I previously mentioned, during the interview process, we take into account various factors before making a decision, including how a candidate responds during the white boarding session. Something my boss always likes to say is you know someone’s true colors when they are under stress. Information security will have to communicate with other departments: it is critical to our survival and our success. So we need to know that no matter what role you have in information security, you are going to be able to handle yourself in a potentially high pressure situation; you may need to communicate the issue, collect information from users as part of an investigation, or work with third parties to resolve an incident.7
Now, how to dig in and find those behaviors that you need? Just as developing the actual behavioral question sets is a learnable skill which requires expertise and practice, the same is true of the interview itself. A good starting point for understanding how behavioral questions work and to prepare for conducting your first behavioral interview is to review and become familiar with the sample questions sets in appendices A and B. Based on our interviews with industry experts, our experience and expertise with behavioral interviews, and analysis of the NICE Cybersecurity Workforce Framework, the appendices contain common cybersecurity behavioral characteristics and sample question sets which will extract differential data for those behaviors. Working through the appendices will trigger your thinking, both regarding how the interview should be managed and in developing behavioral questions which are more finely tuned to your needs. That’s the point—the appendices are provided as a jumping-off point for you.
We also recommend that you take some training in interviewing. Again, this is a learned skill, but it is a skill everyone should have. The simple truth about interviewing is it’s a skill which can only be developed through practice, and that means role-playing. Particularly, in HR departments or recruiter firms, ongoing practice in interviewing is crucial to effectively extracting differential data in a behavioral interview. This book is devoted to the process of how to identify, winnow, and select the employees you need for cybersecurity roles in your company. And while interviewing is a crucial part of the process, it’s a complex enough skill to warrant a separate volume. Even if you don’t regularly conduct interviews yourself, you simply must know enough about interviewing to be able to evaluate the skill level of those in your company who do.
As mentioned earlier, how a candidate responds to the digging process is revealing and should be a part of the assessment you do both during and after the interview. For the interviewer, switching from setting up and asking the behavioral question to an open-ended elicitation is a critical part of the behavioral interviewer’s skill set. Having the ability to conduct both a behavioral interview and an open-ended elicitation to support the behavioral questions is important. That’s why interview training and practice, whether through role-playing or by conducting many interviews, is crucial to success.
What you should care about, the thing you need to accurately assess a candidate, is good stories which address the behavioral question sets identified in your job description. Be willing to take any path which gets you to good stories. Any path which prevents you from getting to good stories is a blind alley to be avoided. Through experience, Adam Bricker has developed his own strategy for approaching the behavioral interview:
Adam: While building my professional practice and hiring staff all over the world while serving in a humanitarian aid organization, I learned to ask the questions in an “orthogonal” way. Not, “Are you a good athlete?” Not, “Are you smart?” But answer to ask, “Describe to me a situation in your life where you proactively, of your own volition, put yourself in harm’s way just because you thought it was fun, and describe a situation in that field where you had to navigate through a ‘fog bank’ of decision-making—you had to get to the other side without any clear path.” I use that question to this day, and with extremely rare exceptions (sometimes they still don’t get along well with other people—there’s a required team dynamic there), it has literally never failed. Also, that turns out to be a pretty interesting question to leverage on cybersecurity interviews as well.8
This deeper dive allows Adam to find that concrete grasp of candidate behaviors. John Kolb describes his approach:
John: I’m very intentional in the questions I ask, but I’m not necessarily looking for direct answers or “yes or no” answers. I am looking for something that teases out somebody’s character and personality a little bit. And once again, how ethical do I think this person is, will they stand in the wind if something comes up that’s against where they might want to go or where they think there’s a problem.9
The final observation about conducting the behavioral interview is that the order of questions is often irrelevant. In fact, sometimes jumping between question sets or moving on temporarily from a question which is not lending itself to a story can be a good way to “shake things loose” and open new trajectories for the candidate. The reason question sets are used, rather than just one question for each behavioral characteristic, is to keep your focus on the stories—and therefore the candidate, rather than on the questions. The key is to understand the candidate through good, deep, relevant stories for each behavioral characteristic you have identified in your job description. The interview should end only when you have enough data, or it is evident that a story will not be forthcoming for one or several behavioral characteristics. You’re looking for a candidate who demonstrates the behaviors you need, you are not looking for a candidate who just interviews well.
Spotlight: Who Interviews?
Standard interviewing is a learned skill. It’s the process of directing a conversation with the purpose of extracting information from the person being interviewed. It takes practice, and you’re going to get it wrong before you can get it right. And behavioral interviewing is a step deeper. It’s still a purpose-driven conversation, but rather than extracting information, the behavioral interview extracts demonstrations of previous behavior from the person being interviewed. The bottom line is you have to know how to interview, and that means get trained and then practice, practice, practice. We make this point for a deeper purpose: the companies that make the best hiring decisions always have the hiring manager be a part of, if not fully conduct, the behavioral interview.
At a minimum, the hiring manager needs to observe each interview, but the best practice is for hiring managers to learn how to conduct behavioral interviews, and to do the interviews themselves before making the hiring decision. Why? Because hiring is a consequential decision. If you are interviewing somebody and hiring them to go work for somebody else, you are very likely to evaluate candidates differently than if you are hiring candidates who are going to work for you. Holistically, the interview needs to be done by a person who’s going to be impacted consequentially by the hire—this could be the candidate’s (potential) direct boss, but it can also include the team members that will work with the (potential) new hire as well.
Why is this important? Because, as we’ve mentioned in previous chapters, interviews are where biases most easily show up, because interviewers who are not adequately trained and prepared often decide on the fly what to ask of whom and how to interpret the answer. Everyone knows some executive who is absolutely certain he knows the one question that will really predict good candidates (“If you were stranded on a desert island …”). The Big Mistake, anyone? And there are also studies available which have researched interviews for elite positions, such as those in professional services firms, and concluded that hobbies, particularly those associated with the rich, feature prominently as a selection criterion. Hiring officials simply must learn how to conduct behavioral interviews that extract data which actually correlate with job performance. The Can–Trust–Will process can help executives understand why their “one question” system fails so often, and that learning how to interview is the key to effective hiring and low employee turnover.
This includes interviews that focus on assessing “fit with our culture,” which is the number one hiring criterion employers report using, according to some surveys. And by this time, we already know why this is on the road to The Big Mistake—because it’s one of the squishiest attributes to measure. Few organizations have an accurate and consistent view of their own culture—and even if they do, understanding what attributes represent a good fit is not straightforward. The generic notion of “fit with culture” simply does not correlate to job performance.
For example, does the fact that an applicant belonged to a fraternity reflect experience working with others or does it reflect elitism or does it indicate misogyny? Should it be completely irrelevant? The answer for this example lies in whether “belonging to a fraternity” correlates to job performance. We predict it does not correlate at all—we’d be surprised to learn it predicts behavior of any kind, positive or negative. It simply requires deeper analysis. Letting someone with no experience or training make such calls is a recipe for bad hires and, of course, discriminatory behavior. But it may be helpful, especially for smaller teams, to have a few of the potential new hire’s colleagues sit in. Martin Durst expressed agreement with this idea:
Martin: I think it’s important for the group to sit in on interviews so that we can get a feel for the potential teammate, and let our opinions be heard.10
Many organizations follow this approach. The team as a unit may offer important insights, as Marie Chudolij points out:
Marie: When you’re working with smaller, tight-knit organizations or really just smaller groups, they tend to understand the need for the team to be a cohesive unit. That’s where they are going to have more understanding for other folks to be involved. You know, it’s not necessarily how a traditional hierarchy works, where only the manager or director is able to make these decisions. It’s a team decision. If they’re hiring a new director or manager that I am going to report to, and I didn’t get along with that person, that could pull the whole team apart.11
The key here is to understand that the purpose of the interview is to extract differential behavioral data, and nothing should be allowed to interfere with accomplishing that goal. Having observers is fine, especially if one of the relevant behavioral characteristics is the ability to perform under pressure. Being the focus of attention in a consequential environment full of strangers could certainly be revealing. In addition, we have found it is useful for people learning the behavioral interview process to observe behavioral interviews. It can also assist current members of the team to observe, as part of providing input for the new hires, onboarding program. Still, it is important to set boundaries which should include identifying to the candidate the reason for all the nonspeaking people in the room. And finally, while there may be an egalitarian desire to “let everybody ask a question,” we find this practice does little to improve the outcome and only makes the interview more cumbersome.
Spotlight: Ending the Interview
Asking for candidate questions at the end of the interview is a waste of time. Many of our clients are initially offended at this advice, and from an egalitarian perspective, their reaction is well-taken. The reasons we give this advice begins with the reality that the practice doesn’t reveal anything useful, and it starts you down the road to The Big Mistake. Candidate questions don’t reveal differential data because they are usually crafted in advance so that the candidate can create rapport, show interest, and demonstrate how well she has prepared for the interview itself. It’s irrelevant to the behavioral interview. You’re both mentally drained (as you should be) and now we’re going to talk about hours, pay and dress code? All of that can be addressed during contract negotiations with the candidate who gets the job offer. Additionally, taking steps to “ensure the applicant is left with a positive impression of the interviewer and the organization,” as some generally available advice recommends, is directly contrary to the purpose of the behavioral interview. The purpose is not to impress the applicant, it’s to understand who the applicant is and how he is likely to behave in the future. Simply thank him for participating in such a difficult process and let him get on to the rest of his day while you get on to the next interview.
Analyzing the Interview
Once the interview itself is done, the analysis can begin. This is where you spend a few minutes reviewing the data revealed by the candidate’s stories and make your final decision regarding whether to recommend offering the candidate a job. It does not matter if the analysis is done in a twenty-minute session after each interview and before the next, or if the last two hours of the day are blocked to conduct reviews on the interviews done that day. And it doesn’t matter if the hiring manager conducts the interviews and analysis solo, or if the interviews are done with observers and the analysis is done by the group that was in the room. All that matters is that the behavioral characteristics demonstrated by the candidate’s stories are structured into the list of behavioral characteristics prepared for the job role, and a final assessment is made for each one: Failure–Coachable, Failure–Noncoachable, and Success.
Obviously, Success behaviors support a hire decision, while Failure– Noncoachable behaviors support a no-hire decision. But what you will most often find is most candidates are Failure–Coachable. And we’ve mentioned this previously—it’s actually pretty rare to find exactly the candidates you need, and that’s why we’ve devoted a chapter to new employee onboarding. And since most candidates will be Failure– Coachable, most hiring decisions will be driven by complex factors such as training budget, and the differing levels of time and cost to address the different combinations of Failure–Coachable assessments. Consequently, post-interview analysis will focus on Failure–Coachable behaviors. Most of your candidates will be in this category, and the results will give a good indication of the training needed and the learning curve likely to be faced by a prospective employee. It is often a good idea to prepare and include in the hiring recommendation a first draft of the training and coaching the candidate will need to be successful during the onboarding process.
The easiest evaluations will be those candidates who have a high level of Failure–Noncoachable indicators. These are the quick “no” decisions, but there is a significant note of caution here. Failure–Noncoachable candidates should not be rejected from further consideration across the board. Rather, they should only be identified as not suitable for the specific job role the behavioral interview covered. To extract the most value from your resources, each candidate’s behavioral interview results should be reviewed to determine whether another job, one with different behavioral characteristics, provides a suitable match for the candidate.
More obviously, candidates with high levels of Success indicators are very nearly to the point of being eligible for a job offer. The key with candidates who have high levels of Success indicators is to confirm they have low levels of Failure–Noncoachable indicators. A candidate with high levels of both Success and Failure–Noncoachable should be treated as if they only had high levels of Failure–Noncoachable, because Failure– Noncoachable behaviors have significantly more impact on the work environment than any other factor. In addition, candidates with high levels of Success indicators should also be reviewed for Failure–Coachable indicators to ensure an appropriate onboarding process is developed for that candidate. Just because they are on a success trajectory does not mean they should be ignored at onboarding.
Part of this process is to ensure that all question sets are asked during the behavioral interview. It’s a mistake to stop an interview midway through because the candidate has provided Success indicator stories to that point. It’s critical to ensure each candidate is evaluated across all behavioral characteristics. Missing one or two question sets can result in missing Failure–Noncoachable behaviors which lead to a bad hire. The exception is for a candidate who has a high level of Failure–Noncoachable indicators. If it’s clear that the candidate will not be suitable for any job role in your company, continuing on to the bitter end is not worthwhile.
Most candidates or, more precisely, most successful candidates, will have Success indicators, almost no Failure–Noncoachable indicators, and a noticeable level of Failure–Coachable indicators. After all, we are dealing with people. Nobody is perfect, and everybody can improve. The trap which we have discussed previously is to insist on candidates who only have Success indicators and no Failure indicators. This is an unrealistic approach which not only dismisses the reality of people but also anticipates an effortless onboarding process based on the presumption that the perfect candidate will immediately become the perfect employee.
Onboarding is a crucial process which builds relationships, enhances communications, and develops high-performance work units over time through consistent interaction across the enterprise. A mindset which seeks perfect new hires is one which also anticipates perfect performance without effort or communication, and which necessarily devolves into frustration and dysfunction when things don’t go perfectly—which is inevitable.
A correctly developed and executed behavioral interview will reveal the candidates who should be offered jobs. These candidates will be those who have measurable Success indicators, almost no Failure– Noncoachable indicators and sufficient Failure–Coachable indicators upon which to build an onboarding and new employee integration plan.
So Much Goes Into the Interview: Here’s a Summary
At this point, many of our clients become overwhelmed. They understand the preparation work that needs to be done before the behavioral interview, but the amount of work to be done after the interview is completed seems huge. And what about level setting? How do you ensure candidates who have been interviewed by different people in HR are treated equally? How can the hiring manager be assured all candidates have been evaluated the same? Let’s re-emphasize a crucial point we made earlier in the book. The hiring manager simply must participate in the behavioral interview of all candidates. Even if the hiring manager is just an observer, neither conducting the interview nor taking the notes, it is the hiring manager who must be present to hear the stories told by every candidate, to make the final evaluation, and then to select the candidate who will be offered the job. There simply is no way around this.
Remain aware that The Big Mistake lurks constantly. Avoid falling into the trap of hiring a candidate based on some form of, “I like this guy.” It really doesn’t matter if the candidate is likable or if you had great rapport during the interview—“likability” is simply not relevant. Stay focused on whether the candidate demonstrates a history of performing the behaviors you need for the job you’re filling and whether any deficits can be remedied through training. And never make the mistake of hiring someone because “they interviewed well.” You’re looking for someone who demonstrates a track record of ability and willingness to do the behaviors the job requires. Unless “interviewing well” is a part of your business model for which it makes sense for you to pay people, it’s not relevant.
Remember that if a candidate gets nervous or uncomfortable sharing what they consider to be private information with a stranger, that tells you something about how they behave. Working with a person requires understanding who that person is—how can you possibly work with somebody if you don’t know how they’re going to respond or react? If a candidate is not willing to share themselves in an interview, they’re not going to be willing to share themselves on the job. And if sharing yourself on the job is important because you will be working in a team environment, then that person may be better suited to the story of our delivery driver who is happiest alone, getting the job done, and not having to interact. In that case, you’re not going to be all that concerned about a candidate not wanting to share the details. It all comes back around to what behavioral characteristics you’re looking for. And all of the candidate’s responses in the behavioral interview are factors which give you insight into the behavioral characteristics which correlate to job performance at your company.
As you gain experience conducting behavioral interviews, you’ll get better at estimating the time you’ll need. Initially, plan on ten to twenty minutes to introduce the candidate to the process and ten to fifteen minutes per question thereafter. Depending on the behaviors and the job you’re filling, most basic behavioral interviews for entry-level jobs will go about two hours.
Much of the field of cybersecurity deals in unknowns. Whether you are talking threat vectors, zero-day exploits, rapidly evolving malware, new technological releases, and the latest requisite patches, there is always an element of the unknown. Gail Gottehrer refers to it as the gray zone:
Gail: It’s about being flexible enough to say, we work and live in an area that’s all in the gray zone. No matter how many hours I work, I will not be able to anticipate everything (even though I want to). Lawyers and people in the cybersecurity field like to know that if we work hard enough and spend enough hours preparing, we’ll know the question before it’s asked and have the answer to every question that’s asked. That doesn’t work in cyber. It’s an unattainable goal, so you need someone who can accept the fact that everything’s gray—the unknown is going to happen no matter what you do; you have to be comfortable with uncertainty, and comfortable telling senior management that you don’t have all the answers, but here’s your best plan given what you do know, with the understanding that when you start doing an investigation into an alleged cyber incident, you know very few facts, and that what you infer from the facts you have may turn out to be wrong.12
One of the important aspects of this “gray zone,” for many cybersecurity roles, is being able to be open about making mistakes. As Andrea Markstrom explains, it’s about accountability and growth:
Andrea: I learn every day, I make mistakes every day—it’s what we learn from it, and it’s being humble, it’s being open, it’s being accountable—those are all very important skill sets to me in order to be a good team player. And to be able to learn. Because in order for you—or somebody on your team—to grow, they have to be open to learning, and they have to be open to feedback. And they have to also not be afraid to say, you know what, this is what happened, this is a mistake I made, but this is what I learned, I’m accountable, and move on. So that level of accountability and that level of being able to be responsible about it is super important.13
As Wheeler Coleman explains, he seeks out detail when discussing how people handle crises:
Wheeler: I look at whether I get a general answer versus a specific answer. I want to know whether a person will open up to me and give me an honest account of when they lost “their hair.” For example, I want to know the details about a time when they lost their cool when somebody “pushed their buttons” the wrong way. I want to know exactly what happened, what they learned from that experience and how they reconciled the situation.14
They may reply that it’s too personal and they may be genuinely uncomfortable sharing. While the reply itself may be valid, what does it reveal? It indicates a person who is not open or trusting, perhaps someone who is insecure. This may indicate someone who may not form relationships quickly or who may have difficulty trusting others. None of these possibilities are disqualifying, but they do give insight into the candidate’s personality. And that helps you build an understanding of who the candidate is—which is what you need to know before a correct hiring decision can be made.
Some candidates may have difficulty thinking of a suitable story. This is an opportunity for the interviewed to engage in an open exchange with the candidate, a discussion about what the interviewer is looking for and a dialogue designed to assist the candidate to dig through memory to find a story which helps. These exchanges are useful because they often become structured conversations which show how the candidate is likely to interact with others when employed.
The type of story the candidate shares and how they characterize it can also reveal a good deal about the candidate’s personality. For example, a candidate whose “most stressful experience” was missing a connecting flight is likely to be a very different person from the candidate who shares the story of moving across the country for a year to give end-of-life care to a grandparent. Again, there is no correct answer. Rather, it’s the type of story and how it’s told which reveal characteristics of the candidate’s personality. Once you have an understanding of who the candidate is, correlating the personality to the success/failure factors of the job you need done becomes more straightforward. The key is to have proper question sets and to learn how to conduct the interview, so all relevant data are captured.
Ongoing analysis of a candidate’s answers must also be performed by the interviewer during the interview to ensure proper follow-up questions are asked and the candidate’s best possible stories are found. And a final analysis of the interview is necessary to properly structure differential data into Failure–Coachable, Failure–Noncoachable, and Success, to ensure an accurate hiring decision, and to inform the onboarding process.
1 A. Berglas, in discussion with the authors. June 30, 2020.
2 J. Kolb, in discussion with the authors. July 28, 2020.
3 A. Markstrom, in discussion with the authors. June 24, 2020.
4 A. Bricker, in discussion with the authors. July 13, 2020.
5 R. Petersen, in discussion with the authors. July 28, 2020.
6 A. Michaels, in discussion with the authors. July 03, 2020.
7 A. Tilley, in discussion with the authors. July 02, 2020.
8 A. Bricker, in discussion with the authors. July 13, 2020.
9 J. Kolb, in discussion with the authors. July 28, 2020.
10 M. Durst, in discussion with the authors. June 29, 2020.
11 M. Chudolij, in discussion with the authors. July 24, 2020.
12 G. Gottehrer, in discussion with the authors. July 09, 2020.
13 A. Markstrom, in discussion with the authors. June 24, 2020.
14 W. Coleman, in discussion with the authors. August 12, 2020.