Chapter 33. Immersive Interviewing and Hiring

Your new hire, the one you spent weeks recruiting and screening, has turned out to be a bad fit. You’re now in a tight spot and have to decide whether to cut your losses or invest in fixing the problem. Meanwhile, the work you hired the person to do is piling up and your project is suffering.

The good news is that you can avoid this situation if you screen for the correct things when hiring. The story that follows illustrates an immersive interviewing style that can help ensure you hire the right people for the job.

Prior to partnering with Janeen, Scott had worked at a large software company, where he felt the hiring policies got in the way of finding the best people. He wanted to establish better procedures at their company while it was still small enough to do so. He brought up his ideas during a meeting with Janeen. She was not immediately convinced.

“Scott, I’m not feeling the same urgency you are about revamping our hiring process. We’ve built a great company by hiring people this way. Why fix what isn’t broken?” Janeen responded.

“I agree that we’ve had success so far,” replied Scott. “As we grow, though, you and I won’t be able to personally screen every new hire. We need to have a scalable model that builds on what we value as a company. Think about it. Even with you and me personally screening people, we’ve ended up with a few bad hires. Remember Victor? That guy was awful when it came to working with people. Sure, he knew his stuff, but he made everyone around him miserable.”

“Ugh. I remember. He just never made a connection with the team. But when we gave him something to do that didn’t require collaboration, he was focused and we got a lot out of him. We hired him for his skills, not his personality,” Janeen reminded Scott.

“I agree, although even when he was focused, his attitude toward people was awful. I’d rather have a team player who can work with others and have half the skill of Victor than go through that again,” explained Scott.

Janeen nodded. “True. And what’s worse is that bad hires tend to snowball into other losses. Which reminds me, did I ever tell you that Victor was the real reason we lost Mark?”

“Nooooo!” exclaimed Scott in sudden realization.

“Yep, he left because of Victor. He didn’t tell most people Victor was the reason, but he did tell me. He said life was too short to deal with people like that,” explained Janeen.

“That was a hard loss. We’re still feeling the impact from that,” Scott said with a sigh.

“I know. It was. But the thing is, Scott, we get it right nine out of ten times. I’m not sure we can do better than that. We’re going to make mistakes,” Janeen replied.

“True, but remember, even if we accept that we’re going to miss sometimes, we’re still going to need to scale what we are currently doing. Right? I mean, we’re going to have to make changes eventually anyway. I’d like to try something new while you and I can still be involved in the process, so we can personally see whether it works or not. Now, I’ve been giving this a lot of thought, and I think I have a way to hire that will screen for the pertinent items and is scalable,” Scott said enthusiastically. “Want to hear about it?

“I’m intrigued,” said Janeen. “Go on.”

“We need to start with what we value as a company,” Scott explained as he wrote the company values out on the whiteboard.

Scott continued, “Right now, as you said, we’re focused on every candidate’s technical capabilities. And we assume that because we’re hiring mostly senior-level people, they have a similar set of values. But we don’t have any way to screen for them—nothing we do helps us determine if a candidate shares our company values.”

“True, true,” agreed Janeen.

Scott continued, “I’m convinced we need to look at competencies and values as much as we look at technical skills. Technical skills can be learned in months, but the ability to work well as part of a team is something that takes a very long time.”

Janeen nodded in agreement. “Okay, so what do you propose?”

“Well, I think we should take some lessons from how the teams have been pairing and build on that.”

“Team interviews, then,” Janeen said thoughtfully. “So, you’re suggesting that the candidates pair with existing team members—do immersive interviewing. I like the idea in concept, but how would we standardize that—keep it objective and make sure we’re being consistent from person to person?”

“First, I think we would need to spend some time training the team members about the process itself and how the interview is a kind of forecast. We are forecasting whether the person will be a good fit and how long it’ll take that person to become a contributing team member,” said Scott.

“Yes, and we’d need to show them the things we need to consider when making that forecast: the competencies, skills, and the level of the role we’re hiring for—” added Janeen.

Scott interrupted, “The level?”

“Yes. Do we need someone junior-level, senior-level—or is either one acceptable depending on the candidate? The level of maturity and expertise we’ll expect will be different depending on the position we’re hiring for.”

Scott nodded and added, “Good point. And we’d need to understand and communicate why we’re hiring. Are there organizational issues that are blocking progress? Is the skillset the company is lacking one that could be filled by a team consultant, or do we truly need a new core team member? Does the team need someone in a learning role or a leadership role? The answers to these questions directly affect who we’ll hire.”

“Last but by no means least, we need a way to make these determinations in a repeatable, consistent way across the company,” added Janeen.

“Yes. Absolutely. We are inconsistent with our approach, and we treat it more like an individual burden for an hour or two on any given day than the team event it should be,” agreed Scott.

“You’ve sold me, although I have an idea for how to move forward. We’ve got that one team who’s been clamoring for a new member. Maybe we sit down with the existing team members, explain our goals, and work with them to create a preliminary model? Then we run a trial with that team—they can be the first to hire someone using the new process,” suggested Janeen.

“I like it,” said Scott enthusiastically. “Let’s get started!”

However the interview is structured, most organizations fail to ask the most important questions, not of the candidates but of themselves:

1. Why are we hiring? What problems are we really trying to solve?

2. Should we even be hiring? Is there potentially an impediment inside the company that is going unresolved, and a new hire is only the mask?

3. What are the most important skills and competencies to consider?

4. How do we decide who to hire?

However, before delving into these questions, I want to begin with the definition of an interview that Scott puts forth in this chapter’s story: a forecast.

Brooks’ law states that “adding manpower to a late software project makes it later” [BROOKS]. One of the reasons this happens is that some companies hire people to mask or cover up larger issues, when what they really need to do is solve the underlying problem. For example, complaints of not enough testers is often a sign of poor test automation. Reports of not enough developers can sometimes point to a lack of collaborative teams. I’ve even seen people hire additional project managers to bring a project in on time when the real problem is lack of team commitment, goals, and business alignment.

Before creating a job description or making an existing one public, make sure that a new hire addresses the real problem you are trying to fix. It doesn’t matter who you hire, good or bad, if you don’t understand the reason you’re hiring in the first place.

Remember Victor from this chapter’s story? Victor was classified as a bad hire because he could not work well with other employees. To make matters worse, his conduct caused one of the good hires to leave the company. It’s no wonder company culture suffers when the wrong people are hired!

Let’s look at the factors to consider when hiring that might help you avoid these costly disasters.

Let’s reverse that, then, and imagine we are looking for a very experienced C# programmer but all of the candidates are uncomfortable working in pairs. This means they may not want to improve or branch out from C#, resulting in lost opportunities for learning new languages. Further, they may not enjoy mentoring others or pairing, meaning no one else benefits from their extensive skillset. No matter how senior or experienced the candidates are, they would not be a good fit on any agile team, because of the misalignment of values.

Based on those two scenarios, screening candidates should always include competencies—how they think, how they interact with others, and what they value. These will impact how quickly they can assimilate with a team and how rapidly they can learn new skills. After all, you might need these same people to learn yet another new language a few projects down the road. In general, you should hire for long-term fit, not for short-term expertise.

Scott and Janeen wanted to find an interview style and question set that was repeatable, consistent, and helped screen for the company values. What they decided to do was to apply agile techniques to hiring—more specifically, pairing.

Immersive interviewing is a hiring and forecasting technique that relies on screening rather than a series of questions. It shares some characteristics with a technique from Menlo Innovations called Extreme Interviewing [MENLO] but is not nearly as scripted. In immersive interviewing, teams have a list of items to validate with candidates. Are the candidates a good cultural fit? Are they able to give and accept criticism? Are they comfortable with conflict? These questions are best answered through immersion.

Recall that an interview is a forecast and how important it is to make an accurate one when hiring. Remember, too, that in the Agile Manifesto, the preference is for individuals and interactions, working software, and responding to change. Why? They all lead to success. Agile hiring should be no different. You need to see how a candidate works, thinks, and behaves in the real world—the metaphorical equivalent of working software. You want to know how a candidate will interact with the other members of the team. And you want to understand whether the candidate is comfortable in an environment of feedback, continuous improvement, and change.

When it comes down to it, traditional approaches to hiring only demonstrate how well the candidate can answer the questions of the interview. While those answers might be an honest reflection of a candidate’s ability and beliefs, they are analogous to an artificial shell, showing the intended functionality of a software feature, without anything backing it. I don’t know about you, but I always feel much better about my hiring forecast if I interact with the person in real-world scenarios, with all team members. Research confirms that I have a good reason to feel this way. In a 2002 white paper, Menlo cites that out of 20 people hired through their Extreme Interviewing method, they lost only one through attrition [MENLO]. My experience has been similar. Companies that I am working with today report much lower attrition levels for team members that are hired with an immersive interview approach as opposed to a standard interview approach.

The potential for a more accurate forecast is why teams I’ve worked with prefer to immerse candidates in one or more days in the life of a project team. The hiring manager or an HR person works with the team to identify what competencies and skills the candidate should display. The team asks each candidate to pair with team members in the team space. The team members then look for evidence of the necessary attributes while working with the candidate. At the same time, the candidate is encouraged to write code, test, and document—to act as a team member for the day. Candidates should be invited to the daily standup or the demo or retrospective if the timing works out.

The point of the immersive approach is to put candidates in the environment in which they will be expected to perform. This enables team members to evaluate the candidates before reaching consensus on which candidate would be the best fit, and it enables candidates to evaluate whether the company is a true fit for them as well.

During an immersive interview, it’s easy to gauge a candidate’s skills and experience. Coding not up to snuff? It’s easy to see. Trouble with test-driven development (TDD)? That’s obvious, too. There’s no hiding when your hands are on the keyboard and someone is watching how you work. Competencies might seem a little harder to judge, but they really aren’t. It’s just a matter of learning what to look for and comment on the competencies a person exhibits while working with you.

For example, let’s say Scott and Janeen’s company is hiring a new candidate for a five-person team. The candidate will be scheduled to work in a pair with each member of the team throughout the day. During that time, the team members are not going to be asking typical interview questions such as, “Tell me about a time you demonstrated a willingness to learn” or “When making mistakes, what do you do to take accountability for them?” That would be a waste of time. Instead, the team members will be able to witness firsthand whether or not the candidate seems willing to learn and will observe what happens when the candidate makes a mistake.

Also keep in mind the difference between hiring senior-level people and junior-level people. When interviewing senior people, you would expect a certain level of maturity, expertise, professionalism, and insight. The primary focus of the screen, then, is to expose moments of unprofessionalism, discomfort when working as part of a team or with others, and similar signs of immaturity. You’ll also have a chance to see their technical expertise, including the way they approach problems and their openness to feedback, during the pairing sessions. I find that most high-performing companies would rather teach knowledgeable and experienced people new technical skills than try to change their mindsets, no matter how talented they might be.

On the other hand, if you are interviewing junior people, perhaps fresh out of school, you should be more forgiving of lack of expertise. What you would want to see technically is a basic level of competence coupled with a willingness to learn, ability to communicate, and excellent problem-solving skills. While you would generally expect more immature individuals, you will still need to make sure they are able to accept feedback and work as part of a team.

That being said, remember that there is actually some advantage to the lack of experience found with junior people. Experience can sometimes blind people and force people into ruts. Of course, senior-level people can also be open-minded and attack problems without letting experience getting in the way. That’s why it is so important to hire individuals who possess the willingness to learn, the ability to teach, the openness to share knowledge, and the desire to help create a learning organization—no matter their experience level.

To collect all of the data from the various interviewers, use an online table, such as the one shown in Table 33-2, and a scale of 1 through 10, where 1 is the least often observed skill or competency and 10 is the most often observed. These are subjective numbers. It’s a good idea to discuss what each value means so that the team has a relatively consistent understanding of what constitutes a 9 as compared to a 2.

As the candidate leaves one team member and moves to the next, the interviewer should take a moment to jot down scores that correspond to how the candidate performed. The chart will not be completely filled in at the end of the interview, but you will have enough data to have relevant discussions to determine if the candidate should be hired or not.

Think of this like Planning Poker but not in real time. Each team member shares his or her own estimate of the candidate’s fit in each of the key areas. Then, once the interview ends, the team gets together to have the all-important conversation. If there are outliers, such as Janeen’s rating of 2 for C++ in Table 33-2, the team should talk about the reasons for the extreme difference of opinion and try to reach a consensus. If all the ratings are similar, such as the ones for Architecture and design and Doesn’t have a sense of entitlement, the discussion will be more about general observations than reaching consensus.

Regardless of whether the numbers match or are wildly different, the point of the table is to serve as a real-time data collector for future conversations. Do not, under any circumstances, use the table to make a hire-or-no-hire decision without also having an in-depth team discussion. Otherwise, you might miss something vital.

When using immersive interviewing to hire nontechnical people, ensure you have a well-defined set of competencies that are understood by all. An immersive experience for a nontechnical hire would include attending meetings, participating in brainstorm or creative sessions, or maybe even doing a small portion of a presentation. Yes, there are risks here, but remember, we are looking for an immersive environment that will help everyone arrive at an accurate forecast of each candidate’s competencies and skills.

Hiring people for any role—manager, ScrumMaster, product owner, team member, or architect—is challenging. Hiring mistakes negatively impact the values and/or culture of the company. To help mitigate risks, aim to build a repeatable process that focuses on finding strong people with certain key competencies, developing skills when necessary, and making smart investments.

A repeatable process is also rooted in the company values and principles. The larger a company gets, however, the harder it may become to ensure that new hires align with the company values and principles. Consequently, it’s a good idea to hold regular “How to hire at our company” training sessions to help keep all teams fresh and up to date. This helps prevent bad hires and also refreshes and refocuses current employees on the company culture and, more important, the company values.

Finally, a repeatable process is one in which everyone understands what the rating system means, and that, as with Planning Poker, the initial ratings are meant to be the basis for discussion, not a standalone measure of a candidate’s worth. The value of this kind of immersive interview comes from the conversations—those that happen among the team members and those that happen with the candidate. Immersive interviews have the added benefit of ensuring that people, once hired, are familiar with the company’s hiring process, as they have gone through it, which adds to its repeatability.

Skills, on the other hand, can be taught rather quickly, especially given that most, if not all, of your hires will have some sort of background in the hiring area. For example, perhaps the candidate knows Oracle systems and you are hiring for a Microsoft stack—the candidate won’t have to overcome a childhood fear or a deeply seated habit to learn Microsoft. For someone with a technical background, this skill would be relatively easy to learn. Of course, that assumes that the person has the competency of willingness to learn. Having the ability and courage to say, “I don’t know how to do that, will you please help me learn?” is a competency that takes years to master and can only be practiced safely in a culture that supports it.

See Chapter 3, “Using Team Consultants to Optimize Team Performance,” for more about competencies and skills.

[CAREERBUILDER 01] Career Builder.com. 2011. “True Cost of a Bad Hire.” http://thehiringsite.careerbuilder.com/2011/12/16/true-cost-of-a-bad-hire/ (accessed 1 August 2015).

[CAREERBUILDER 02] CareerBuilder.com. 2012. “What Bad Hires Really Cost Companies (Hint: It’s More than Just Morale).” http://thehiringsite.careerbuilder.com/2012/12/13/cost-of-a-bad-hire/ (accessed 1 August 2015).

[MENLO] Menlo Innovations. 2002. “Extreme Interviewing.” Menlo Institute, LLC. http://www.menloinnovations.com/by-reading/PDF/Extreme-Interviewing-Final.pdf (accessed 1 September 2015).

[VALVE] Valve Software. 2012. Handbook for New Employees. Bellevue, WA: Valve Press. http://www.valvesoftware.com/company/Valve_Handbook_LowRes.pdf (accessed 1 August 2015).