Chapter 4. Organizational Research

Hell hath no fury like a bureaucrat scorned.

—Milton Friedman

You’re an individual with a goal. If you’re a designer, you probably want to create something new that delights other individuals and becomes personally important to them. Designs that change the world do so because millions, or billions, of individuals adopt them.

Design doesn’t happen in the deep, cold vacuum of space. Design happens in the warm, sweaty proximity of people with a lot on their minds. People create and join organizations to accomplish greater things more efficiently. As an organization grows, it becomes more complex. The oral culture of how to get things done begins to outstrip the documentation. Various internal groups might develop different perspectives on high-level goals, or even different goals entirely. Essential relationships develop that don’t map to any org chart.

A design project is a series of decisions, and making sure the right decisions get made can seem tricky in a complex organization. Take heart. You have more influence than you might think, as long as you take the opportunity to understand the inner workings.

The design process is inextricably bound with the nature of an organization. Budgets, approvals, timing, and resource availability can all depend on successfully negotiating an organization. The ultimate success of a product or service depends on how well it fits into everything else the organization is doing and how well the organization can and will support it.

The habits of organizations and the people within them can be powerful. You’ll be working directly with specific individuals, but your interactions will be more or less successful depending on your understanding of the organization as a whole. As much as managers talk about data, organizations are the social context in which decisions are made, and that context matters more than most want to admit.

Think of an organization as physical terrain. A small startup is like an island. It might spring up out of nowhere and sink down under the waves just as quickly, but for the duration of its existence, you can get a clear view of the landscape pretty fast. A large corporation is more like Australia: it’s impossible to see the whole landscape at once and there are so many things capable of maiming or killing you.

Fortunately, at any size, an organization is just a set of individuals and a set of rules, explicit and implicit. Once you understand that environment, you’ll be able to navigate it and create the best possible product.

Put an MBA Out of Work

Organizational research—determining what drives a business, how all the pieces work together, and the extent of its capacity for change—is traditionally the purview of business analysts. However, researching an organization is very similar to traditional user research and can be incredibly helpful to interactive design and development projects.

Many external agencies interview client stakeholders—people whose jobs or roles will be directly affected by the outcome of the project—as a part of the standard requirements-gathering process. Doing this is essential when you’re coming in cold to work with an unfamiliar organization.

Internal teams may have to do a bit of role-playing to gather the same information: “Talk to me about how you interact with other members of the marketing team as though I don’t work here and we’re speaking for the first time.”

In organizational research, the observer effect (how an observer’s presence can alter what is being observed) can actually be a force for positive change. Asking hard questions of people throughout an organization will force those people to come up with answers, leading to at least a modicum of reflection. Asking the same question of different people will reveal crucial differences in motivation and understanding. And listening to people who might not feel heard is a tremendous source of goodwill. Asking a lot of questions will also make you sound quite smart. Or get you fired. No guarantees.

If you are at a smaller, more nimble organization, such as a very early-stage or rapidly growing startup, the enemies aren’t complexity and stasis. Rather, you may have to contend with the desire to maintain momentum and “fail fast,” as well as pressure to avoid questioning the core assumptions that attracted funding in the first place.

To support “not failing at all, if we can avoid it,” identify the assumptions that pose the greatest risk and suggest activities to address those assumptions. The GV Library (http://bkaprt.com/jer2/04-01/) is your ally. The GV (formerly Google Ventures) team has assembled this collection of design and product articles specifically for startups, including the handy “Field Guide to UX Research for Startups” (http://bkaprt.com/jer2/04-02/).

As for how to go about organizational research, it’s pretty straightforward and adheres to the same principles discussed in Chapter 3. The major difference is that you’re talking to current stakeholders instead of potential users or customers.

Who Are Stakeholders?

The stakeholder concept emerged in a 1963 internal memorandum at the Stanford Research Institute (http://bkaprt.com/jer2/04-03/). It defined stakeholders as “those groups without whose support the organization would cease to exist.” Your research should include anyone without whose support your project will fail.

Be generous in your stakeholder selection. A few additional hours in conversation will help ensure you’re both well informed and protected from an overlooked stakeholder popping up too late. Include the following groups:

• Leaders will help you understand the overall company mission and vision and how your project fits into it.
• Managers will frequently be concerned with resource allocation and how your project affects their incentives, monetary or otherwise, and their ability to do the work.
• Subject matter experts will provide you with essential background information. You can find them by identifying those design-critical areas where you have the least background knowledge and by asking for introductions. The expertise you need may be outside the building.

Make sure you balance out the executive perspective with people who do the day-to-day work. In particular, find anyone who has knowledge of the end users. Customer-service representatives and salespeople can offer valuable insight.

It is essential to talk to a broad swath of the staff, making sure to cut across silos. Some of the most useful information is the degree to which goals and beliefs are shared or in conflict with one another in various disciplines and departments.

In some organizations, the board members are either influential or highly knowledgeable. In others, they are more removed and therefore less useful. Inquire about their level of interest or concern with the project before arranging a conversation. Especially at nonprofits, board members can be tremendous allies, or the other thing. You want them on your side before you find yourself presenting an exciting new direction to them, trust me.

Interviewing Stakeholders

Interviews with project stakeholders offer a rich source of insights into the collective mind of an organization. They can help you uncover areas of misalignment between a company’s documented strategy and the attitudes and day-to-day decision-making of stakeholders. They can also highlight issues that deserve special consideration due to their strategic importance to a business.

—Steve Baty, “Conducting Successful Interviews with Project Stakeholders” (http://bkaprt.com/jer2/04-04/)

The term stakeholder is a bad bit of jargon, but there really isn’t a better alternative. Think of stakeholders as people who could potentially put a sharp stick in your back unless you win them over. But don’t fear them! Stakeholder interviews—sitting down individually with people who will be directly affected by the project—have many benefits.

What stakeholder interviews are for

Hearing the same issues considered by people in different roles relative to your work will give you great insights and a more complete perspective. Some individual interviews are valuable on their own, and some higher-level insights are only possible in the aggregate.

Stakeholder interviews will help you understand the essential structure of the organization, how your work fits into the organization as a whole, and the approval process for various aspects of your project. They’ll also provide you with some less obvious opportunities to influence your project’s chances of success.

Neutralizing politics

Organizational politics are a depressing reality in most companies. If you pretend they don’t exist, you’ll be in for a world of pain. A significant benefit of organizational research is political. You don’t want your hard work to get trampled in a turf war you didn’t know existed.

You may find that someone in the organization is deeply opposed to your work. If you know why, you may be able to get them on your side. Talking with stakeholders is an excellent opportunity to sell people on the value of your work in terms that matter to them.

Better requirements gathering

Business requirements are frequently defined as though the project were taking place in a frictionless ideal state, but the product or service you’re developing doesn’t exist in a vacuum. You have your own reasons for wanting to build or design it in a certain way. Similarly, you need to understand how your work might solve or create problems throughout the organization, and how the organization will prioritize those problems and solutions.

A list of business requirements should include the rationale and objectives from the perspective of every group in the organization whose work will be affected by the project at hand. Creating this list requires thorough internal discovery and clear definition.

Don’t forget to inquire into technical requirements and constraints. Many beautiful visions crash into functionality and performance issues that might have been anticipated just by talking to the right people early.

Understanding organizational priorities

How important is the work to the organization, really? The answer might surprise you. It makes a big difference whether the project at hand is genuinely valued by the organization. Many years of design consulting have taught me this: the more important a project is to an organization, the more successful it will be. The stress level might be higher among people working on an absolutely critical, number-one-priority project, but they will be giving it their full attention.

Tailoring the design process

“Let’s not reinvent the wheel” is a popular cliché, but make sure you’re using the right tires for the terrain. During interviews, ask about the typical workday as well as how decisions are made within the team and the organization. This is especially critical if the project at hand brings together cross-functional teams, or teams who have never worked together before, or a new set of external partners. Since the design team might be in a place to define the decision-making structure that everyone has to follow, your life will be a lot easier if you adapt your process to existing work styles rather than try to change ingrained habits. Your project manager will thank you.

Getting buy-in from stakeholders

For the definitive word on making influential people feel heard, I encourage you to read Paul Ford’s excellent essay “The Web Is a Customer Service Medium” (http://bkaprt.com/jer2/04-05/). Here is the heart of it:

“Why wasn’t I consulted,” which I abbreviate as WWIC, is the fundamental question of the web. It is the rule from which other rules are derived. Humans have a fundamental need to be consulted, engaged, to exercise their knowledge (and thus power).

Asking someone for input before you get started is a peerless prophylactic against that person rearing up late in the game with insurmountable objections. Inquiry is flattery. Inviting people to participate empowers them.

Take it from internet trolls. Never underestimate the ability of a single individual—no matter how seemingly unimportant or obscure—to really fuck things up for you once they set their mind to it.

What are your assumptions about how the organization functions, about how different disciplines interact, about what the workflow is and how well it’s working, about how much people know or need to know about what you’re doing? Now think of the worst-case scenario if you’re wrong. What happens if marketing doesn’t understand how your work supports the brand, if the salespeople can’t see the benefits, if the production team has no incentive to give you any of their time? This is your opportunity to educate as well as listen, and to get everyone on board.

Understanding how your work affects the organization

If you are creating something new, the very existence of the new system will require everyone in the organization to change. Similarly, those people will affect what you’re creating. Even if you’re the sole author of an application, you require the participation of others for it to succeed.

You’ll benefit from learning the perspectives and priorities of everyone who will be affected, even those who don’t directly use the product, service, or system you’re designing. Executives will have to defend it as a part of the overall strategy. Customer service will have to support it. Salespeople will have to sell it. Production staff will have to maintain it. Founders may be using it as a proof of concept to raise more capital from investors. Company representatives might expect to field questions about it when they’re out at conferences.

Don’t wait for people inside the organization to come to you, and don’t rely on a higher-up to tell you whom to talk to. It’s up to you to evaluate the project’s goals and then determine whose input you need.

You can identify which people or departments will have to put in the time, effort, money, and other resources to cope with the changes. You can learn whether enough resources will be available or whether the organization will need to buy more servers or hire more writers.

Understanding how what you propose to build relates to the organization responsible for it means that you can anticipate changes to workflow and minimize them where possible, or prepare people for changes when they’re necessary.

Once you inform the organization how much work it will take to accommodate your project, you’ll find out whether you in fact lack the organizational support you need and thought you had. Then you can make decisions based on that knowledge, rather than have good work wither on the vine, neglected.

Sharpen your tact

Build a wide list of people to interview: founders, executives, managers, technical staff, customer service, and so on. Then prioritize it. In addition to people who are directly valuable to the project, you will likely have to speak with some for purely political reasons. This is also an opportunity for learning.

The maximum number of interviewees is the number you actually have time to talk to. In some large organizations where the project touches on many types of expertise, you might find yourself on an exciting voyage of discovery, talking to dozens of people. Have a firm idea of how much time you have and stick to it.

Once you have your list of people, find out as much about them as possible, just as you would in preparing for a job interview. Use the information you find to inform your line of discussion, but avoid leading with any tidbit your subject would not expect and welcome to be common knowledge. “So, you transferred to this department because you were passed over for a promotion...” will not make you any friends.

Individual interviews

As a rule, and as time permits, it’s best to interview stakeholders individually. The more political an organization, the more important private conversations are to get an accurate picture. You may have to fight a manager who “just wants to sit in.” This sentiment indicates some combination of fear and curiosity—fear that you’ll be gossiping behind that person’s back, and curiosity about what will be said. Explain the observer effect and hold your ground. This is non-negotiable if you want to gather true, useful facts. You’ll need to assure the interviewee that their answers will not be directly attributed, and assure the interested parties that they will get the information they need in an aggregated report.

Group interviews

If there’s a group of people of roughly equal influence who work together closely and share the benefits and risks of the project at hand, you may save time by talking to them together. During the discussion, take care to note whether anyone seems particularly reticent. Follow up with that person with a quick note to give them an additional opportunity to give you information.

Email interviews

For a stakeholder who is remote or impossible to get time with, try for a quick video chat or voice call. In a pinch, it’s better to send a few key questions via email than not get any information from them at all.

Interview structure

Each interview should last from thirty minutes to an hour. Make sure to talk in a private place.

The interviewer should be a calm and confident person, preferably someone who is genuinely very interested in what these people have to say. The conversation should flow naturally. If you don’t quite understand something, ask for clarification, or ask the subject to repeat what they said.

Have someone else taking notes so the interviewer can focus on the conversation. If you need to record the conversation, ask the interviewee’s consent first; recording may make the subject more nervous about speaking freely. The most important thing is for them to feel comfortable talking honestly and openly.

Put the participant at ease and demonstrate respect for their time. Send an agenda and the key questions ahead—not all the questions, but the ones the participant will benefit from knowing in advance. More complex topics might require some forethought. It’s best to avoid making people feel ambushed or unprepared.

The basic flow of a stakeholder interview is as follows:

• Introduce yourself and restate the purpose of the meeting. It should be something like: “We’re starting to work on a complete redesign of our website and we want to get your input. We’ll use it to make sure that the design meets your needs as well as those of other visitors.”
• Explain to what extent the information will be shared, by role or business function. “Please feel free to be totally frank. Honest answers are essential to this process. We’re talking to people throughout the organization, and will group answers together rather than focus on what one person said. If we use a direct quote, we will not attribute it to you personally.”
• Like a good journalist, don’t narc on your sources. Get something in writing from the person directing or approving this research, stating that people can speak freely without fear of reprisal.

Ask questions and let the conversation follow its natural course. It’s very important to keep the interview feeling informal. It’s not an interrogation.

At the end of the interview, restate how you’ll use the information and verify the level of the interviewee’s participation throughout the project. You definitely want to make sure that your expectations match. Make sure it’s okay to follow up if you need more information or clarification.

In addition to name and title, these are the basic questions you’ll want to ask:

• How long have you been in this role?
• What are your essential duties and responsibilities?
• What does a typical day look like?
• Which people and teams do you work most closely with? How well is that relationship working?
• Regarding the project we’re working on, how would you define success? From your perspective, what will have changed for the better once it’s complete?
• Do you have any concerns about this project?
• What do you think the greatest challenges to success are? Internal and external?
• How do you expect your interactions with other people inside or outside this organization will change based on the outcome of this project?

Then, there are the more specific questions that depend on the project. Stakeholders may themselves be users, often of backend systems or administrative functions:

• What are your most common tasks with the system?
• What problems have you noticed?
• What kinds of work-arounds do you use?
• Do you have any concerns about this project?
• Is there anyone else I should talk to?

Dealing with a hostile witness

It’s in the name—stakeholders have a personal stake in the process or outcome of the project. They may be in competition for resources, or they may wind up with a larger or smaller workload if the project is successful.

Stakeholder interviews tend to be interesting when they go well. People enjoy being consulted and treated as experts. However, sometimes stakeholder interviews take a turn for the ugly. This can be very unpleasant, particularly when you’re interviewing in person. The participant you’re interviewing will turn the tables and start attacking the process, or you personally. They may start questioning the value of what you’re doing or even say they don’t understand the questions you’re asking.

If this happens, remain calm, take a deep breath, and attempt to get the interview back on track. Restate your goal, and then try asking a general, open-ended question about what the participant thinks is most important for you to know in the service of this goal. Depending on the reason for the hostility, you may just want to cut the interview short.

Common reasons for stakeholder resistance or hostility:

• The stakeholder wasn’t sufficiently informed or prepared for the process and is suspicious of the motives, or just confused about why they were asked to participate.
• It’s a power move. This individual wants to establish dominance over you or, by extension, the person who authorized the interview or the project as a whole.
• The stakeholder is under pressure to perform in some other area and doesn’t see a benefit from participating. This is common when interviewing salespeople who think they are wasting precious time when they could be selling and earning commissions. You have taken them “off the floor.”

Try to determine in advance whether any of the stakeholders you plan to interview are at risk for a hostile reaction. Make sure they know why you’re asking them to participate, how they need to prepare, how long it will take, and the reasons why their participation is essential to the process. Flattery usually goes a long way.

Remaining calm and confident is essential. Never let anyone bully you when you’re gathering information that’s essential to your work. Make sure you’re prepared to describe the process and justify its value.

Never let anyone take control of the interview from you. Listening to someone go on a rant about what isn’t working can be interesting and useful, but it’s up to you to guide the conversation.

Practice, practice, practice. If you’re new to doing these sorts of interviews, practice with members of your team before doing it for real. Have them throw in some challenging or unproductive responses:

• “Why are you asking me this?”
• “I don’t understand that question. It doesn’t make any sense.”
• “I don’t feel comfortable talking to you about that.”
• “No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.”
• “How much more time is this going to take?”

Documenting interviews

For each stakeholder, note the following:

• What’s their general attitude toward this project?
• What’s the goal as they describe it?
• To what extent are this person’s incentives aligned with the project’s success?
• How much and what type of influence do they have?
• Who else do they communicate with on a regular basis?
• To what extent does this stakeholder need to participate throughout the project, and in which role?
• Is what you heard from this stakeholder in harmony or in conflict with what you’ve heard from others throughout the organization?

Just enough

You’ve interviewed enough people when you feel confident that you know:

• who all the stakeholders are
• their roles, attitudes, and perspectives
• their levels of influence, interest, and availability over the course of the project
• how they stand to benefit or suffer with the success or failure of your work
• the likelihood that any of them have the power or potential to prevent project success
• all the ways the workflow will have to change to make your project a success
• the resources you have available for your project process
• the resources required to support your project once it’s complete
• all the business requirements and constraints
• whether your team and core stakeholders agree on the goals and definition of success
• whether the stated goals are the real shared goals, or whether anyone has a hidden agenda
• how people outside the project team view this project

Once you’re done making new friends, it’s time to reflect on what you’ve learned and integrate all the individual perspectives into a cohesive narrative.

Stakeholder Analysis

Stakeholder analysis itself is often straightforward, while sharing what you’ve learned takes savvy and finesse. (If you’re interviewing members of the organization as users of the system, refer to the ethnographic methods in Chapter 5.) Create a clear statement of what you need to accomplish in order for the organization to consider the project a success. Look across disciplines and departments for themes that contribute to a unified narrative. And, flag any challenges or questions for further study.

Depending on the political situation at the company for which you’re conducting the research, you may have one version for the core team and a more succinct (or polite) report for broader distribution.

Key points

Your goal is to incorporate the specific concerns of everyone you spoke with into a shared understanding of the organization, its priorities, and the path forward. Easy! If you do this well, it provides the foundation for strong decision-making and collaboration.

Problem statement and assumptions

What needs to be solved or improved from a business perspective?

Goals

Every project begins with a rough set of goals, or concepts of success. Every individual in an organization sees them a little bit differently. Gathering these and reconciling them is essential to a functioning project.

Success metrics

What are the qualitative and quantitative measurements that tell you whether the project is hitting the mark? These should be meaningful criteria for evaluation that support the goals. Often, you will discover that part of the work to come is defining metrics. That is okay, and far better than picking the wrong ones because it’s easy.

Completion criteria

How will you know you’re done? It may seem obvious, but it’s always good to validate. Otherwise, the project will never be finished!

Scope

Scope refers to the amount of work included in any project. “Scope creep” is what happens when more work just keeps getting tacked on and the scope grows uncontrollably. The best way to avoid scope creep is to document what is included in as much detail as possible and in language everyone understands. And note who is responsible for what. Scope is a boundary, so it’s also very useful to note that which any of the stakeholders might assume to be included but is out of scope. Not touching the branding this time around? Write it down. Detailed scope documentation makes for happy teams and functional projects.

Risks, concerns, and contingency plans

Want to increase your chances of project success? Then acknowledge anything you’ve uncovered that might lead to failure or unmet expectations.

A designer conducting organizational research might pick up on a lot of information that matters to the project process as well as to the design approach. Some organizations are more functional and well-resourced than others. Maybe key decision-makers will have limited availability. Or perhaps two departments who need to collaborate very closely have historically had a poor working relationship. Every organization has its challenges. If the team understands and acknowledges these, they will be able to work around them more effectively.

All of this information gathering will allow you to anticipate potential problems before they arise. This is an area in which the practitioners (designers, writers, developers, and project managers should collaborate closely. If these challenges are not openly acknowledged (which sometimes happens), be very sensitive in how you talk about them. For your work to succeed, you have to address them.

Getting everything done on a tight schedule is often a major shared concern. A clear, simple—and, most important—publicly documented process for gathering feedback and making decisions helps everyone stay on track. And, if you have heard different concerns from different groups, it’s best to address that head-on. The need to keep the total project cost down might be what you heard from operations, while the product team mentioned the need to have a user experience that compares favorably to a major competitor. The most appropriate solution will address both.

Verbatim quotes

The specific words used are highly valuable in revealing an individual’s perspective and attitudes. Quotes that represent the perspectives of research participants are often the most powerful, portable output of user research. If possible, omit identifying information when sharing quotes.

Workflow diagrams

Who will need to be told about how things have changed, and in what format? A workflow diagram is a good companion to this document (Fig 4).

If you’re working on an internal project or a new customer-facing product that’s likely to change internal workflow, diagram the current and proposed workflows. Throughout the project, you can use these diagrams to track any workflow ramifications and make sure the organization is changing sufficiently to accommodate the new design.

Unpack the Baggage

A solid understanding and honest assessment of an organization and its business is necessary for the success of any significant design project. Organizational habits and capabilities are just as relevant as target-user behaviors and needs, although they’re less frequently included as fundamental topics of design research. And the true nature of workflow and interpersonal relationships is just as ripe for ethnographic exploration.

Even just the process of conducting research can be beneficial, if only because it provides the motivation to open atrophied or nonexistent communication channels. Performed with tact and rigor, organizational research can neutralize politics, clarify requirements, and improve the odds that changes will be fully understood and take hold.