Chapter 11

Instructing the Crowd

In This Chapter

Writing a clear statement of work

Putting together easy-to-follow instructions

Taking feedback on board

When it comes to management and communication, crowdsourcing requires a careful and thorough approach. Unlike on the traditional shop floor or in an office complex, all your interaction with the crowd passes through information technology, and the process strips away lots of useful knowledge. You can’t stop at someone’s desk to see how she’s doing or observe a work team making its plans. You have to gather all your information and give all your direction through technological means.

Crowds may be intelligent, but they don’t always have a direction. And when they do have a direction, they may not be going down the right path for you. Crowdsourcing isn’t magical. You can’t simply make some vague statement of a goal and expect the crowd to find an elegant and useful solution. From time to time, someone makes a brief communication to a crowd and wonderful things happen. Such events, however, are rare.

So to guide the crowd well, you need to master the skill of communicating with words, of using structure to direct activity, and of supplementing instructions with graphs and pictures. Communication works both ways, and you’ve also got to get good at hunting down information to see how the crowd is working, how it’s engaging with your task, and how you may need to adjust your plans. This chapter gives you a good grounding in managing and communicating with the crowd so you get the best possible results.

Preparing the Fundamental Message: Writing a Statement of Work

The first communication that you generally have with the crowd is the statement of work, in which you tell the crowd what you intend to do and how you’ll to involve it. The statement explains the goal and is the rallying point for the crowd.

The statement takes different forms in each version of crowdsourcing, as shown in Table 11-1. For more details on running each kind of crowdsourcing project, visit the chapter indicated in the Type column.

Table 11-1 Statements of Work in Different Types of Crowdsourcing

Type	Name of Statement	Explains
Crowdcontests (see Chapter 5)	Statement of contest goal	The goal of the contest and how any individual can contribute to the effort
Crowdfunding (see Chapter 6)	Statement of project, case statement or prospectus	What is to be done, how much money is required, how that money is to be used, and the benefits of the project (make sure that you stress these!)
Macrotasking (see Chapter 7)	Statement of work	The task to be done (and nothing more)
Microtasking (see Chapter 8)	Statement of tasks big and small	The overall project (in one document) and the kind of tasks that individual workers will do (in another document)
Innovation crowdsourcing (see Chapter 18)	Statement of problem	The problem that is to be solved

Structuring carefully

A good statement of work shares a common feature with a good joke: a sensible structure. In a joke, a bad structure can completely undermine a funny situation. All the humour, absurdity, wit and cleverness can be completely undermined by a structure that presents the elements of a joke in a way that defuses the punch line. In a statement of work, you can undermine all the ideas and vision with a chaotic structure.

In statement of work, the ideas should generally move from big to small. At each step of the way, the reader should understand how the new ideas fit into the structure that’s already been presented. In many cases, the statement starts with an overview of work and then moves into details. Only when asking the crowd to donate its time or resources do you begin the statement of work with the benefits of the proposed project.

This following sections lay down a common structure for a statement of work; follow them chronologically.

Stating the purpose: Explaining what the work is going to do

The statement of work should generally begin by answering the question: ‘What is this project going to do?’ (However, if you’re planning a charitable crowdsourcing activity, see the next section, Presenting the benefits, benefits, benefits).

Make the purpose section succinct, rarely longer than a paragraph. In many cases, you can summarise in a single sentence, which is often all the space you have on a crowdsourcing website.

Here are a few examples of purpose statements:

This project is creating a web page and store for a small firm selling personal care products.

The workers in this project will be transcribing public health records from non-English-speaking countries.

This project is identifying new products that will extend the market for our company.

Presenting the benefits, benefits, benefits

In crowdfunding cases, you present the benefits of the project to the prospective donors. These benefits should be both general and personal. They should give the benefits both to the larger world and to the individual who’s making the donation. Here are some examples:

Our project will improve literacy rates in South America and give you an opportunity to be part of a project that helps bring development and stability to an overlooked region.

The funds will be used to improve safety at a busy intersection and create an elevated bridge that will protect both pedestrians and drivers.

By contributing to our restaurant, you will allow us to meet a need in our community. You will also get a donor card that will give you far more in benefits at the restaurant than the value of your contribution.

Be honest in stating the benefits of your project. Make sure that you don’t inflate the benefits beyond what your audience will find credible. Not every project can save the world. Not every project can end disease. Not every project can bring peace in our time. The crowd can often detect when you’re exaggerating and will not engage with your project.

Making sure that they know the requirements

Start your description by identifying the required elements that are needed for the project to be complete. If a member of the crowd isn’t interested in or capable of doing these tasks, she can skip to another project. Here’s what requirements may look like:

This project involves creating an annual report that includes pictures portraying company activities, charts showing the firm’s economic health, and text describing the plans of each division.

This project will create 200-word descriptions of our products. Each description will correspond to a picture. Each description has to include physical details of the object and its brand name.

This project gathers consumer data. The data has to contain price (in US dollars), date, name of store, and location of store (city, street address and zip code.)

Ensuring accuracy with technical specs

The technical specs are an extension of the requirements (see the preceding section) and they require a detailed knowledge of the product or the work process. They’re often put into a special section and are more detailed than the standard requirements. For example:

The system must be programmed in Python and run in the Linux environment.

The name of the product must always be written in 98-point bold Helvetica.

The colour of the container must be Pantone PMS 2747.

Establishing your desirables

You can describe things that you’d like to have if at all possible – wishes more than necessities. These qualities often differ from the requirements, because you describe them in less concrete, more general terms.

The graphic should be bold and strong.

The English text should be easy to read.

We would prefer workers with Six Sigma experience.

Drawing the line in the sand: Deal breakers

Tell the crowd about things you won’t accept in the work project – the deal breakers:

The material shouldn’t be borrowed from Wikipedia or any other public website.

You should never contact the owners of these websites, even if you don’t understand the text on the front page.

The list of potential sales contacts should contain no names from outside the Sioux Falls metropolitan area (zip codes 57100–57199).

Warning the crowd away from undesirables

In addition to the deal breakers, you often have undesirable items or actions. You prefer not to see these elements in the final product or activity, but you concede that you may have to accept some of them. For example:

We prefer to avoid using a spreadsheet for this project.

We prefer that all purchases for this project be made in Scotland.

The new container shouldn’t resemble the old container for this product.

Summarising acceptance criteria

The last part of a statement of work is usually a special kind of summary. It presents the criteria that you’ll use to determine whether the work is acceptable. This section draws heavily on the prior material. Indeed, it shouldn’t introduce new ideas. In order to keep the document consistent, you may refer directly to earlier sections without repeating them. Here’s an example:

The video will be accepted if all segments of it are properly used, if all transitions are smooth and professionally done, if the branding follows the instructions in the requirements above and the final form is in the H464 standard as stated in the technical description. The final video should be no less than five minutes in length and no more than six minutes.

Making clarity your goal

Whatever form of crowdsourcing you’re using, the better you become at writing the statement of work, the more effective your crowdsourcing becomes.

Using terms of art

Make your statement of work as clear as possible. Use terms that should be familiar to the crowd that you’re trying to reach. Many fields have terms of art – that is, words that have specific technical meanings. Marketing, for example, uses words such as exposure, preference, satisfaction and confirmation, which mean specific things to marketeers. By using terms of art in your statement of work, you can often communicate complex ideas in a straightforward way to the crowd that you’re aiming to reach.

When you use terms of art, you may have to introduce them so that you communicate to the crowd that you know exactly what you want. For example, you might write:

In this project, I’m seeking someone with marketing experience to help me understand customer satisfaction and discomfirmation for our products.

I am looking for someone skilled in website design who can help us understand how our presentation is context-sensitive and leads users to commit errors.

We’d like someone who understands quality control to help develop a control chart for our operations and improve our company.

Giving the right amount of detail

One of the great challenges in writing a statement of work is striking the right balance between giving detailed instructions and overloading the crowd. Of course, you want to obtain the product or service that you desire; therefore, you need enough detail. But at the same time, too much detail can overwhelm readers and make a project seem more complicated than it really is. So you need to include enough detail to obtain what you want, but not so much that you stifle the creativity of the crowd. Here are some examples:

Not enough detail: ‘We need someone to prepare spreadsheets for our company.’

The company may be looking for someone to program marketing models, human resource databases or interactive games for its customers, all of which involve different skills, but this statement doesn’t let the crowd members know whether they’re capable of preparing the spreadsheets.

Too much detail: ‘We’re looking for a worker to help us develop spreadsheets for marketing models. We prefer inverted models with rows and columns reversed, stacked in multiple worksheets with interleaved formulae and pivot charts that link to an external file that presents the results in Geneva, not Helvetica, type.’

This statement has a lot of detail that suggests you know exactly what you want and aren’t going to be satisfied with anything else. It sounds like trouble – if you can understand all the words.

Just enough detail: ‘We’re looking for someone to program complicated marketing models in a multi-sheet spreadsheet. We have a standard spreadsheet that we’ve been using for these models, but welcome improvements.’

This statement suggests the complexity of the problem, lets the crowd members know that the company has models they can use, and suggests that the company is open to new ideas.

Getting arty

If space allows, consider including illustrations in your statements of work to clarify the text. Such illustrations are particularly useful when you want the crowd to provide an artistic or technical product.

Finbar, a scientific author, tried to get the crowd to design a cover for his most recent book. As part of his statement of work, he posted a copy of the book’s introduction and a few key paragraphs. As a result, he received proposals for covers that had not an ounce of originality. Each proposed cover looked like the cover of any general science book. The covers had a few equations scattered across the cover, a scientific graph and a picture of Albert Einstein.

After reviewing the initial submissions, Finbar spent a little time searching the web and found a few pictures in advertisements that captured some of the ideas for his book. He posted them on the crowdsourcing website alongside this instruction: ‘These photos capture the spirit of the book. Do something like them but don’t copy them directly.’ The next round of submissions were much more creative and got much closer to the spirit of the book.

By themselves, illustrations aren’t a complete work statement. They don’t tell people what’s important and what’s not. The description and the illustration should work together.

Looking at an example statement of work

This statement of work for a geography game includes every element that you may need to put in such a statement, as outlined in the preceding sections. Your statements may be considerably simpler.

I need a programmer to create a geography game to teach students of development studies about South America. The game should be similar to the Africa game at www.exampleurl.com.

The game asks students to identify the names of the countries, their capitals, their populations and their major products. This information can be found at www.anotherexampleurl.com.

The game should look and act like the Africa game. The questions should default to English, but a button should allow the user to switch to Spanish or Portuguese.

The game must be programmed in Java and HTML5. It should operate in any of the three most common web browsers.

It should contain no references to politics, political parties, current leaders or their policies.

I’ll accept the game through a four-stage review. A technical committee will review its ability to run on three major websites and be easily fixed or modified. A committee of faculty will review its content and make sure that it doesn’t touch on modern political controversies. A committee of students who are skilled at the Africa game will test the program to see whether they can operate it without any new instructions.

Connecting the Kneebone to the Thighbone: Creating Instructions

Sometimes a statement of work is simply not enough. You can’t just tell the crowd what you want. You have to tell it how to do it. You most commonly have to do this in microtasking, although you may find that occasionally you have to write instructions for macrotasking and other forms of crowdsourcing.

If you’ve ever had to assemble a piece of furniture or a complicated toy, you may know one of the basic problems of instructions: instructions that may be clear to the writer are not always clear to the person who’s trying to follow them. In crowdsourcing, the final judge of an instruction is the crowd; if the crowd members don’t know what the instruction means, then the instruction is simply no good.

To write a good instruction, you need to be careful, clear and organised. You also need feedback from other people. The following sections establish basic principles for creating clear, effective instructions.

Keep in mind that when you write instructions for microtaskers, you often need to add additional material to help them complete the task. Generally, best practice is to separate that material from the instructions. Keep the instructions succinct and clear. Put the additional material in a separate section.

Thinking about who does what to what

The best instructions contain these three elements:

Subject: The person performing the task – the crowdworker

Action: The action that the crowdworker is to perform

Object: The thing or idea that the crowdworker is working on

Here are some examples:

You should click on the URL and review the images on the page.

You should paste your translation into the field at the bottom of the page.

You should verify all the information in the grey box, including name, title, email address and phone number.

Addressing the subject

When you write instructions, you address the crowdworkers and show them how to do the job. You have two choices:

Personal address: You write ‘You should do this’ or ‘You will do that’ or ‘You should be moving’.

Impersonal address: You substitute ‘the worker’ or its synonyms as the subject: ‘The worker should do this’ or ‘The crowdworker will do that’ or ‘The worker should be moving’.

Both forms of address have advantages and disadvantages. When you use personal address, you can make the instructions sound more individual, more connected to the worker. However, if you’re not careful, you can sound as if you’re bossing the crowd with harsh commands.

At the same time, when you use an impersonal form, you put a little distance between yourself and the crowd. You don’t sound as if you’re giving commands to the worker, but you may sound as if you’re talking to someone else, someone who’s not in the crowd.

Describing the action

Whenever possible, describe actions with positive words. Avoid negatives, which can be confusing and actually encourage workers to do the very things that you want them to shun. Rather than telling crowdworkers not to do something, tell them to do a positive action.

Identifying objects

Sometimes you can simply describe the object: ‘the field in the middle of the screen’, ‘the image in the upper right hand corner’, ‘the data from the pop-up window’.

If something is complicated to describe, or if it’s used many times by the worker, you may want to give it a special name. ‘We will call the data in red font the red data’. Such names can greatly simplify the descriptions. For example, you may write:

On your screen, you will see one image on the left, which we will call the target image, and four images on the right, which we will call the sample images. The sample images are numbered from one to four.

Compare the target image with the four sample images. If the target image matches any one of the sample images, then put the number of that sample image in the box at the bottom of the screen. If the sample image matches none of the sample images, put the number 0 in the box at the bottom.

Notice that the following description is a much simpler one that doesn’t give a name to the target image:

Compare the image on the left of the screen to the images on the right of the screen. If the image on the left matches any of the images on the right, then write the number of the image on the right in the little box below the image on the left.

These two descriptions illustrate the advantages and disadvantages of identifying objects with special names. If you give objects special names, you can clearly identify the objects that the crowd has to manipulate. However, you may also produce instructions that are much longer. If you rely on general names to describe objects, you may be able to produce shorter descriptions, but these descriptions may not be as clear.

Don’t get too carried away coming up with special names. You can quickly make a description confusing by giving each object a shorthand name. If you give too many things names, the worker has to take the time and effort to learn all the names.

Deciding the order of instructions

Usually, you write instructions in order: one, two, three. First you do this. Then you do that. Finally, you do the last thing. If any hard-and-fast rule exists for instructions, it’s this: keep your instructions in order.

For microtasking, you want to follow this rule to the letter. The simplest instructions are those that follow a rigid temporal order.

Getting Feedback on Your Guidance

When you’re preparing a statement of work and/or a list of instructions, give it to friends or colleagues and ask them to try to understand the job or follow the instructions. If they’re having trouble with them, the crowd isn’t likely to do any better.

Feedback from the crowd is also essential. Having its feedback early can save you trouble later. If the crowd reviews the instructions, you’re less likely to discover that it can’t understand the task.

Put the statement of work and/or instructions on a crowdsourcing website. Do a small test run. Ask five or six members of the crowd to do the work and then give you a review of the instructions. Their feedback can help you refine your instructions.

Be open to messages from the crowd about the project, too. Working with the crowd involves a dialogue. You offer a statement. You give the crowd a set of instructions. It responds to you. You listen.

You may, however, discover that the crowd doesn’t always communicate with words. You’ve written a statement of work and posted instructions on the crowdsourcing site and you’re waiting for an email message from the crowd or a post on your crowdsourcing blog to tell you that the instructions are good or that they’re bad. However, the crowd may not be talking to you through email or through a blog. It may be trying to communicate with you through the market.

The crowd sends two messages through the market. It tells you that it likes your jobs or that it doesn’t like your jobs. It can like your job because it has a big payment or because the job is easy to do. It can dislike your job because it offers only a low payment, because the job is hard or because understanding what you want is difficult.

For microtasking, the market often provides the only information about how the crowd is thinking and how it’s engaging with your tasks.

If your job isn’t attracting much attention from the crowd, you may be getting a message from the crowdworkers about your statement of work or instructions. The crowd may be telling you that your statement of work is too complicated or your instructions are too hard to follow.

When the crowd doesn’t take your tasks, especially your microtasks, first review your statement of work and instructions. If they’re complicated or confusing, revise them. If the crowd still doesn’t take your tasks after you’ve revised the statement of work and instructions, consider adjusting the wage that you’re offering.