Chapter 17

Crowd Reporting: Using the Crowd to Gather Information and News

In This Chapter

Using the crowd to get a picture of events and feelings

Editing material to piece together the story

Using the Ushahidi crowd reporting software

Skirting around the pitfalls of crowd reporting

The crowd isn’t only a source of labour. It can also be a means of gathering news. It has eyes and ears and minds that can gather and process information. It can be deployed across a large geography and be organised at short notice. When you listen to the information that the crowd has gathered, you’re engaging in crowd reporting. This activity is essentially a form of microtasking, although it can include macrotasking as well.

The crowd may not be a perfect means of gathering information. To see its weaknesses, all you need to do is watch a Twitter feed for ten minutes and try to follow the rumours, repetitions and odd ideas that sprint across the screen. But even with its weaknesses, the crowd is often the best means for learning about the impact of natural disasters, for monitoring large social rallies and events, or for keeping track of any detailed activity.

To get good information from the crowd, you need to be a good manager and a good listener. You need to help the crowd find the right information, you need to help it organise it, and you have to find the patterns of truth within the stories that the crowd sends you. This chapter helps you expertly handle each aspect of crowd reporting management, so you can be aware of the kinds of stories that the crowd can cover and the topics that it finds difficult.

Using the crowd to report news isn’t the same thing as using the crowd to gather uniform bits of information for a business process. You may want to use the crowd to collect product prices for a market report, or find contacts for a sales campaign. In such cases, you’re really doing traditional microtasking (see Chapter 8).

Understanding Why People Use Crowd Reporting

Crowdsourced news reporting brings breadth and depth to the process of gathering information. It brings breadth because it can deploy reporters around the globe in a way that no conventional news organisation can match. It can find reporters in the most distant communities and get them to the scene of a natural disaster. It can gather details from any place at any time. If you want broad coverage, you turn to the crowd.

Crowd reporting has an equal ability to find a depth of detail that’s hidden to most reporting organisations. When a member of the crowd lives in a neighbourhood, he knows the residents of the area, the personalities of the leaders, the issues that touch everyone that lives there, and even the impact of the cracks in the road. If you need to know every little detail, turn to the crowd.

Sorting Eight Billion Stories

If you’re going to use the crowd to report news, you need to sort the information that you receive. The world is filled with far more information than any of us can manage. Some of this information is captured by social media, where it’s directed into concentrated streams. These streams give insights into the vast experience that the eight billion inhabitants of this world have every day. However, few of these streams are truly useful. To get good streams of information, you need to be a good editor. You need to choose which ideas are important and which are not.

Two crowd-based websites, Wikipedia (www.wikipedia.org) and The Huffington Post (www.huffingtonpost.com), illustrate the importance of good editors. Both solicit information from the crowd and both have a strong editorial staff. At Wikipedia, the editors are part of a volunteer community. They're writers who have earned special privileges on the Wikipedia site. They can remove contributions and suggest information that may be needed for specific entries. The Huffington Post has a paid editorial staff, a group that resembles the editors of a traditional newspaper. The editors solicit material from the crowd and choose the stories to publish.

The following sections outline key considerations for crowd reporting that help you be an expert editor.

Helping the crowd focus

When you use the crowd to report the news, you need to provide the members with help and structure, even if you’re asking them to find information that you can’t describe in advance. If you tell them to go and find something interesting, they may find things of interest to them but not necessarily to you. Therefore, you get the best results if you give the crowd fixed boundaries for its work.

In general, you find that the crowd works best if you restrict the assignment by time or space or to a specific object. So, you can ask the crowd to report on:

An emergency that was caused by an obvious issue such as an earthquake or blizzard.

An event that has a clear timeline, such as an election.

Simple events that need to be observed regularly, such as common city services, For example, cities or utility companies should provide water to every neighbourhood, every day of the week. If you need to verify these services, you can deploy a crowd to check every single residence.

In each of these situations, you give the crowd a story that has missing pieces that the members can find. If you ask the crowd members to report on an earthquake, they’ll look for the effects of that quake on people and infrastructure. If they’re following an election, they know how the story should progress and look for deviations from that story. If they’re monitoring a city service, they look for evidence that the service isn’t being performed as it should be.

The crowd can’t always combine news easily. Each member of the crowd has a point of view and can easily convince himself that he’s at the centre of the action. With crowd reporting, you need ways of combining and weighing information to get a full and balanced picture.

Combining amateurs and experts

When journalists create reports, they try to get gather information from multiple sources and different viewpoints. They do this, in part, to be fair and open minded. However, they also do it to avoid being embarrassed. If they listen to only one side of the story, they can easily be misled.

The crowd can naturally give you many points of view if the members work independently and see the same things. However, you often deploy the crowd over a large geographic area. In such a case, one crowd member may be the only person covering a particular part of the land. To get a different point of view, you can try deploying a few experts or professionals to see whether their experience supports the reports coming from the crowd.

When you send experts to an area, you don’t need to send highly trained journalists. You only need to send people you trust, people who know the information you want and can report it to you. If you want information on a tsunami, they can recognise a damaged building. If you’re monitoring an election, they know how to recognise voter intimidation. If you’re checking snow removal, you send people who know which streets should be cleared first and how they should be cleared.

If your experts and crowd agree, you’ve reason to believe that you’re getting good information from the group, but it’s no guarantee. Both the experts and the crowd can be wrong. Also, if the crowd and experts disagree, you shouldn’t assume that the crowd is wrong and the experts are right. It may be the other way around.

When you’re trying to draw conclusions from the crowd, you receive more benefit from being disciplined than from being an expert. You’re an expert when you know a lot of facts and theories. You’re disciplined when you approach a story in a systematic way to avoid convincing yourself that something is true when it’s not.

You carry opinions and prejudices. Everyone does. But when you’re disciplined, you try to keep those opinions and prejudices from letting you accept a conclusion as true when it’s not. A disciplined reporter is always asking questions such as ‘Is this fact true?’, ‘How is this fact pushing me towards a certain conclusion?’ and ‘Would I accept that conclusion even if I did have that fact?’

When you work with crowds, being disciplined is easier when the members gather facts than when they reason their way to conclusions. When the crowd is gathering facts, you can ask the members to double-check their sources, and you can ask multiple members of the crowd to find the same information. Getting the crowd members to check and verify the reasoning that led them towards a conclusion is more difficult. When you reason, you make jumps in logic that are difficult to explain. You might see a cloud no bigger than a man’s hand and conclude that a tornado is on the way. You may reach that conclusion through years of observation, by careful reading of books about meteorology, because your neighbour has told you that tornados always follow small clouds, or you may be inventing a theory that has no basis all. When you try to get a reasoned conclusion, you tend to get better results if you can test each step of the reasoning process.

Gathering Information Geographically with Ushahidi

Ushahidi means testimony in Swahili, and is the name of a commonly used piece of crowdsourcing software. It was developed by a Kenyan, David Kobia, to monitor the 2008 elections in his country. It still retains strong ties to Kenya, even though the software is maintained by a team spread around the world and supported by a foundation based in Florida, USA.

As a type of software, Ushahidi is a kind of geographical information system. It enables the crowd to identify news and items of interest with specific locations on a map. The crowd members use the system to say that a fire has occurred at this house or a protest is happening in that square, or that food can be found at a distant office building. They can use email, text message, or a smartphone app or laptop computer to record the information. The crowd members identify an event, mark the spot on the map and then type in a description. As the crowdsourcer, you can review these reports, look at the patterns and download the information for further analysis.

Ushahidi has been used for many different kinds of crowd reporting, although the software is most commonly associated with monitoring elections and natural disasters: elections in Kenya, Namibia, India, Ethiopia, Brazil and Burundi; after the 2010 Haitian and Chilean earthquakes and the 2011 Louisiana oil spill; and to monitor other issues, such as crime in Atlanta and Washington, DC, USA.

In the summer of 2012, the London Olympic committee deployed the Ushahidi software to monitor events that affected public life and public safety during the games. The committee used the software to combine news from official press releases, official notices, reports from police and security officials, as well as incidents identified by the general crowd. Figure 17-1 depicts the summary map of Ushahidi for the Games.

The dots on the map represent reports of different incidents for one day at the Olympics. Clicking on a dot gives you a summary of the incident. (The one identified in Figure 17-1 is a road closure.) A further click on the report gives more detail about the incident, including links to further information. The record also allows further input from the crowd. The crowd members can confirm the report or indicate that they can find no evidence of the incident it describes. Figure 17-2 shows an incident report.

Reproduced with permission from Juscomms (www.juscomms.com).

Figure 17-1: Ushahidi at the London Olympics.

Reproduced with permission from Juscomms (www.juscomms.com).

Figure 17-2: Ushahidi incident report.

The following sections show how you can harness the power of Ushahidi for your own crowd-reporting project.

Rallying the crowd to Ushahidi

Like crowdfunding (see Chapter 6), crowd reporting requires you to recruit a crowd to your project. You can start with your social circle and then begin to search beyond it. You want the usual tools to promote your work: a web page describing the project, videos that show the issues you’re trying to monitor, and tweets and emails that ask the recipient to help with the recruiting.

Because you may often organise a crowd to support civic good, you can find organisations that will assist you. You can turn to business clubs, churches, volunteer organisations, and even to companies that may be of help.

Use the recruiting process as an opportunity to train your crowd. Show the members a video of the kinds of things you’re trying to monitor and ask them to take part in the project.

Deploying Ushahidi

Individuals, small non-profit organisations, large organisations and everything in between can use Ushahidi. Whichever type of organisation you come under, you can either find a site that hosts the software and allows you to use it on its servers, such as Crowdmap (https://crowdmap.com), or you can obtain a copy of the software and install it on your own server. Many people like the simplicity of Crowdmap, but if you have a large project you may need a server of your own.

If you need a server, you can either have a machine of your own or use a commercial web server such as one at network solutions (http://networksolutions.com) or DreamHost (http://dreamhost.com). You also need a person who's familiar with the basic issues of web hosting and who can help you set up a website for your version of Ushahidi.

Ushahidi is a fairly simple package to deploy. You need to:

1. Find space on your host.

2. Set up the appropriate URL.

3. Create a database.

4. Install the software.

5. Upload a map for your project.

6. Connect the appropriate email and text message accounts to the system.

7. Write the instructions for your project.

Then you’re ready to organise your crowd.

Ushahidi has a detailed manual that helps a technically trained individual deploy the software.

At least one organisation, Crowdmap, provides the Ushahidi software as a hosted service. Instead of installing the software on your site, you simply open an account on Crowdmap and use Ushahidi on the organisation’s servers. As may be expected, the Crowdmap version of Ushahidi isn’t quite as flexible as the version you can install on your site. However, many smaller organisations find the Crowdmap version perfectly adequate for their needs.

Summarising the results

After you’ve created an Ushahidi website and have started collecting information from the crowd, you have to summarise that information. For many applications, the basic Ushahidi map, the kind you see in Figure 17-2, is sufficient. It shows all the incidents and pins the incidents to a map.

For some applications, however, you want to create a summary of the information across geography. For the 2012 London Olympics, for example, the organisers wanted to summarise for the public the kinds of issues they faced each day. To do this, they used a word cloud to display the types of incidents. In a word cloud, the size of a word indicates how often that word appears in the data set.

Figure 17-3 shows a word cloud for a single day during the London Olympics. In this cloud, four words are prominent: Underground, Traffic, Emergency and Medical. These four words were the most common words that appeared in the Ushahidi postings on that day. The cloud tells you nothing about where the messages originated and nothing about how the words were used. The words might have appeared individually or together. London might have had a series of medical issues, some problems with the underground, a number of emergency calls, and roads that were relatively free of traffic. Or the city might have had a lot of medical emergencies in the underground that snarled traffic. The word cloud gives you a summary and suggests how the Olympics staff spent their day.

Figure 17-3: Sample analysis of a report in a word cloud.

Getting the Benefits while Avoiding the Perils of Crowd Reporting

Crowd reporting is one of the more challenging forms of crowdsourcing. The challenges are not found in the software or in the work of recruiting the crowd. You can install Ushahidi (see the preceding section) if you have the skills to install any modest network software, such as a blogging system.

What makes crowd reporting challenging is that it’s vulnerable to errors and mistakes in a way that macrotasking or even microtasking is not. If you ask the crowd for judgements, you can easily receive inaccurate information. In the earlier section ‘Combining amateurs and experts’, I recommend having experts check some of the reports, but even then potential for misinformation exists. For this reason, many commentators, and even a few experts on crowdsourcing, think that crowd reporting is fundamentally flawed and can never be trusted.

Crowd reporting’s unreliability comes from four different sources:

Uncooperative workers who don’t want to provide good information

Crowds that don’t represent an area or region in the way you believe they should

Members of the crowd following the lead of one or more individuals rather than thinking for themselves (this problem is usually called the crowd effect)

Bad information driving out good information (this problem is called Gresham’s Law)

You can limit the influence of uncooperative workers with the same kind of methods that you use to screen and train workers in microtasking (see Chapter 8). You can give them a test to determine their interest and skills, and then carefully train the workers who pass the test to make sure that they know what you want.

As the following sections outline, the other problems have to be handled by careful management and by the use of trusted individuals to review the work of the crowd.

Understanding the nature of the crowd

When you gather information from a crowd, you need to know the nature of the crowd. In the simplest example, when you use Ushahidi to collect data about the damage caused by a natural disaster, you need to know where the crowd members are located. If you think the members are in one area but in fact they’re in a different place, you’ll get bad information through no fault of the crowd.

In many situations, you need to go beyond the location of the crowd and know something about the demographics of its members. If you believe that you have a liberal crowd, you may interpret their reports in a certain way. If the group is actually conservative, your interpretations are going to be wrong. If you believe that you’re getting a picture that’s balanced by equal numbers of men and women in the crowd, but in fact the crowd is entirely male, you may again make bad judgements from the information provided by them.

When you listen to the crowd, always ask, ‘What can go wrong if I make a false assumption about the crowd?’ In some cases, the answer is ‘nothing’ and you don’t need to worry about the nature of the crowd. However, if you’re running a project over a long period, you may assume that the nature of the crowd doesn’t change. Some individuals may leave the group, but they’re replaced by people who are similar. If you’re dealing with a natural disaster, such an assumption may not be correct. The people who rush to a disaster are often people who get great satisfaction from working in emergency settings, and they may leave as soon as the immediate danger has passed. Those who come later are often individuals who want to see the situation return to normal.

Understanding the nature of the crowd is similar to understanding the nature of a mass market. You can do expensive and detailed market surveys, but if those surveys don’t tell you about the individuals who are part of the market, they really tell you nothing at all.

Every marketing professional has a story about a survey that failed to describe individual consumers properly. It usually involved consumer products such as clothing or household furnishings. These stories usually start with a sudden rise in the demand for a product. Excited by the rise, the product manufacturer does a quick survey of the market and decides to invest in new plants to create more of the product. But just as the plants are starting to operate, the manufacturer discovers that the survey failed to capture the true nature of the market. The manufacturer had assumed that its product was being used by a stable cohort of customers: middle-aged men, perhaps, or women with two or more children. To its horror, the company discovers that its product is being purchased by the most fickle of market segments, young teenagers. In just a few weeks, the teenagers abandon their product. The manufacturer is left with excess factory capacity and low sales. In the most dramatic version of these stories, the manufacturer is deeply in debt and never recovers from the mistake.

When you gather information from the crowd, you should know something about the individuals who are responding, why they’re joining the crowd, and how committed they are to your work. You can find this information during the recruiting process and in the activities you use to train the crowd.

If you’re using Ushahidi to monitor an election, you probably want to know something about the politics of the people who are reporting on the political activities and the polling stations. If a group of people are reporting that one party is stuffing the ballot boxes, knowing something about the people who are reporting the abuse would be useful. Are the reporters members of the party that’s accused of voter fraud? Are they from the opposition party? Do they claim to be neutral observers? If the reporters are in the opposition, you may respond to the story of fraud by sending neutral observers to confirm the stories. If the stories came from neutral reporters, you may move more quickly to file a complaint of fraud.

In this example, to give you time identify and learn about the party of crowd members, you might enable people to register early. In this early registration, you ask reporters to identify their party affiliations. You can then verify these affiliations with the voter registrar. If a reporter doesn’t register or registers late, you might assume that he’s reporting an event that was instigated by the opposing party. You can then try to send a neutral observer to confirm the report.

Knowing who’s talking: The crowd effect

Follow the leader. That’s the crowd effect. You get it when individual members of the crowd surrender their judgement to a few individuals. Rather than processing information independently, individuals often look to a leader and accept the judgement of that person.

Some of the recent research on the crowd effect suggests that this effect commonly occurs when each member of the crowd can see only part of the information that may be useful to him. When the crowd members find out about decisions from other members of the crowd, they conclude that those members must know something that they don’t. Rather than make an independent judgement, they repeat what they hear.

The crowd effect can be so powerful that some members of the crowd ignore or discount concrete information that they possess. For example, they may be looking at a polling station that’s run well and has no evidence of problems, yet because they’ve heard so many stories that they believe of voter fraud, they decide to report that something’s going wrong, even though they see no evidence.

Limiting the crowd effect is hard. The following two sections look at your options for doing it.

Isolating the members of the crowd

The first strategy is often difficult. In some cases, you can easily isolate individual members of crowd. In certain cases, the members work by themselves and have contact with only one or two others. They don’t see all the information until it’s been given to you. However, in situations such as natural disasters, you have a crowd of people who want to work together and have contacts that you can’t control. If you can’t control the interaction of the crowd, you want to deploy experts who can guarantee to give you independent information.

To isolate members of the crowd, you often have to physically separate the individuals and prevent them from communicating with each other through text messages, emails or tweets – something you can rarely do in the real world. Most members of the crowd want to communicate with other people and resist giving up their communication tools.

Of course, the crowd effect can be influenced by many other factors: laziness, strong leaders, weak followers, contradictory information and many other things. You can’t always limit the impact of these factors by isolating the members of the crowd. To reduce the impact of these factors, try to identify problematic individuals when you recruit them, just as you give a test to workers who want to do microtasks (see Chapter 8). Create a questionnaire that may test the character of a potential reporter. Ask ten questions, for example, about difficult situations. You might ask how the reporter would respond if he saw a house burning and heard people calling for help. Would he try to put out the fire, call for help or take a picture and post it for his friends? Determine the answers that you’d expect from a good reporter and select for your crowd the people who answer seven of the ten questions as you want.

Instructing crowd members to focus on the information that they can see and ignore others’ opinions

You can more easily instruct the crowd to ignore other peoples’ opinions. For example, you might say ‘Suppose you see an event that’s very different from what others see. Considering only what is before you, what do you conclude?’ The problem is that the crowd doesn’t always follow those instructions.

Instructing the crowd to ignore the ideas of other people is not a perfect way to avoid the crowd effect. People will still ignore your advice and listen to other members of the crowd. Instructions, however, usually have some effect, and instructing the crowd is much easier than isolating each member.

Knowing what the crowd believes: Gresham’s Law

‘Bad money drives out good.’ Gresham’s Law is an observation on economics that’s named after 16th-century English financier Thomas Gresham. Applied to crowdsourcing, Gresham’s Law means that the crowd sometimes loves a good story more than it loves the truth. It circulates rumours that sound interesting rather than stories that it knows to be true. If you check with the crowd, you may easily find that the Great Wall is the only manmade structure that can be viewed from space, that Elvis Presley is alive and living in Kalamazoo, and that the United Nations has appointed an ambassador for UFOs. These are interesting stories and fun to tell. However, they aren’t true.

Some people confuse the crowd effect with Gresham’s Law. They’re not the same thing, although they are related. When you see the crowd effect without the effects of Gresham’s Law, you receive the judgements of only a few people, but the information may be valid. When you get the effects of Gresham’s Law without the crowd effect, you hear an odd rumour from certain members of the crowd, but most of your information is good.

In crowdsourcing, you most commonly find the impact of Gresham’s Law in emergency situations. At these times, the crowd is often filled with emotion and willing to accept dramatic stories. The crowd members may report that buildings have been destroyed by earthquakes when, in fact, the structures are sound. They may report rioting in the streets when the city is calm. They may think the worst of their neighbour when they’ve no evidence to support their opinions. They may claim that their opinions are typical when they’re not.

In some ways, crowdsourcing encourages rumours, because the transmission of rumours is a form of divided labour. Few people repeat a rumour without modifying it. They add details that make the rumour seem real, and drop information that isn’t consistent with the rumour. Such things are readily done by the crowd.

You find Gresham’s Law harder to address than the crowd effect. You can’t simply isolate the members of the crowd from each other and expect to get better results. While communications experts know how to counteract a rumour when they know the truth, they don’t have a good approach for identifying rumours without outside information.

Rumour detection is the work of professional journalists. They detect rumours by listening to different sources, comparing information and balancing different points of view. However, even the most experienced journalist can make mistakes or be misled. Sometimes, you can be fooled by false information from the crowd.

If you’re using the crowd to gather information, you may find the best protection from Gresham’s Law to be a small group of experts who you trust to give an accurate result.