Preparation Phase

To create the cards, I use the Mail Merge feature of Microsoft Word. The input is simply an Excel spreadsheet with one row per card and all the relevant information separated into columns (with column headers). Word allows you to customize the layout of the cards by using the Mail Merge fields.

It is important to have one thought/comment per row. For surveys, the best approach is to design the survey in a way that allows different thoughts to be entered separately. For example, instead of having one text box for all five responses, Andrew Begel and I provided one text box for each response: “Please enter your first response.” TEXTBOX. “Please enter your second response.” TEXTBOX. And so on. Unfortunately, it is not always possible or practical to design surveys in such a way. In that case, you might have to split responses manually (sometimes with support from your survey tool), ideally before you print your cards. For large documents like webpages, interview transcripts, or PDF files, quantitative coding tools such as Atlas. TI or QDA Miner (which has a free Lite version) can be used to extract relevant pieces of text from large documents.

On each card, I print demographics, if available, and always a card identifier to uniquely label the card. The demographics provide you with a better context during sorting. The identifier allows you to combine the results of the card sort with additional data, which can enable additional analysis later on (see the following for an example). I simply number the cards to get my identifiers, eg, 100, 101, 102. I typically start at higher numbers (like 100 or 1000) so that all identifiers have the same number of digits; I found that this helps to enter card numbers more easily into the computer later on.

I usually prints cards 4-up on a letter page. Print the text in a large font (at least 20 point); the larger the better, and the easier it is to read. After the mail merge, you can manually reduce the font size for any cards that don’t fit. A trick I recently discovered is to sort the Excel sheet by response length (put longer responses in the first rows to prevent Excel from cutting off text). That way, you have to only go through some cards to reduce the font size, or you could split the responses into two or more sets (eg, one set for short responses in a large font, one set for long responses in a smaller font).

Execution Phase

Get a large room for your card sort. Besides the cards, bring pens, sticky notes, markers, envelopes, and rubber bands. Use the pens and sticky notes to create descriptive titles once you have several cards in a group. Use the markers to highlight important text, for example, when the text is very long. Plan about 2 hours for a card-sorting session, but not more than three hours. Longer sessions give you brain freeze. If you have a large number of cards you will need multiple sessions. Don’t forget to take breaks.

The first card is the easiest one to sort. It always starts its own group. For the second card, decide if it is similar to the first group. If it is similar, put it in the first group, if not start a new group. Repeat for each of the following cards: decide if it fits an existing group, and if not start a new group. Keep going until you run out of cards.

Don’t overthink where to put a card. If it’s difficult to decide where to put a card, put it at the end of the pile and come back to it later. The groups are not fixed and can change during the course of the card sort. It is fair game to split, merge, or even resort groups. For example, when a group gets very big, you might want to revisit the cards and see if there are important subthemes. While sorting, you might discover some cards that make no sense or are off-topic; I put those in a special Discard group.

Card sorting in teams can help you derive better themes and sort a larger number of cards. Jointly sort the first few cards with the entire team (calibration phase), then divide up the cards, but still communicate while sorting. Let others know when you start a new group. When you have an ambiguous card, read the card out loud and ask others for feedback. Avoid having more than four or five people for a card sort. Too many cooks spoil the broth. Always have one team member lead the card sort.

Don’t forget to take photos of your card sort. That will help you to later reconstruct the spatial layout of the cards. Photos are also great to impress friends or to explain the process in a presentation. Please refer to Fig. 1.

After you are done with the card sort, use the envelopes and rubber bands to store the cards. I put the sticky note to the side of the top card and then pile up the groups. Have the sticky note stick out a little from the top card. That helps you to later separate the pile of cards into the individual groups.

Analysis Phase

After you are done with the card sort, it’s a good idea to go through the cards one more time to check for consistency within the groups. It’s still okay to move cards around, though at some point you want to freeze the groups.

As the last step, take a look at the groups and see if you can deduce more general categories and themes, especially if you have many groups. In the card sort of data science questions, Andrew Begel and I ended up with 60 groups… too many to make sense of the data! We then used affinity diagrams to find general themes. The result was a hierarchy of themes and categories. Each group from the card sort was a category. We then combined categories into themes. For example, the groups “Productivity Measures,” “Measuring the Individual,” “Impact on Productivity from Build and Process Tools,” and “Tradeoffs between Vendors and Full Time Employees” all became part of the theme “Productivity.”

If you have extra data, you can do a quantitative analysis of the themes. For example, for the card sort of bug reports questions, we had the response time for each question. With the identifier, we could tie back the response time to the different themes and check for differences. Before you can do the analysis, you need to enter the results of the card sort into the computer. Always enter card numbers in pairs: one person reads out the numbers, the other types in the numbers. Both check for errors. To combine card sort categories with the original data, I use the VLOOKUP function in Excel.

Last, a word of caution. A common mistake is to give more importance to themes that have more cards; however, quantifying inherently qualitative data is dangerous. Here’s a simple example for the question from the beginning “What problems are stopping us from meeting our deadlines?” If someone does not mention the slow build system as a problem, it does not automatically mean that they are happy with the build system. They might not have recently worked with the build system or they just did not think of it when they took the survey.

That’s all you need to get started with your first card sort. Good luck!