Web-based software comes and goes even faster than regular software, and to be honest, we think 98% of all tools we write about now will either evolve unrecognizably or vanish in five years’ time. But we recognize that it’s helpful to cover the software tools you’ll be using to conduct research, so here we go.
This book isn’t an instruction manual, so we’ll avoid the obvious “click-here, click-there” tutorial steps. You can find how-to guides on the Web site or in the manual of whichever service or software tool you use. This chapter is more of a reference guide to the best parts of each tool, in our own ludicrously biased opinion. We describe how different tools work for the purposes of remote research, as well as any neat highlights of the tool so that you’ll be able to make an informed decision about which ones to use. We cover a handful of the most useful and interesting solutions for screen sharing, screen recording, and automated research, and some tricks we’ve discovered for getting around their limitations. For most tools, we’ll give you a brief summary of their most important features. You can visit their Web sites to learn more if you’re curious.
As always, check out http://remoteusability.com for a more up-to-date list of remote research tools and services.
You’d think something as simple as screen sharing would be the same for one tool or another, but the tools out there today have plenty of important differences. You not only have to see the participant’s screen, but you have to consider these factors:
Cost
Security
Operating system compatibility
Observing and recording features
The amount of work the participant needs to do to set it up
Firewall and spyware blocker compatibility ...and more.
GoToMeeting is one of the most commonly used screen sharing tools simply because it is low cost, works on all platforms, and isn’t horrible to use. But other tools might better suit your purposes, so fortunately, you have us to help you sort out this stuff.
Adobe makes everything else, so why not screen sharing? Adobe Connect (officially “Adobe Acrobat Connect”), a slightly newer bit of software, is a mid-priced Web-conferencing solution with a few neat twists (see Figure 8-1).
Figure 8-1. Adobe Connect: cross-platform, Web-based, quick to set up, and includes integrated video chat.
Integrated video chat. The most interesting new feature is its integrated webcam chat. If you’re running a study in which you know that the users will have webcam access and you’d like to see their faces and desktops simultaneously, Connect is your tool. We’re likely to see more and more computers with built-in webcams, so it’ll just be a matter of time until you’re able to recruit users live and then have the option of seeing their faces minutes later.
Choice of monitor or window to share. Your participants can choose which monitor or window to share. (Still no way to share both, though.)
Things to note:
The first time you use Acrobat Connect, click Share My Screen. This will prompt you to download and install a plug-in, and that’s the only time you’ll have to do it. After that, it should always launch using the plug-in rather than a tab in your Web browser.
There is integrated recording, though again, since it requires you to go through its conference call service, we’ve always just preferred to record separately.
There’s a crucial difference between Adobe Acrobat Connect and Adobe Acrobat Connect Pro, the most notable of which is that the latter costs nearly 10 times as much as the former. Be sure you’re ordering the version you want. We made that mistake once, and getting it switched was a real Old Testament struggle.
It comes with a number of conference-calling options at additional cost.
To allow your participants to share their screen, you need to select the Auto-promote Participants to Presenters option in the Meeting menu; otherwise, the Share My Screen option won’t come up.
Your participants should enter as Guests. After they enter, you’ll need to allow their entry into the room on a little pop-up dialog box that appears in the bottom-right corner of the window, which is easy to overlook.
Participants have to install the Adobe Connect plug-in at the beginning of the session. It’s not large, but it does not uninstall automatically at the end of the session. That doesn’t mean that you’ll be able to see the participants’ screen again, but it does mean that you won’t be able to say that “nothing gets installed.” You may want to walk your participants through the uninstallation process, unless they’re okay with keeping it on their computer. We have no idea how to uninstall the thing.
Web site: www.adobe.com/products/acrobatconnect (to buy), connect.acrobat.com/[your connect address] (to join meeting).
Pros: PC, Mac, and Linux compatible; supports simultaneous webcam/screen sharing.
Cons: Adobe Connect plug-in doesn’t automatically uninstall from users’ computers.
Requirements: Participants need to have Adobe Flash Player 8 (or later) on their Web browser and a ~450mHz or faster processor. A wired connection is recommended.
Price: For standard version: $39.95/month. For premium version (which includes session recording and other nice things): 32 cents/minute/user on a pay-per-use plan, $375.00/month for 5-user capacity, $750.00/month for 10-user capacity.
Final Word: Solid. The webcam option is pretty interesting but not strictly necessary. For now, we suspect it could be an unnecessary bandwidth and processor hog.
TechSmith’s UserVue is currently the only screen sharing solution out there that’s specifically designed for moderated user research, so it’d be foolish if we didn’t mention it here (see Figure 8-2). Unfortunately, it might not be around after mid-2010, but here it is anyway.
Observer and moderator video annotating. While the sessions are ongoing, the observers and moderator are able to type in notes that are integrated into the video file as markers. This capability can be handy for tagging interesting observations and issues for ease of later reference.
Observer–moderator and moderator–participant chat. Instead of throwing moderator, participants, and observers into the same chatroom, UserVue wisely insulates the observers from the participants so that they can chat with the moderator and among themselves, invisible to the participants, preventing disruption.
Integrated audio and video recording. When you finish a session, you get one tidy little recording of the entire session, with the phone conversation synced to the video. How does UserVue do this, you ask, if you’re using a normal phone line? Here’s what happens: after you enter your phone number and your participant’s phone number into UserVue and click Call, your phone will ring. When you answer the phone, UserVue will tell you to dial “1,” which will ring the participant’s phone as usual. When the participant picks up, the conversation begins just like normal, and both sides of the phone call get routed to UserVue’s servers, which syncs the audio with the video.
Figure 8-2. Uservue: integrated user dialing, video and audio recording, chat, and observing. To be discontinued in 2010.
Things to note:
TechSmith tells us that it intends to discontinue UserVue by the second quarter of 2010, due to lack of demand. That totally sucks, but maybe (if this book spikes interest?) there’s a chance TechSmith will keep it around.
You can’t dial outside the Americas directly from UserVue, although there is a way to do it indirectly (see the following sidebar).
Limit 10 observers at a time.
UserVue takes anywhere from 5 to 15 minutes after each session to render out the video recording of the session, and you can’t start a new session until the rendering is finished. The render time depends on how fast your computer is and how long the session was. You can’t batch-render the videos on other computers, as far as we’re aware.
If your or your users’ Internet connection is disrupted during the session, or one of you accidently exits the UserVue session, you will see a message in the UserVue window that reads “Your connection to the participant has been lost.” Unfortunately, this means that you have to exit the session, render and save the video file, and start with the setup all over again from step 1 to resume the session. Sorry, that’s just how UserVue works.
Like most current screen sharing solutions, UserVue has a decent frame rate and less than a second’s lag with a good connection, but not really decent enough to follow along smoothly with any videos that your users are playing. Videos will look choppy on your end.
There have been rare occasions when the UserVue servers have gone down on testing days, making it necessary for us to resort to a different recording method on short notice. There’s really nothing to be done about it, which is why it’s important to have backup recording methods on hand.
The session recordings can be rendered in either WMV format or the proprietary Morae format, which you’ll need a TechSmith program like Camtasia to open.
Web site: www.techsmith.com/uservue.asp (to register), http://uservue.techsmith.com (to host/join a session).
Pros: Automatic integrated audio and video recording, observer chat and participant chat.
Cons: No international dialing; PC-only for user, moderator, and observers; 5–15-minute video rendering wait time after each session; have to set up a new session for each user; not cheap.
Requirements: PC for moderator, participant, and all observers; user needs to download and launch an .exe file.
Price: $149/month, 14-day free trial.
Final Word: If you have the money, it’s really the best solution for one-on-one moderated testing. Just be sure to have a backup for those cases when it falls short: testing non-PC users, supporting non-PC observers, dialing internationally, and so on.
GoToMeeting is a popular Web conferencing service similar to Microsoft’s NetMeeting (see Figure 8-3). It’s a little quicker for users to set up; requires only a lightweight, temporary plug-in download for all attendees (no .exe file); comes with free conference calling; and supports observing.
Figure 8-3. GoToMeeting: popular Web conferencing with built-in recording, observer and chat support, free conference calling, and remote keyboard and mouse control.
Gives keyboard and mouse control. This feature not only enables you to view your participants’ screen, but also allows your participants to see and even control your screen. This feature enables you to use two remote research techniques we call “reverse screen sharing” and “remote access,” which we’ll describe in Chapter 9.
Free conference calling with TotalAudio. Although it doesn’t boast 100% integrated calling like UserVue does, GoToMeeting does offer a free VoIP conference call system called “Total Audio” to subscribers. The big problem with this solution, however, is that it requires users to make a long-distance call to access the conference number. The solution to this is to use a two-line phone to get the users into the call (see the following sidebar).
Choice of monitor and window to share. Your participants can choose which monitor on their computer to share, in case they’re using multiple monitors. (Can’t share both, unfortunately.) They can also share just a specific window, if there’s a privacy issue around sharing the whole window.
Pause screen sharing. You can put the screen sharing “on hold” for whatever reason, without having to disable and reconfigure it entirely.
High compatibility. We’ve had a lot of success using UserVue with most participant computers, regardless of firewall settings or whether they’re international. It also allows for observers using a Mac (but not participants, sadly).
Things to note:
Unlike in UserVue, in GoToMeeting the calling is performed separately from the screen sharing, so you can use it to test internationally.
Also unlike in UserVue, in GoToMeeting you don’t need to start a new session for each new user; you can just keep the same meeting window open and let it sit between sessions.
Although there is chat functionality in GoToMeeting, neither you nor the observers should use it, since your user (who is just another meeting attendee) will be able to read everything, which is distracting to say the least. Use an IM client instead to chat with observers.
Observers also need to mute their phones so that they won’t be overheard by the user during the session. If you’re using Total Audio, you can manually mute them yourself if you need to.
Web site: www.gotomeeting.com (to register/host a meeting), http://joingotomeeting.com (to join a meeting).
Pros: Free audio conferencing, recording, ability to give control of keyboard and mouse.
Cons: Observer chat is visible to the participants; the audio conferencing requires users to dial long distance.
Requirements: Users must have a Java-enabled browser; Firefox or Internet Explorer 6+ should work fine. Fully PC compatible; Mac compatible for moderator and observers only (participant must use PC).
Price: $50/month or $468/year, with free 30-day trial.
Final Word: Lacks the nice UX-specific features of UserVue but is more than suitable for remote research, and at one-third the cost of UserVue, we ain’t complaining.
Here’s a departure from the rest of the screen sharing pack: the online service LiveLook is a 100% browser-based screen sharing solution (see Figure 8-4). There’s no software and no need for anyone (participant, moderator, observers) to download or run anything, which means that it can get around most spyware or firewall blocks. On top of that, it can run on any computer with a JavaScript-enabled Web browser, which is basically all of them (Mac, PC, or Linux). It’s also pretty good for testing on a slow connection, requiring relatively low bandwidth.
Note
HAVE BACKUP SCREEN SHARING TOOLS
Having more than one screen sharing solution on hand is ideal. You’ll want to have options in case one fails or doesn’t work with the participant’s setup, especially if recruits are scarce. LiveLook is a great fallback because of its platform compatibility and ease of setup, but the lack of security and the necessity of giving out account info to strangers might be a deal breaker for you. Either make sure to have backups, or double-check in advance that your users will be able to use your screen sharing solution. Also, since no tool is ideal for every situation, have a few tools on hand that will cover all your bases. We have UserVue, LiveLook, GoToMeeting, and Adobe Connect all at the ready.
Neat Features:
Completely browser based. Like we said, the fact that LiveLook requires no installation or plug-in download makes it the screen sharing option most likely to work on any computer. You can get around all but the most heavy-duty firewalls and spyware blockers.
Really quick to set up. All your user does is visit a link and give you a number; setup takes barely a minute.
Things to note:
To allow your users to share their screen, you’ll have to use LiveLook’s “LiveLook Assistant” product. This tool allows you to give a link to your users, which your users click to obtain a session number and begin a screen sharing session. The users give you the session number, allowing you to join the session and view their screen.
Like in UserVue, in LiveLook you need to begin a unique session for every participant.
The LiveLook Web site claims that the screen sharing is secure, but we’re not really sure how that works, so caveat emptor.
Unlike Adobe Connect and GoToMeeting, LiveLook doesn’t resize the participant’s screen resolution to fit on your screen; instead, the browser adds scroll bars so that you can scroll around to view different parts of the screen. This limitation can be bad if you need to see the entire screen the whole time, and your participant’s resolution is larger than yours.
Since there’s no built-in sound, you’ll need to take measures to enable observers to hear the sessions. Use the “Two-Line Phone” technique described earlier in the GoToMeeting sidebar.
Right now only annual subscriptions are offered for the LiveLook Assistant product. The standard LiveLook product offers per-minute and monthly subscriptions as well, but it requires you to give your users your LiveLook login information to be able to share their screen. If you don’t mind giving out that info, the per-minute and monthly accounts are an option. (The sales rep we spoke to mentioned that the company is considering other pricing options but has “nothing solid” yet.)
Web site: www.livelook.com
Pros: The only 100% browser-based, no-download solution we’re aware of. Compatible with most operating systems. Very quick to set up. Supports observers.
Cons: No integrated sound, recording, observing—just straight-up, one-on-one screen sharing.
Requirements: Users must have a Java-enabled browser; Firefox, Safari, or Internet Explorer 6+ should work fine.
Price: $360/year for a single account license; $900/year for a “concurrent user” license, which lets you have multiple (6–7) people using the account at the same time.
Final Word: Cheap and bare bones, with a high level of accessibility and compatibility.
Here’s a list of screen sharing tools and services that we’re less familiar with, but we’re sure they’re just as lovely as the ones we’ve listed in the preceding sections.
Cisco WebEx. Similar to GoToMeeting, it’s a Web conferencing/online meeting tool that can be used for remote screen sharing. Compatible with Windows, Mac, Linux, Solaris (?), HPUX (??) and AIX (??!?!). $69/month, $708/year. www.webex.com
Glance. Similar to LiveLook, a browser-based screen sharing solution. Allows for remote access, like GoToMeeting. Up to 100 people in a session at once. Mac and PC compatible. $9.95/day, $49.95/month, $249.00/month (corporate). www.glance.net
Skype. Skype users can view each other’s screens in addition to video chat. Requires both parties to have Skype installed and registered. Screen sharing feature is currently Mac only. www.skype.com
iChat for OS X Leopard. On Mac OS X 10.5 and up, there’s a feature in iChat that allows two iChat participants to video chat, share screens, and give remote desktop access. Comes standard with Mac OS X 10.5 (Leopard) and above. http://www.apple.com/macosx/what-is-macosx/ichat.html
QualVu. This is a brand new one, which we’ve never used before. Uses users’ webcams to run “online video qualitative research” studies, in which you can video chat with panel users, watch them interact with and respond to your interface, and take notes and instantly create highlight clips. Sounds weird and interesting. www.qualvu.com
Yugma. Web conferencing service with lots of nice Pro features: real-time whiteboard and annotation collaboration tools, building in recording, free teleconferencing, file sharing. Integrates with Skype. Installation required for both hosting and joining a session. Basic screen sharing with up to 20 participants is free; Pro is $14.95 / month. www.yugma.com
Yuuguu. Screen sharing and collaboration software. Can import IM contacts from other services (AIM, GChat, Skype) Requires download and install; Mac, Linux, and PC compatible. www.yuuguu.com
Note
YOU CAN’T SHARE MULTIPLE MONITORS YET
As far as we’re aware, no screen sharing tool out there allows you to see more than one of the participants’ monitors at once, except for VNC, which is probably not an ideal remote research tool, since it takes a long time to set up and is pretty invasive to install on participants’ computers. A few tools, such as GoToMeeting and Adobe Connect, allow your participants to select which monitor they’ll share.
Recording your sessions is a comparatively simple and straightforward consideration. If you’re using a screen sharing tool that doesn’t provide recording, you’ll need two things: a way to get the audio and video of your session both going through your computer and a program that captures the audio and video output of your computer. The recording software often takes a bit of processing power, so make sure your computer is swanky enough to run it alongside whatever else you have running.
Our recording solution of choice, Camtasia Studio (by TechSmith, makers of UserVue), is a simple tool that can record the video and audio output on your computer in just a few clicks (see Figure 8-5). We like it because it gives you the option to record just a portion of your screen or just a single window on your computer, which helps keep the recording file size trim.
Figure 8-5. Camtasia Studio: record computer video and audio, full screen, or portions of the screen.
Things to note:
The trickiest thing about using Camtasia is making sure the audio is being recorded. If you’re using a voice chat service like Skype to chat with your users through your computer, the audio should automatically be picked up by Camtasia. Otherwise, you’ll need to route the signal from your phone to your computer using a phone tap and amplifier. Once you’ve got it hooked up, go to the Tools menu, select Options, click the Audio tab, and you’ll be able to test if the telephone sound is going through or not. If it’s not, you probably need to change the audio input device setting, which is also under the Audio tab.
Be sure to set the video files to save as AVI, which is compatible with most media players, as opposed to the default proprietary CAMREC format, which plays on almost nothing. (To do this, go to Tools > Options > Capture tab > Save as .avi.)
The Mac version has a slightly different set of options, so check the Web site to make sure it has what you need.
There’s also an Automatic file name option (Tools > Options > Capture > File Name Options) that will automatically add a prefix and a number to the filename of each video file that gets produced. We recommend using the study name as the prefix; it provides an easy way to numerically organize the session recordings. So, for example, if the prefix is RemoteStudy, your first recording would come out as “RemoteStudy-1.avi.,” the second would be “RemoteStudy-2.avi,” and so on.
Be careful not to obscure the window you’re recording with any other windows.
We’ve never dealt with this before, but some people have reported problems where after four or five sessions, sometimes Camtasia wouldn’t record the next one properly, erroneously citing a lack of hard drive space. Copying the files onto an external hard drive after each session seemed to prevent this problem.
We’ve found that there’s a bug in older versions of Camtasia (up to at least version 3), which prevents the audio from merging into the recording if the file size grows past a gigabyte, which makes the file effectively useless.
As always, extensively test the tool with real telephone conversations, making sure both sides of the conversation are being picked up, the volume level of the recording is clear, and the video file works with your media player.
Web site: www.techsmith.com/camtasia.asp
Pros: Reliable, simple.
Cons: Older versions have been buggy.
Requirements: Windows version: Microsoft Windows XP or later, Microsoft DirectX 9 or later, 1.0 GHz processor (2.0 GHz recommended), 500MB RAM minimum (2GB recommended), 115MB of hard disk space for installation.
Mac version: Mac OS X v10.5.6 or later, Mac computer with an Intel processor, Quartz Extreme support, 1GB of RAM (2GB recommended), ~4GB of available disk space, QuickTime 7.5.5 or later, CD drive required for installation.
Price: $299 for PC; $149 for Mac.
Final Word: A bit pricey for software, but you’ll have to buy it only once, and it’ll cover you for screen recording until the very end.
Note
MAKE SPACE FOR RECORDINGS
Make sure that you have enough room on your hard drive to store the recordings you generate during the sessions. Depending on the recording quality, a 40-minute session recording can be upwards of a gigabyte. Another option, if you’re low on space, is to get an external hard drive to transfer over each recording file after each session. And be sure to securely back up those files whenever you get the chance.
Turns out that some wonderful intrepid programmer out there has created a 100% free, open source feature clone of Camtasia called CamStudio (see Figure 8-6). All the core functionality of Camtasia is there: the option to record a portion of the screen on a specific window, the ability to capture audio and video, and so on.
Web site: http://camstudio.org
Pros: Free; adjustable recording quality; able to convert AVI files to more compact SWF flash video.
Cons: PC only; as a free product, there’s no professional customer support, but there is a support forum on the Web site.
Requirements: Windows XP or later; system requirements not listed but probably the same as Camtasia.
Price: Zero (but pitch in a few bucks, eh? They earned it.)
Final Word: The price is right. Just make sure it works perfectly for you because you won’t have anyone to call if it breaks.
And now, for you Mac users: shinywhitebox’s iShowU HD is a very full-featured recording solution, with tons of nice video and audio recording options, options to record portions of the screen or windows, and customizable hotkeys (see Figure 8-7). A Pro version of the software has even more options, such as a “Low CPU mode” to help avoid slowing down your computer.
Things to note:
Set the Mouse Mode menu to Fixed. Otherwise, you’ll notice that the recording follows your mouse around on the screen.
Also be sure to disable the Keys switch in the toolbar, or else everything you type during the session will be plastered across the video.
If you’re using a phone tap to route audio to the computer, go to Advanced > Sound. Make sure the “Record sound from input device” box is checked and the menu is set to Built-in Audio. If you’re talking through the computer with Skype or some other voice chat program, check “Record audio from applications.”
iShowU HD is set by default to stop the recording automatically when your hard drive has only 200MB remaining. You can change this setting in the preferences.
As with the other recording solutions, be careful not to obscure the window you’re recording with any other windows.
Web site: www.shinywhitebox.com/ishowuhd/main.html
Pros: Feature rich; lots of video and audio rendering options; adjustable video quality; Pro version has Low CPU mode; cheap.
Cons: Tricky default settings; lots of options make it slightly harder to learn.
Requirements: Mac OS X 10.5 Leopard or later. (An earlier version of the product, iShowU Classic, is still available for Tiger 10.4.) System requirements not listed.
Price: $29.95 for standard, $59.95 for Pro, which includes extra features (watermarking, key recording, audio mixing, low CPU mode, Final Cut compatibility).
Final Word: If you’re going to record on the Mac, we wouldn’t recommend anything else.
Adobe Captivate. Expensive, many-featured screen recording tool. PC only. $249. www.adobe.com/products/captivate
Quicktime Pro. Mac OS X’s video player also has audio/video recording capabilities, as well as some basic video trimming features. Mac and PC. $29.99. www.apple.com/quicktime/pro
Screenflow. Records screen, webcam, mic, and computer audio simultaneously. Also includes editing and overdubbing tools. Can export to WMV with an additional plug-in ($49). Mac 10.5+ only. $99. www.telestream.net/screen-flow
Snapz Pro X. Records screen and audio to QuickTime format. Mac only. $69. www.ambrosiasw.com/utilities/snapzprox
WMCapture. Basic screen and audio recording. PC only. $39.95. http://wmrecorder.com/wm_capture.php
ZDSoft Screen Recorder. Screen and audio recording, effective at recording anything that uses special rendering, like video playback and games. PC only. $39. www.zdsoft.com/products.html
For even more tools, visit the screen capture page at the All Streaming Media Web site: http://all-streaming-media.com/record-video-from-screen.
Note
USE AS FEW TOOLS AS POSSIBLE
The more tools you use to test, the more likely one of them is going to mess up, and the harder it will be to figure out what’s going wrong. Using Skype for voice chat with a participant, a screen recorder, an audio recorder, an IM client for observer chat, a stopwatch to keep time, an Excel spreadsheet for notes, etc., you’re going to be at the outer limits of professional reliability. If possible, try to use a setup that uses as few tools as necessary to get the job done, and always have backup alternatives on hand in case any part of the setup fails.
When it comes to automated research for a given task (card sorting, surveys, etc.), most tools and services you use will be relatively similar, with only a few features available or missing. We already covered the methods for using automated services in Chapter 6, so in this section we’ll just provide you with a wealth of services to check out, with features, pricing info, and links to relevant Web sites. We’ve divided them up here into the same categories as in Chapter 6.
Note: These are not exhaustive feature lists; they’re summary paragraphs emphasizing noteworthy features of each tool. And we probably missed a few tools; we’ll be adding more to http://remoteusability.com as we discover them.
Click Density. A UK-based service offering interactive heatmaps, which you can segment and filter along several variables (like user path, browser, screen size, etc.) and export into presentations. Requires a line of code to be inserted into your Web site. Free basic account, $5/month (10,000 clicks, 1 site), $20 (45,000 clicks, 10 sites), $100 (250,000 clicks, 25 sites), $200 (1,000,000 clicks, 50 sites), $400 (5,000,000 clicks, unlimited sites). www.clickdensity.com
ClickHeat by LabsMedia. This is a humble open source heatmap program, and as such, it’s pretty bare bones on features—just a heatmap overlay that goes over an image of your Web site. It also requires much more tech prowess than the commercial solutions. Still, if free is what you want, this is it. www.labsmedia.com/clickheat/index.html
ClickTale. Records user interaction with forms, links, and other elements on your Web site (see Figure 8-8). Generates heatmaps of places users are clicking and scrolling, how long users spend, and how many page views each section gets. This tool can also record real-time movies of user behavior for later playback. There’s no task elicitation involved; it just tracks natural user behavior. Requires a few lines of code to be inserted into your Web site. Free basic account, $99/month (email support, 10 domains, 20,000 page views), $290 (priority support, 25 domains, 80,000 page views), $790 (phone support, unlimited domains, 240,000 page views). www.clicktale.com
Figure 8-8. ClickTale’s “Attention” heatmap, which illustrates where users click and how far they scroll down the page.
Clixpy. Tracks mouse movement, scrolling, and form input, capturing it in video format. Requires pasting a few lines of JavaScript into your site code to install. First 10 video captures free; 100 for $5, 200 for $10, 600 for $20, 1,000 for $30. www.clixpy.com
CrazyEgg. CrazyEgg has a bunch of tools for visualizing the clicks you get on different elements of your site, including heatmaps, colorful clickmaps, and site overlays. You can share the results with others using a custom link, export the results to Excel, and receive notifications about the status of your study via email and RSS. Requires one line of code to be inserted into your Web site; $9/month (10,000 reports, 10 pages), $19/month (25,000 reports, 20 pages), $49/month (100,000 reports, 50 pages), $99/month (250,000 reports, 100 pages). www.crazyegg.com
Chalkmark by Optimal Workshop. Allows users to complete tasks on static images, providing heatmap feedback for prototypes and mockups. Free account provides surveys of up to 3 tasks; $109/month, $559/year. www.optimalworkshop.com/chalkmark.htm
Google Analytics. Most people don’t think of this service as a “research tool,” but that’s exactly what it is. The only difference is that none of the tasks are elicited. It’s a great tool for slicing and dicing your Web traffic data, and it has functionality that allows you to also track which paths users are taking through your site, which is great for monitoring “funnel tasks” (linear, step-by-step tasks, going through a checkout process, registering for an account, etc). Best of all, it’s free. www.google.com/analytics
LEOTrace by SirValUse. Analytics service and tool providing log data, mouse movements and clicks, and data reports such as user behavior video and heatmaps. SirValUse offers full-service consulting as well as licensing its tool. Price upon request. www.leotrace.com
Loop11. Pretty straightforward task elicitation: a browser bar prompts users to perform a task on any interface that can be displayed in HTML, and the users indicate whenever they have either completed the task or have abandoned it (i.e., given up). All interactions captured into real-time reports. Automated analysis includes task completion rate, time on task, first click, clickstream, most frequent success/fail points, etc. Available in 40 languages, so international testing is a go. It’s currently only in private beta, which means you can register by invite only. www.loop11.com
m-pathy. A remote usability tool that tracks mouse movements and clicks without installing anything on the user’s computer. It’s in German, so we don’t actually know what the heck the Web site is saying, but from the looks of it, it appears to offer heatmaps of clicks and mouse movement and also mouse tracking. http://m-pathy.com
OpenHallway. Prompts users with instructions and tasks and then records their behavior on the site and spoken think-aloud comments into a sharable streaming video recording. Sessions are limited to 10 minutes. No software to install for participants or researchers. Firefox, IE 7+ and Safari supported; participants must have Java 1.5+ enabled. Recruit your own participants by sending them a link to participate. Free 30-day trial; $49/month for 3 hours of recording storage, $99/month for 10 hours, $199/month for 30 hours and the ability to download videos. www.openhallway.com
RelevantView. RelevantView is a user research consulting company that also offers its automated research tools for self-use. Its task elicitation tools allow you to track time on task, task completion, and click paths. These tools also allow you to do competitive research and test prototype sites. Pricing upon request. www.relevantview.com
SMT (Simple Mouse Tracking) by Luis Leiva. An open source project, providing mouse movement and click tracking functionality; it also tracks scroll bar, mouse wheel, and highlighting activity. Like ClickTale, it allows you to replay user behaviors in real-time, and the activities can also be exported to XML. If you’re tech-savvy, you can customize it. It’s free, licensed under Creative Commons. http://smt.speedzinemedia.com/smt
Testled by ’80s Business Guys. Record user behavior with no installs or Javascript required. Participants can use any browser or OS. Currently in invite-only beta. http://testled.com
Usabilla by Usabilla BV. Basic task elicitation on Web sites or static images; you have to provide your own recruits. Data are visualized as “datapoints, heatmaps, cluster clouds, notes.” Also includes screen annotation. Currently in free beta. www.usabilla.com
Usability Exchange by KnowFaster.com. Automated task elicitation for testing disabled users (blind, dyslexic, etc.); disabled participant pool provided by Usability Exchange. 5 users and 4 tasks = £299, 10 users and 4 tasks = £599+, custom number of users and tasks = at least £299, simple questionnaire = £199 www.usabilityexchange.com
UserTesting.com. Task elicitation in which users’ spoken think-aloud comments are captured and synced with videos of their Web site behavior as they perform tasks of your choosing. Videos are downloadable, editable, embeddable, and can be annotated with comments. You can also see users’ demographics and system info. Users are recruited by UserTesting.com from a preselected panel; $29/user. www.usertesting.com
UZ Self-Serve Edition by UserZoom. UserZoom, an automated research company, has its remote research tools available for do-it-yourself studies. This Web-based tool allows you to manage multiple automated research projects; gather clickstream data; prompt users to perform Web site tasks, card sorts, and surveys; and recruit users from a panel, your email contacts, or your own Web site. It also offers full-service UX testing. The tools are currently available only by yearly subscriptions; pricing available on request. www.userzoom.com
WebEffective by Keynote. Another Web-based tool for conducting in-depth customer experience, branding, and market research studies. Users answer survey questions and complete tasks in pop-up windows, with no download required. Keynote employs a panel to provide quantitative clickstream and behavioral data, survey feedback, and task completion data. You can also intercept users from your own Web site. Studies are available in 17 languages. Pricing available on request. www.keynote.com
Webnographer by FeraLabs. Full-service automated research company that will soon be releasing self-service tools. Tracks clickstream data, allows you to present questionnaires to the user. No code needs to be modified on the site being tested, and no user download necessary. All you do is send participants a link. Cross-platform and cross-browser compatible. Full-service research is $1,200/day; average time is about 1.5 days but can range from 1–5 days, depending on the complexity of the interface and the number of participants. Can recruit from your Web site, Webnographer panel, or your email contacts. Pricing to be announced. www.webnographer.com
We repeat: Surveys aren’t really within the domain of remote UX research; mostly, they’re a tool you use to gather opinions, not record behavior. But since so many of us are tempted to put up surveys, we’re including some of the best tools here in the hope that you’ll use them only sparingly, to accompany your behavioral research.
HaveASec. This is one of the first entries in a sure-to-be robust ecosystem of mobile survey research tools, targeted at iPhone developers gathering feedback for their apps, but it’s compatible with some other devices as well. Developers can either embed the survey within their iPhone app or provide a link somewhere within the app that takes users to a mobile-optimized Web site containing the survey. We see great promise in using apps like these to live-recruit participants for mobile remote research studies, which we discuss briefly in the next chapter. Free basic account, $50/month for premium (up to 1,000 responses per survey), $500/year for platinum (up to 5,000 responses per survey). www.haveasec.com
PollDaddy. A nice, cleanly designed survey tool with 11 question types, conditional branching, full foreign language support, multipage surveys, and the ability to customize surveys with HTML. Charges by the response, rather than on a subscription basis. Free basic account, $200/year (1,000 responses), $899/year (10,000 responses). www.polldaddy.com
RelevantView. In addition to task elicitation, you’re able to design your own surveys with branching logic and view and export results in summary, raw, and cross-tabulated formats. Surveys can be designed in all non-Asian languages. Pricing upon request. www.relevantview.com
SurveyGizmo. Skip logic and survey branching, browser redirects, a handful of question types (see Figure 8-9). Tools to filter, cross-tabulate responses, and remove duplicate entries. You can export data to text and Excel and export reports to Excel, Word, and PDF. Reports can be visualized in three kinds of charts. The survey code comes in JavaScript, HTML, and iFrame, so it can be embedded into both Web sites and email. Twenty types of questions. You can manage and track email invitations to participants, as well as send out reminders and follow-up emails. Template surveys available for beginners. Like PollDaddy, it charges by the number of responses, and it also offers discounts to nonprofit companies. Free basic account and 14-day trial; $19/month (1,000 responses, 5,000 invites), $49/month (5,000 responses, 10,000 invites), $159/month (50,000 responses, 10,000 invites), $599 / month (1,000,000 responses, 100,000 invites). www.surveygizmo.com
Figure 8-9. Sample SurveyGizmo survey. Remember, surveys give opinion-based feedback, not behavioral feedback!
SurveyMonkey. One of the most popular online survey tools out there. When designing a survey, you can choose from 20 kinds of form questions, support for all languages, response error checking (requiring users to input valid answers). This tool has a built-in manager for sending out and keeping track of survey invitations, as well as a pop-up feature to get people to fill out your survey. You can filter and cross-tabulate the results of the survey, share them with others by sending them a link, and export them to Excel, XML, and HTML format. Free basic account, $19.95/month (monthly pro), $200.00/year (annual pro). www.surveymonkey.com
UZ Self-Serve Edition. Takes an interesting approach to surveys: rather than displaying a pop-up when users arrive at the Web site, it makes a browser window that users see after closing down the site, so the survey is more like an exit survey. This allows you to see whether users are able to complete whatever tasks they were on the site to perform, as well as assess user satisfaction. Pricing upon request. www.userzoom.com
WebEffective. Also offers basic survey tools, in addition to the task elicitation tools listed previously. www.keynote.com
Zoomerang. A survey tool with all the usual nice features: skip/branching logic, exporting, and the ability to post the survey to your site. It also maintains a panel of survey respondents if you don’t feel like recruiting from your own Web site. With the premium account, you’re able to create mobile surveys that users can respond to on their mobile devices. Free basic account, $199/year (pro), $599 (premium). www.zoomerang.com
For even more commentary on what to look for in online survey tools, check out Idealware’s “A Few Good Online Survey Tools” article: www.idealware.org/articles/fgt_online_surveys.php
OptimalSort by Optimal Usability. One of the first and foremost online card sorting services, OptimalSort uses a simple drag-and-drop interface to allow participants to sort cards into groups and also make additional comments in a text field at the bottom of the screen (see Figure 8-10). You’re able to customize the study introduction, a preliminary questionnaire, the study instructions, the cards and categories, a debriefing “Thank You” message, and you can ask for users’ contact info for incentive payment. Helpful study templates are available for beginners. The results are loaded into an exportable table, which you can organize by card and category labels, participants, and comments. You can also view the individual users’ raw data. The nice folks at OptimalSort offer free basic advice and support for using their tool (provided it “doesn’t take a lot of their time”), and also paid consulting services. Free basic account, $109 (30-day subscription), $199 (90 days), $559 (annual). www.optimalsort.com
Figure 8-10. Optimal Sort: a simple drag-and-drop online card sorting tool, with templates and exportable data (which can be used with the Card Sort Cluster Analysis Tool).
RelevantView. Allows users to drag and drop pictures, text, symbols, diagrams, and images. The Web site seems to indicate that it supports only the closed-sort method. Pricing upon request. www.relevantview.com
WebSort. Like OptimalSort, WebSort gives users a simple drag-and-drop interface to sort cards, create groups, and comment. It allows you to customize the instructions and thank-you message, share the results with others, import studies from OptimalSort, and view the results of the study in tree diagram form in addition to table form, with the ability to export them to an Excel file. Free limited account, $79 (single study), $299 (5 studies), $199/month (unlimited studies). http://websort.net
UZ Self-Serve Edition. Also packs a card sorting tool, allowing users to sort up to 100 cards into up to 12 categories. You can ask context questions to find out why users have decided to categorize things the way they do, and you can also include other UserZoom automated testing elements besides the card sort in the same study. The data can be viewed either raw, in a matrix, or in a dendrogram. Pricing upon request. www.userzoom.com
Card Sort Cluster Analysis Tool 1.0. An online tool you can use to take exported results from OptimalSort and organize them instantly into an affinity chart. Handy! Free! www.userpoint.fi/card_sort_cluster_analysis_tool
Ethnio. Sorry, we couldn’t resist hyping our own product again. We think it’s worth another mention here, not only because it’s the only tool out there specifically made for live recruiting, but also because it has a hidden benefit: it allows you to make an extract of every visitor who’s filled out the recruiting screener, producing an interesting bit of quantitative data, even if you’re just recruiting for small-sample moderated research. Since Ethnio exports the data into a spreadsheet, you can organize that data later by response. Note that since respondents are self-selected from your Web site (rather than randomly sampled), these data aren’t up to the standards of rigorous statistical analysis; still, they’re interesting bits of data to include along with your qualitative findings. Requires a line of JavaScript to be inserted into your Web site. First 20 recruits free; $400/200 recruits, $2,000/2,000 recruits. http://ethnio.com
Google Docs (Forms). Allows you to make very simple forms for surveys and other online feedback. Building and editing forms are a breeze, and you can send a direct link to the forms or embed them into your site with a bit of code. The results are loaded automatically into a Google spreadsheet, nicely formatted and timestamped. The reason we wouldn’t recommend this tool for recruiting or sensitive data gathering is that Google technically possesses all the data, and we haven’t looked into how secure or private those data are. Maybe they’re fine! At any rate, this tool is 100% free with a free Google account, and it seems the obvious choice for gathering nonpersonal feedback automatically. http://docs.google.com
iMarkIt by ITracks. A concept testing tool for screen annotation. Participants can mark up video, Web sites, images, and text with emoticons, arrows, colors, and text boxes. Seems as though participants are recruited from a panel, but we couldn’t tell. Pricing upon request. www.itracks.com/Products/ConceptTestingiMarkIt.aspx
TreeJack (Beta) by Optimal Workshop. This is sort of but not really like a task elicitation tool. Instead of having users perform tasks on the actual Web site, you load your Web site’s navigation structure into TreeJack and have users perform tasks on a hierarchical tree diagram that represents your site structure. For example, you can ask users where in the navigation they would find “information about usability research,” and they navigate the site’s structure, indicating which page they would expect to find it on. This is a handy tool for information architects who want to test the intuitiveness of their navigation structure. Currently in beta, so it’s free. www.optimalworkshop.com/treejack.htm
Wufoo. We covered this all-purpose automatic form-building tool in Chapter 3, showing you how you could use it to recruit and obtain consent, but it can also be used to develop surveys that can be embedded right into your Web site. You can search through the entries (i.e., user responses) and export them to Excel and text formats. If you’re using this tool to recruit from your Web site anyway, it seems like a logical choice for basic surveys. Free basic account, $9.95/month (10 forms, 500 entries, 200MB storage), $24.95/month (unlimited forms, 3,000 entries, 500MB storage), $69.95/month (unlimited form, 15,000 entries, 1GB storage), $199.95/month (unlimited forms, 100,000 entries, 3GB storage). http://wufoo.com
The most important factors to consider in a screen sharing tool are cost, security, OS compatibility, observing and recording features, ease of setup, and firewall/spyware compatibility.
Your recording solution shouldn’t slow down your computer, and it should output to a standard video format and record audio and video. Remember to have enough hard drive space to store recordings.
With automated tools, pay attention to pricing and cost, setup requirements (for both you and the user), what data can be captured, qualitative feedback options, and results formatting.