In Voice User Interface Design, Cohen, Giangola, and Balogh define persona as follows:

Humans are wired to anthropomorphize. A photo of a bathtub with holes where the faucet used to be becomes a surprised face (Figure 3-1). We talk to our pets as if they were humans.

Figure 3-1. Surprised bathtub (photo by author)

Trust that your users will be doing the same thing with your VUI. Whether you intended them to do so, users will ascribe personality traits to it. The best strategy is to decide on the personality yourself, while designing, rather than leave it up to chance.

When you’re using an avatar, it becomes even more crucial to think about the personality behind it. When you create a visual representation, people are even more likely to see personality traits. Begin with personality traits and then design the visuals to match. When thinking about personality, here are some questions to ask:

Some virtual assistants, such as Cortana, let you access a log of everything they know about you to edit it. According to the Business Insider article “Why Microsoft Doesn’t Want Its Digital Assistant, Cortana, to Sound Too Human,”^[16] this “breaks the illusion that Cortana is a human, which is just fine by Microsoft, while also making it a more useful tool.”

The personality of your VUI will affect its behavior across the board: how it asks questions, how it copes with errors, and how it provides assistance.

Cortana’s writers spent a lot of time thinking about her personality:

Cortana’s writers even have a strategy for dealing with the inevitable harassment that some users will give a female virtual assistant. As Cortana writer Deborah Harrison shared in an interview with this book’s author:

On the flip side of persona design, make sure you don’t go too overboard with personality traits when they’re not needed. If you’re designing an entertainment app, you may want to go full bore with witty dialog and one-liners (á la, You Don’t Know Jack), but imagine if Google suddenly began imbuing that kind of behavior into its virtual assistant. Consistency in your persona design is important: as Katherine Isbister and Clifford Nass note, “Consistency in others allows people to predict what will happen when they engage with them.”^[18]

If you’re designing an app for a particular set of users, you can be more free with the personality. Some people won’t like it, but some people will love it. If you’re designing something more universal, be more cautious. In this case, you might have a subtler personality that is neither loved nor hated.

You might want to give your users choices for the avatar or voice used in your app. It is nice to give people options, but you can’t always swap out an avatar or voice and leave everything else the same. Different looks and different voices imply different personalities, and the interactions different personalities have will not be identical. Ideally, rather than offering a different voice or different avatar, you offer different personas, which then contain different components, including visual and audio. Having a mismatched persona and avatar can cause inconsistencies and lead to distrust.

The book Voice User Interface Design by Cohen, Giangola, and Balogh is a great reference for learning more about persona and VUIs.

One of the most common use cases for which designers do create avatars is virtual assistants. Websites often use a photo or an avatar (Figure 3-2) for their customer service chatbot (even those who do not speak and cannot be spoken to). Many mobile virtual assistants also have an avatar component.

Figure 3-2. Examples of websites with an avatar (left, Jetstar—not animated; right, CodeBaby—animated)

However, many of the most prominent VUI assistants, including Google, Siri, Hound, and Cortana, have no avatar. Some do have simple visual cues, such as a glowing circle. (HAL 9000, a fictitious but very interactive virtual assistant from the 1968 movie 2001: A Space Odyssey, had a glowing red light.) Avatars are not essential to have a successful voice interaction. VUIs can still provide empathy and important visual feedback without using a picture.

The designers of Cortana made a conscious decision to stay away from an anthropomorphic representation. Deborah Harrison, Cortana editorial writer at Microsoft, says:

A good avatar takes more work, both on the design and development. There are third parties out there that will provide an avatar for you to integrate with, or you can invest in building your own.

In “Sweet Talking Your Computer,” Clifford Nass notes:

Microsoft’s assistant, Clippy, was a failure, but it came with some important lessons. Clippy (Figure 3-3) was oblivious to context. In one cartoon making fun of Clippy, someone has started a letter with “Dear World, I just can’t take it any more” and Clippy has popped up to say “It looks like you’re writing a suicide letter. Let me help with that.”

Figure 3-3. Microsoft Office assistant, Clippy

If your VUI is tone-deaf, the user will notice. Another Nass experiment had subjects “drive” a car simulator and used two different voices: one that was upbeat, and one that was morose. Sad drivers hearing the happy voice had approximately twice as many accidents as those who heard the sad voice. Happy drivers had fewer accidents and paid more attention to the road when they heard the happy voice. When voices matched mood, drivers also reported enjoying driving more and that they liked the voice more.

If your app is just looking up search results and setting timers, emotional tone is not as important. But if the app deals with more sensitive topics such as mood or health, be sure to recognize these instances and handle them appropriately. It’s better to have a neutral response than to have one that is inappropriately happy or sad. Making your VUI appear aware of what the user has said (or done) is crucial.

In a study done by Ann Thyme-Gobbel during her time as head of UX/UI experience and design at healthcare company 22otters (and presented at the Mobile Voice 2016 conference), 72 participants were shown a video with the same instruction done in one of five ways (Figure 3-4 shows some of these):

Figure 3-4. Photo, static avatar, illustration

Participants heard the same prompts (recorded by a person, not text-to-speech) delivered in one of those formats, and then were asked which one they preferred. Generally speaking, most people preferred the animated illustration. When asked specifically what they thought of the moving avatar, those over the age of 40 rated it the highest.

Subjects appreciated variation—text, for example, is great for listing medications; illustrations are useful for instructions on using a device; and avatars are good for building rapport.

The study began with a static avatar introducing each topic, which as Thyme-Gobbel said, “grounded the voice and made a nice connector throughout the app.”

Though it was a small study, it’s a reminder that avatars are not appropriate all the time. Whenever possible, prototype your VUI early on to discover in advance whether people find your avatar engaging—or creepy.

When should you use an avatar? This section outlines where an avatar or animated character can enhance your VUI and keep users more engaged.

Storytelling

A great time to use an avatar/character is for storytelling and engagement. If you want to draw your user into your world, whether it’s for a game, or for educational purposes, having a character is a great way to do so.

An example of this is from the ToyTalk children’s entertainment app SpeakaZoo. In each scene, an animated animal talks to the child, and when it’s the child’s turn to talk, a microphone button lights up (Figure 3-5).

Figure 3-5. ToyTalk’s SpeakaZoo

Based on the child’s responses, the animated character responds in different ways. In this instance, the app does not even use that much speech recognition—it often reacts the same regardless of what the child says. It still creates an immersive experience, although with repeated exposure, kids do catch on that the character is always doing or saying the same thing.

Another example of an interactive game with an avatar is Merly, from Botanic.io (Figure 3-6). Mark Stephen Meadows describes Merly as follows:

Merly, a character that provides backstory on a kid’s mystery fable, answered questions that the story brought up, provided solutions on how to play the game, and stitched together several elements of the story that were not clear on first read. It was a new way to read, and someone that, like the Greek chorus in plays, added meta-commentary by breaking the fourth wall. Narrative bots have to break that fourth wall and what that means, from an interactive narrative point of view, is that they will be an increasingly important part of storytelling as VR eclipses passive narrative media like movies and television.

Figure 3-6. Merly from Botanic.io

Teamwork

Another way an avatar can add to your user’s experience is that it allows the user to have a teammate/collaborator when completing tasks. When you speak to Amazon Echo’s Alexa, for example, she’s the one completing the task; you’re making a request.

With an avatar, the VUI can be a separate entity from the system or the company with which the user is interacting. For example, if your doctor has requested that you take your blood pressure every day, a nurse avatar can assist you with that, so together you accomplish the required task someone else has requested.

SILVIA’s Gracie prototype of an avatar (Figure 3-7) allows the user to sing along with her (she’s an animated character). When you want to talk to Gracie, you tap the screen.

Figure 3-7. Gracie lets you sing along with her

Gracie will continue to say things to you even when you haven’t tapped the screen, which adds to the feeling you’re having a real conversation.

Video Games

Video games are another format in which VUIs can enhance the user’s immersion in the virtual experience; however, it must be done well.

Some video games use voice to allow the player to issue commands. For example, in Yakuza Studio’s third-person shooter game Binary Domain you can issue commands to your team such as “Cover me!”, “Regroup!”, and “Fire!”—and they might or might not do it, depending on whether they like you (Figure 3-8). Although you’re allowed to speak and issue these commands at any time during game play, at certain points characters ask you questions, and show options on the screen.

Figure 3-8. Voice options in Binary Domain

If the system did not understand you, you’ll hear, “Don’t get ya. Tell me later.”

Another game, In Verbis Virtus by Indomitus Games, uses speech recognition to empower the user to cast spells. For example, the user can say, “Let there be light!” and the screen will light up (Figure 3-9).

Figure 3-9. Casting spells via voice in the game In Verbis Virtus

An interesting twist is that the spells are in a made-up language called Maha’ki, making it even more immersive, since it feels like you’re speaking a magical language. Many users report really enjoying this feature of the game.

There Came an Echo, by Iridium Studios, is a strategy game that allows you to control your team of four via voice commands (Figure 3-10). One nice feature is that you can set up your own aliases; for example, you could substitute “burn it down” for “fire when ready.” You can even call characters by other names of your choice. It allows you to choose your mode: push to talk, or “always listening”. The commands you issued show up on the screen, as well, so you can keep track of how you’ve been directing your team. Characters respond to your commands with “Yes, sir” and “You got it,” adding an element of realism (as well as letting you know it worked).

Figure 3-10. Voice commands in There Came an Echo are listed in the upper-right corner

There Came an Echo has mixed reviews, but users seem to feel the voice command is what makes it a better game. Here’s a YouTube review by a user named TotalBiscuit (emphasis added by this book’s author):

The voice command really is the key factor for this particular title...It’s the voice command that gives it the pacing, and it’s the voice command that makes it challenging, because everything’s being done in real time and it requires you to think on your feet quite rapidly. Not to mention there is a serious amount of cool factor involved in the voice command...This one does it really well because I think it sticks within its means. It doesn’t go overboard with the voice commands. It doesn’t use too many of them, and it limits what you can do to the point where the voice commands are accurate and they’re going to be quick & responsive...The game does not play anywhere near as well without it...This game is more than the sum of its parts thanks to the voice command system.

Most of these games have a calibration setting to help improve accuracy (Figure 3-11). Binary Domain, for example, asks the user to run through various phrases and gives it a score to let the user know how well they can expect performance of speech recognition during game play.

Figure 3-11. Voice calibration setting in There Came an Echo

Another unique example of adding VUI to a video game was 2008’s Eagle Eye Freefall, a short interactive experience that went along with the release of the movie Eagle Eye. The game was developed by Telefon Projekt, and the experience began when the user entered their phone number on the game’s website. Shortly thereafter, the user would receive a phone call on their actual phone, and a mysterious voice would direct the user to follow instructions on their computer and speak to different characters over the phone.

There are very few examples of VUI apps out there right now that use real human actors for having interactive conversations. Using real faces can be an extremely engaging way to involve users, but it is also a much more expensive endeavor.

At Volio, we created interactive conversations in which the majority of the screen on a mobile device was filled by the actor’s face, along with a small picture-in-picture element that displayed the user (Figure 3-12).

Figure 3-12. Volio app, “Talk to Ron”

Having the screen mostly taken up by the actor’s face made the experience very personal and one-on-one for users. Users naturally responded via voice to the actor, with no instructions. Despite knowing the content was not live, many users reported feeling as though they were engaged in a real conversation with the person on the screen.

Technically, however, creating these types of interactions takes a lot of planning. It requires a studio with professional lighting, and for each take, the actor must look the same and have the same head position. This makes it very difficult to add or fix content later; even if you can rebook the actor, replicating the exact lighting, appearance, and head position is challenging.

Adding VUI to prerecorded video only makes sense if the VUI part is also a rich experience. Another example of a video app had a sports celebrity who answered questions. After he spoke, he would end with a question such as “What do you want me to tell you more about?” A screen would then pop up to the let user know what they could say; for example, “Say SPORTS or OFF THE FIELD.” At this point, the user would press the microphone button and say one of those items.

Allowing the user to speak one of these choices did not add to the intimacy of the experience; it was clunky, not conversational. The same could have easily been accomplished by pressing buttons.

Now that we’ve run through some examples of when it’s a good idea—or not a good idea—to use an avatar/character/actor in your VUI, let’s get into some of the best practices for when you do use one.

Should My Users See Themselves?

In some cases, allowing the user to see herself talking back can provide more engagement. You are probably familiar with the FaceTime model, in which the person you’re speaking with takes up the majority of the screen, and there is a small rectangle showing your face, as well.

A clever example of this in a storytelling context is in the ToyTalk app The Winston Show. In one scene, Winston appears on the bridge of a spaceship and announces he’s just discovered an alien lifeform. He asks his shipmate to put the alien “on screen”—and there is the face of the child using the app (Figure 3-13).

We used the picture-in-picture model in our Volio apps, as well, which was generally a familiar-feeling element because of user’s experience with FaceTime, Skype, and so on. Occasionally users said they preferred not to see themselves on the screen, however, so it is worth considering making this a user-controlled feature.

Figure 3-13. Example of an alien lifeform, aka the author’s son

What About the GUI?

Another key question is whether your avatar or recorded video will have any graphical controls. With mobile devices, allowing the choice of speaking or using a graphical user interface (GUI) lets users decide how they want to respond. (Note: this is dependent on the type of experience you’re designing. In the Volio case, UI elements were not shown unless the user was struggling, because we wanted to remain as much as possible in the speaking world, to keep the user engaged.)

Figure 3-14 shows an example of an avatar interaction in which the user can speak or press a button to respond.

When designing a multimodal app (one that has VUI and GUI), normal VUI timeouts are not applicable: if your user decides to press a button and not speak, it doesn’t make sense for the avatar to suddenly say, “I’m sorry, I didn’t get that,” while the user is mulling over their choice of which button to push. In this case, treat it the same way you would if it were GUI-only: let the user have as much time as they want to either speak or press a button.

Figure 3-14. In the case of GUI responses, the Sensely app allows the user to talk or tap

That being said, take note of places in your app where the user does not answer for a long time; there could be an underlying issue that’s causing confusion, or perhaps reluctance for the user to respond.

Additionally, GUI controls do not make sense for all interactions. For example, if you have an open-ended question (such as “How are you feeling?”) and you want to encourage your users to speak naturally, don’t put in a list of possible responses. An additional case is when there is a very long list, such as “Which song would you like to hear?” Let the user first attempt to respond via voice/natural language; after a couple of failures, provide them with a way to select an answer or type one in.

Maintaining Engagement and the Illusion of Awareness

Regardless of whether your VUI uses an avatar/video, keeping your users engaged is paramount.

What are the best ways to create and maintain the illusion of awareness? Some great tips come from the creators of the trivia game You Don’t Know Jack (Figure 3-15). Originally released in the late 1990s as a PC game, You Don’t Know Jack later won both the Most Innovative Interface and Best Overall Interface at Apple Computer’s distinguished Human Interface Design Excellence Awards.

Figure 3-15. You Don’t Know Jack trivia game

It does not use speech recognition, but the game does an amazing job of sucking users in as if they were playing a live game show with a live host. It does this with all prerecorded content.

Harry Gottlieb, the game’s creator, wrote “The Jack Principles of the Interactive Conversation Interface” in 2002. In his paper, Gottlieb outlines tips for creating the illusion of awareness in a conversational system; specifically, he suggests responding with human intelligence and emotion to the following:

• The user’s actions

• The user’s inactions

• The user’s past actions

• A series of the user’s actions

• The actual time and space that the user is in

• The comparison of different users’ situations and actions

How can you accomplish this? As mentioned in Chapter 2, keeping track of the past is very important. If the user has just requested information about primatologist Jane Goodall, and then she follows-up with the question, “Where did she go to college?” don’t act like you don’t know who the user is talking about.

Avatars can even react to a user’s inactions or to errors. If a user hasn’t spoken at a point in the game when they’re expected to, the avatar could cross its arms, tap its foot, or say, “I’m waiiiting!”

By using an avatar or actor, you can help maintain the illusion of awareness even more. An example of this was used in a Volio app that allowed the user to have a conversation with a stand-up comedian. If it was morning, the actor would say, “What are you doing up so early?” to start the conversation. If the time of day is relevant, use that information. Don’t offer to transfer the user to an agent if it’s outside of business hours. Use information about the time to make it more real.

In “Jack Principles” Gottlieb also outlined tips for maintaining the illusion of awareness:

• Use dialog that conveys a sense of intimacy

• Ensure that characters act appropriately while the user is interacting

• Ensure that dialog never seems to repeat

• Be aware of the number of simultaneous users

• Be aware of the gender of the users

• Ensure that the performance of the dialog is seamless

• Avoid the presence of characters when user input cannot be evaluated

Another way to build trust with an avatar is to follow human conversational conventions when appropriate. For example, when users call to speak with a call center agent, they don’t usually launch into a full description of the problem. Here’s an example of a nonstandard interaction:

• AGENT

  Hi, thanks for calling Acme Cable Company.

• CALLER

  It says on my bill I owe $5.99 for something called federal tax, and I’ve never seen that before and I want you to take it off.

Instead, the agent tries to establish some initial rapport and ease the caller into the conversation, building up trust:

• AGENT

  Hi, thanks for calling Acme Cable Company. How are you doing today?

• CALLER

  I’m good, how are you?

• AGENT

  I’m fine, thanks. What can I help you with today?

• CALLER

  I have a question about my bill.

• AGENT

  OK, I can help you with that. What seems to be the issue?

• CALLER

  Yeah, well, there is a charge for $5.99 I don’t understand...

Notice how the agent begins with a question unrelated to the cable company: the “How are you doing?” question. Then, when the agent asks how she can help, the caller breaks it into pieces—first, the general topic (“my bill”) and then the specifics. You can do the same thing with your VUI system, if you plan to have a conversation.

People often like to tell their stories via a narrative. When people go to the doctor, they don’t usually give all the details right up front about what’s wrong; instead, they give one piece at a time. I’ve witnessed this at nurse advisor call centers. I have seen good results with this approach using an avatar, as well. Expecting the user to always throw out all the details of their question or issue right off the bat will often result in not understanding; instead, break it into normal conversational snippets.

Think carefully about the beginning of your conversations. As an example, at Sensely, we sometimes begin with a general “How are you doing?” This gets the conversational ball rolling—people often respond with a simple, “I’m fine, thanks,” but sometimes go into more detail. If you choose to utilize this technique, you’ll need to be able to handle both types of responses.

Avatars and actors are not the only kind of visual feedback your VUI can have. You can use visual feedback to let the user know when your VUI is listening, when it’s thinking, or when it didn’t understand. Some virtual assistants, like Cortana, go further (Figure 3-16).

Figure 3-16. The many moods of Cortana (http://genieblog.ch/)

Here’s how one designer at Microsoft describes designing Cortana’s visual “moods”:

Do the variants on the blue circles work? I doubt users, if shown each circle out of context, could name what emotion or task Cortana was engaged in at that time. However, over time, it’s quite likely users would come to associate them subconsciously. People are very good at patterns and will be quick to pick up on common ones such as the “listening” icon.

As shown in Figure 3-17, the Amazon Echo—a device rather than a mobile app—uses blue lights on the top of its cylinder to let you know when it’s listening (after you say the wake word, “Alexa”). This is a subtle but useful way to let users know that they can speak. After about five seconds, if Alexa does not hear anything, the light disappears.

Figure 3-17. Amazon Echo has a blue ring that lights up to indicate when the user should speak

On the other hand, if you say something that Alexa was completely unable to process, the light pattern changes, and an “end” sound plays, letting you know Alexa did hear you, but was not able to understand. This is particularly useful if you’re talking to it from another room or not facing it. There is one more response for not understanding, which is when Alexa is pretty sure you made a request or asked a question but could not understand it or fulfill it, in which case another light pattern is shown, and she says something along the lines of, “Hmm, I can’t find the answer to the question I heard.”

Jibo, billed as a “family robot,” has a lot of personality without appearing human (Figure 3-18). It’s a small cylinder with a rounded top on a swivel base. Its “face” is a screen that can have animations such as a heart.

Figure 3-18. The Jibo robot

Jibo can respond to voice commands and uses text-to-speech (TTS) to generate spoken responses. Jibo is designed to evoke an emotional response from the user. A review in online magazine Mashable states, “Jibo isn’t an appliance, it’s a companion, one that can interact and react with its human owners in ways that delight.”^[21]

But a word of caution also applies: Jibo definitely evokes emotion, but is it appropriate in all situations? For example, can it be serious enough to serve as a health advisor?

Choosing a voice for your VUI is another important consideration. Will you use TTS or recorded prompts? The voice is a crucial aspect of your design:

Chapter 5 goes more into detail about this topic.

To summarize, let’s look at the pros and cons of using an avatar. This should help guide your decision about whether to use one for your VUI.

Let’s begin with the advantages of having an avatar as part of your VUI. First, avatars can offer more engagement with the user.

In a study conducted at the University of Southern California’s Institute for Creative Technologies, subjects were asked to respond to a series of questions such as “How old are you?” and “What makes you happy?” There were three groups: an animated virtual human, a still picture of a virtual human, and an audio-only version.

There were 24 questions in all, and the subjects were told they had to answer at least 12 questions and had the option of answering all 24. The number of subjects who answered all 24 questions was much higher in the case for which the questions were asked by the animated virtual human. In addition, on average, people’s responses were longer when answering the animated virtual human.

Another USC ICT study had subjects speak with a virtual assistant who helped people suffering from post-traumatic stress disorder (PTSD) after military service. Subjects reported feeling more comfortable disclosing information to the virtual assistant than to a human, possibly because the virtual assistant was less critical.

I have seen a similar effect at Sensely, which uses a virtual nurse to help people with chronic health conditions. Patients often open up to the avatar and tell her when they’re feeling stressed or about other health conditions.

Even simple faces can evoke emotional responses; “a few lines depicting a ‘smiley face’ can lead people to feel sadness, surprise, or anger with a change in one or two strokes of a pen.^[23] In addition, “although people can certainly listen without seeing a speaker’s face, they have a clear and strong bias toward the integration of faces and voices.”^[24] Everyone reads lips to a degree; it’s more difficult to understand someone when you can’t see his or her mouth.

Mark Stephen Meadows, president of avatar company Botanic, has this to say about when we should use an avatar:

He calls avatars the GUI of the VUI. Meadows reminds us of another thing avatars provide—additional clues to personality that will tell us how to interface with them:

Users anthropomorphize computers, even without an avatar face. We take what we know about our interactions with humans and apply them. As Clifford Nass says:

People often extend politeness to computers, as well, as evidenced by an experiment Nass ran at Stanford:

If you are designing a system in which you need a high level of engagement, such as for healthcare or entertainment, consider an avatar, or at least a face.

For tasks with less of an emotional component, such as scheduling meetings, choosing a movie, making a shopping list, or doing a search query, an avatar might not be necessary and could even be a distraction.

Now that we’ve seen some of the advantages of using an avatar, let’s look at the other side.

Avatars do take a lot of extra work, and if they’re not done well, they can annoy or put off your users. Avatars that merely blink but show no emotional reaction or awareness can be more of a hindrance than an asset.

As of this writing, the majority of avatars in the virtual assistant space are female, usually young and sexy. Unless you’re writing a virtual assistant for an adult entertainment app, there’s no reason for this. I recently downloaded a virtual assistant that had a young attractive cartoon woman as the default avatar. She doesn’t do anything besides occasionally blink (or wink). The app allows you to choose different avatars, and the list that came up was a series of scantily-clad women. Plus, a dog.

If you have an avatar, spend time deciding on who it will be. Don’t just let your user pick one at random. Begin with the avatar’s top personality traits. Should the avatar be more authoritative or more caring? Professional? Knowledgeable? Create the persona first, and design the avatar that speaks to those qualities. Persona is not just about looks; it influences how your VUI speaks and responds. If the image of the VUI changes, the persona and prompts should change, as well. If your VUI has an avatar, another set of questions will need to be answered: how much of the avatar will be shown: just the face? The face and upper body? The entire body? Will it be two-dimensional or three-dimensional? How many emotional expressions can be shown? How will the lip synching be done? Will your avatar occupy the entire screen or just a part of it? All of these things influence how your users will respond and interact with your VUI.

One thing to keep in mind when designing your avatar is that a very strong personality can have a negative effect. As Thyme-Gobbel says, “the stronger its persona, the more polarized the user reaction.”

You might have people who love it and some who hate it. You might have users who love having an avatar, and others who do not. Depending on your app, that might be OK—sometimes it’s best to not design for everyone; instead, you might find it advantageous to design for a particular user group that might appreciate that “strong” personality. For example, fans of a particular video game might love an avatar that is “in character” and demonstrates that character’s personality traits. On the other hand, an avatar that’s supposed to be a personal assistant for a whole host of user groups should tone down some of the more extreme traits.

In addition, don’t make assumptions about what your user will want. For example, don’t assume that everyone wants a female avatar over a male avatar. As always, do as much user testing as possible with your avatar choice and prompts. If your avatar is too gimmicky—for example, using a lot of slang—it might make it more difficult to predict what the user will say back.

The Uncanny Valley

Another thing to be aware of when designing an avatar is not to fall into the uncanny valley. The uncanny valley refers to the shudder of horror you feel when you see something that is very close to being human—but not quite. Figure 3-19 shows the dip into the uncanny valley—for example, a zombie is at the bottom of the valley and very disturbing to us.

Figure 3-19. The uncanny valley^[26]

One way to avoid the uncanny valley is to make the avatar not photorealistic, or make it nonhuman (such as an animal).

As Meadows points out, having a recorded human voice with a cartoon doesn’t trip our “uncanny valley” sensors: just think about Pixar movies.

It’s important to match the avatar’s expressions with the words and emotions it is expressing, or it can be very jarring and unpleasant. An example of this can be seen in a conversation with a robot named Sophia. She has just said, “I feel like I could be a good partner to humans. An ambassador,” but at the end of that sentence, her lip curls into a sneer (Figure 3-20). It makes one doubt the sincerity of her words—a worse outcome than having no expression at all. As Nass says, “Facial expression needs to match what is said, or people perceive an attempt at deception.”^[27]

Figure 3-20. Sophia by Hanson Robotics—why the sneer?

Spend time deciding whether your VUI will have a visual component. Don’t just add an avatar because it’s cool. Is your system conversational? Entertaining? Empathetic? These are better candidates for an avatar or actor.

Investing in an avatar is not a small task. To build your own avatar, you’ll need a design team just for that piece, or you can use avatars made by third parties. Having your avatar show emotion, without looking creepy or repeating the same gestures over and over, is challenging. Think carefully about why you want to use an avatar, and do user testing on prototypes to make sure you’re on the right path.

Using a real actor can be an incredibly engaging experience, but this comes with a cost, as well: because everything must be filmed in a studio, you really need to get it right the first time.

A virtual assistant can still be very successful, even without a face. Do take advantage of other types of visual feedback, however, to let the user know when the system is listening or doesn’t understand.