THE BRAVE NEW WORLD OF EMOTIONAL LIFE

Despite the best efforts of amateur and expert alike, emotions remain elusive. Better sensed than described; better depicted through art and film than analyzed as data; better lived than simulated. We don’t understand emotions unless we feel them, and then that experience becomes a blind spot. But if we look closely, we can begin to see that our emotional life is changing in a very real way.

In my own research, through a series of Future Feeling Labs held all over the world, some patterns emerge. I see the rise of six new types of feelings.

Macro—emotional climate

You might have a good sense of the mood in your own home or at your office or even at a concert. You might also have a good sense of the emotional relationships, whether a reaction to a shared experience or an accumulation of everyday interactions. Humans are remarkable emotional climate sensors. But now we don’t need to rely only on our own read of the room.

Sentiment analysis in combination with geo-tagged social media posts, like the University of Vermont’s Hedonometer (Figure 7-3) or Humboldt University’s Hate Map, makes collective mood visible on a larger scale than we can imagine on our own. The latest emotion AI will go further, lending a more nuanced note to detecting and displaying emotion. The XOX wristband, worn by event-goers, registers and reveals crowd emotion at concerts and conferences, to build excitement or reinforce a communal feeling. On the flip side, it’s not difficult to see how it could just as easily divide the crowd.

Figure 7-3. The emotional climate of a nation (source: Hedonometer)

With new exposure to emotion at scale, emotional contagion takes on a new shape. Emotional contagion, in which your emotions trigger another’s emotion, happens on a small scale every day. We synchronize with the expressions and gestures of those around us. Neuroscientists study mirror neurons that embody this resonance. When it’s a positive, it’s usually called attunement. Contagion seems negative, even though that might not always be the case.

The network effect amplifies emotional contagion. Researchers at Facebook demonstrated this by skewing what roughly 700,000 people saw in their feeds.¹ By tinkering with more positive or more negative posts, they were able to show just how easily emotion traveled from person to person. People shown negative status messages made more negative posts; people shown positives posted positive. Emotional contagion isn’t just face to face.

It turns out, certain feelings take precedent in this macro world of emotion. Just a few years ago, researchers thought that positive emotion, like curiosity and amazement, was the fuel behind viral content.² That seems to be shifting toward the extremely negative, mostly outrage. Either way, high-arousal emotions like joy and fear and anger seem to travel with more speed and force—so much so that we are often highly aware of the emotions of others, especially certain emotions, whereas we miss many others that don’t travel quite so far, like sadness or loneliness or relief.

At the same time, we simply don’t register other network effects. In another recent experiment, psychologists showed people sad and happy faces in fast succession. Afterward, people were asked to drink a new type of beverage. People exposed to the happy faces rated the drink higher than those people shown the sad faces. The researchers interpreted this as evidence of unconscious emotion, or feelings we have without really even being aware.³

Macro emotions are never fully our own. Our awareness of the degree and impact of emotional climate is already changing with the aid of technology. Growing awareness of the emotional climate will likely prompt us to reflect on how we feel about feelings, too.

Meta—emotions about emotions

In everything from art appreciation to child rearing to romantic relationships, researchers have studied the feelings we have about other people’s feelings. You might tend toward dismissive, disapproving, or even overly permissive in your assessment of other’s feelings, but right now that judgment is usually reserved for those close to you. What happens when tech becomes involved?

As we become more aware of the tiny moments, the rising tides, and the grand movements in emotion, meta-awareness is likely to be a part of our everyday life. Take a moment to review Lev Manovich’s experiment, Selfie City (Figure 7-4). In this collection of Instagram self-portraits from five cities across the world, you can see the range of expression across cultures. By using a slider, you can gain an understanding of how emotion takes shape, no doubt affecting your own affect.

Figure 7-4. Feelings about feelings that shape more feelings (source: Selfie City)

A big part of our life is a tacit comment on other people’s feelings. We might perform emotional work at work, out of consideration for others’ feeling or to influence them. Online we perform emotion, often in an exaggerated way to make sure that emotion is felt by others. As we become aware of technology detecting our emotion, we likely start performing emotion in a certain way for the machines in our lives, too.

As we are exposed to our own emotions in new ways, whether we see it as aggregate data to detect patterns or receive nudges based on how we are feeling, we’ll start to have feelings about that. Feelings that will, in turn, create new kinds of feelings.

Machine—feelings by, for, and of technology

Researchers have long studied the emotional relationship between humans and machines. As noted in Chapter 4, it doesn’t take much for us to attribute emotion to machines and to attach to them. Companion robots are probably not going to replace human companionship; nevertheless, they will certainly be a part of many people’s lives in the near future (if not already). And they’ll take all kinds of forms: child-like droids to childlike simulations (Figure 7-5), disembodied therapy bots to rubbery sex bots, friendly giants to fluttery fairies. Some will be amazing, some will be awful, many we’ll love anyway.

Figure 7-5. Some of you will feel real love for this unreal baby (source: Soul Machines)

It can be a red herring to debate whether they will “have” emotions, because we don’t really know what that means anyway. David Levy argues in Love and Sex with Robots (HarperCollins, 2007) that if a robot behaves as though it has feelings, it can be difficult to argue that it doesn’t. With devices exposed to human emotion on a scale more extensive than many of us experience in a lifetime, bots will certainly have some kind of emotional intelligence, though. More than that, just as certain breeds of animal have coevolved with humans to understand our emotions and express emotion so that we feel that they have emotions like ours, bots will likely move toward that, too.

Annette Zimmermann, vice president of research at Gartner, said, “By 2022, your personal device will know more about your emotional state than your own family.” Gasp if you must, but that might already be the case. You probably wouldn’t hand over your phone even to someone close to you, much less a mere acquaintance. Whether you believe that our devices will know more about us than our family members, they will certainly begin to know not just what we do but how we feel.

For now, the worry is that technology is taking too much of a toll on emotional intelligence. People are less self-aware as emotions are exaggerated online. Empathy might seem diminished in the wake of more screen time. Impulse control becomes more difficult in an era of attention-grabbing. So far, that’s more correlation than causation. And it’s not the entire story. For each of these negative takes, there are also positives. As our emotions atomize or crescendo or radicalize, we’ll relearn emotional intelligence, too.

Technology will reshape our consciousness in ways that won’t be evident for years to come. The new tech-infused emotional intelligence will reshape not only our private lives but our public lives as well.

EMOTIONAL INTELLIGENCE AND PUBLIC POLICY

In the 21st century, emotional health is a key goal of public policy, championed by psychologists, economists, charities, and governments. Those lucky enough to enjoy emotional well-being are less likely to suffer from a range of mental and physical disorders, such as depression, addiction, anxiety, anorexia, irritable bowel syndrome, or heart disease.

Scientists, doctors, philosophers, and politicians now talk about emotions such as anger, worry, fear, and happiness as causes or symptoms. But what is the perfect recipe for emotional health? Who decides which emotions we should feel, and when?

In the past few decades, the emotion driving policy has been loosely defined as happiness. Smart Dubai, for instance, use sensors and analytics to promote happiness (Figure 7-6). By tracking what people do and how they behave in combination with online sentiment, they aim to cultivate happier citizens. The citywide Happiness Meter makes happiness transparent to policy makers, affecting spending on transportation, health, and education. Smart Dubai is an ambitious initiative, but it’s not only one. From the Bhutan Gross National Happiness Index to Santa Monica Well-Being to the World Economic Foundation sustainable development goals, happiness is an emotion that drives policy big and small.

Figure 7-6. Citizen happiness already shapes public policy (source: Smart Dubai)

Whereas happiness is an explicit emotional goal, there are others that drive public policy without formal adoption. Legal institutions, charged with regulating individual emotion and responding to public sentiment, are shaped by anger and remorse. Outrage, simultaneously tracked and fueled by technology, is driving political action all over the world. Empathy, whether through real-life or VR-based refugee experience simulations, assists policy makers to assess impact as well as influence public support. For example, MIT’s Deep Empathy initiative simulates how cities around the world would be transformed by conflict using deep learning. Emblematic Group walks people through devastated Syrian cities (Figure 7-7).

As cities begin to profile more data through cameras, sensors, and biometrics, the emotional experience of public life will become even more evident. The opportunity to improve the design of cities and to raise the well-being of all citizens is real. In the near future, designers will be tasked directly with using this data to create environments that support emotional well-being.

Figure 7-7. Virtual witnessing potentially influences policy (source: Emblematic Group)

EMOTIONAL INTELLIGENCE IN THE ORGANIZATION

Emotional intelligence, at scale, is not just changing our personal lives but is becoming foundational to organizational culture. I challenge you to go a day without coming across a headline in Harvard Business Review, Forbes, or Inc. about the importance of emotional intelligence in the workplace. The World Economic Forum, in its report “The Future of Jobs,” ranked socio-emotional skills as increasingly critical for future careers. Business schools are adjusting their curricula to include emotional intelligence. Companies like Google offer emotional intelligence courses. Organizations like the School of Life or Brain Pickings are on a mission to bring emotional intelligence to the masses. Tools to measure EQ or track employee happiness (Figure 7-8) are near-ubiquitous.

Figure 7-8. How do you like me now? (Source: TinyPulse)

Although organizations seem to value the emotional intelligence of leaders and the emotional well-being of employees, emotional labor itself is undervalued. Customer service agents, caretakers for the young and the elderly, and social workers are not paid well relative to other professions. The concept of emotional labor isn’t new. Arlie Hochschild first published The Managed Heart: Commercialization of Human Emotion (University of California Press) in 1983. But the mental health issues associated with emotional labor are more relevant than ever.

It looks like this is an area where emotion AI is ready to assist. AI to augment the emotional intelligence of human customer service reps is already in use. Japan is leading the charge to develop emotionally intelligent AI to care for the elderly, from robot therapy pets (Figure 7-9) to home health chatbots. Perhaps the required relentless cheer of Disney employees and Walmart greeters will be next for an emotion AI upgrade.

Figure 7-9. Where humans won’t go, robot seals dare tread (source: PARO)

Whether it’s leadership or team building or individual success, creativity or collaboration, organizations have already embraced emotional intelligence for humans. Emotional labor is beginning to get its due, with emotion AI potentially supplementing roles or even providing relief. What will it take for emotional intelligence to become a part of product design in the organization?

The market for emotion AI is estimated to grow exponentially. It’s easy to see why. Companies will want to get a competitive advantage by learning more about what people feel. For tech companies relying on personal data, it’s a wide-open world of customer insight. And it’s easy to imagine how that could veer toward manipulation, especially given the trend toward optimizing for transactional emotion.

All of the tech giants have tried to crack the psychological code for emotion, however indirectly. So far, the tech industry optimizes around key metrics that favors some aspects of emotion. In-the-moment behaviors like time spent or scroll depth or number of clicks stand in for in-the-moment engagement or even delight. For Amazon, it might be evident in product recommendations. For Facebook, it’s determining which posts show up in the feed. For Candy Crush Saga, it’s knowing what to offer for free or not. Optimizing for short-term emotional gains is still the primary mode of operation for tech companies, and it often comes at the expense of emotional sustainability.

Moving forward, products and services that are more emotion-aware will provide real value. Market positioning is one factor. If given a choice (and in certain sectors that is less and less the case), frictionless can translate to blandly predictable. Think already of the experience of planning travel, where a typical journey consists of 20-odd sites that are quite disposable after booking. Think of the speed at which people try to “get through” certain experiences, often not just out of time-saving or convenience.

Easy and efficient, those pillars of user experience are still relevant, of course. You probably appreciate the ease of tracking travel expenses or making a remote deposit at your bank. But easy and efficient are often synonymous with expendable and interchangeable. An emotionally satisfying experience is one that stays with you.

People are becoming increasingly aware of how some apps make them feel distracted or even creeped out. More and more, people are beginning to wonder about the emotional toll. People are actively looking for products that support emotional well-being in all kinds of ways, from fitness to mental health to mindfulness. Emotionally intelligent products will become a key selling point.

With the advent of emotional AI, expectations will shift to demand more of technology, beyond ease and efficiency. Rana el Kaliouby, CEO of Affectiva, says that “10 years down the line, we won’t remember what it was like when we couldn’t just frown at our device, and our device would say, ‘Oh, you didn’t like that, did you?’”⁴ Products that acknowledge emotion and enhance well-being without breaking trust will stand out.

Attracting talent will drive further organizational change. The talent marketplace has never been so competitive. A company offering an emotionally intelligent product or service will attract and retain talent, particularly among designers and developers who have a strong impulse to change the world for the better. And by showing that the company values its relationship with its customers, by extension, it means that it’s likely to value the relationship with employees.

Whether organizations slowly adopt new measures or radically shift values or try to continue along their current path, change will happen anyway. Metrics like page views and retweets and likes will inevitably fade. Persuasive tactics that work now will be exposed and become less effective. These measures will have to be replaced by something. Future-forward companies will start to think differently about creating a relationship with customers by doing some of the following:

• Consider the emotional layer of experience, not just the functional layer.

• Measure on multiple factors, including emotion and other aspects of subjective well-being.

• Develop for long-term goals that offer emotional support, personal growth, and social connection.

• Frame value to a broader range of stakeholders, not only shareholders and business partners.

• Make emotional factors legible so that the relationship is meaningful and respectful.

The value of a long-term relationship aligns with growth. If we look at what people choose, when given a real choice, it’s often emotional. If we look at what makes people stay, it’s an emotional investment. If we look at what contributes to a good life, it’s technology that supports emotional well-being.

EMOTIONAL INTELLIGENCE AND ETHICS

Designers struggle with the unintended consequences of the products they create, from spreading hate speech to exacerbating mental health problems. Beyond “do no harm” or “don’t be evil,” conversations about ethics in tech are often framed around personal autonomy. But I wonder if this isn’t another area in which we should let emotional intelligence guide us. Where there are strong or conflicted feelings, you are sure to find the most pressing ethical conundrums.

Philosopher Martha Nussbaum, in Upheavals of Thought: The Intelligence of Emotions (Cambridge University Press, 2001), argues that emotions are central to any substantive theory of ethics. The most important truths about humans are not grasped by intellectually activity alone, but by emotion. Emotions expose our values. Grief, for example, presupposes the belief that whatever has been lost has tremendous value. So, if you believe someone is important in your life and that person dies, you will experience grief. This extends to bigger issues of identity, too. If someone says they are a feminist but witnesses an act of abuse without reacting, people might question the sincerity of those convictions, for example.

Anger seems like one possible emotional starting point for tech ethics. If there’s any one emotion that characterizes the internet, it’s anger. We’ve evolved from flame wars to fake news. We inexplicably feel the urge to argue with strangers on the internet. Since we disassociate a bit when we’re online, we might express our rage more virulently. Moral outrage prompts comments and sharing. That rage multiplies. After we log off (or whatever, you know what I mean), we might feel a kind of anger hangover for a while or a longer-lasting rage flu. Anger shows what we value, certainly. But it takes on its own life. And it’s rewarded by attention, human and machine. What if we started with anger and, instead of letting it spread or fester (or profiting from it), we looked at ways to encourage the hard work of self-examination or civic engagement? The more we understand the emotional undercurrent of experience, the more emotions can become an ethical compass, guiding us toward what we value.

Human emotion has been proposed as the antidote for a future where artificial intelligence takes over so many of the things we do now. Quite honestly, we are not all experts at emotional intelligence just because we are human, either. We have trouble recognizing our own emotions, we often don’t respect the basic humanity of other people, we discount emotion or go to great lengths to avoid it. So, while I’d like to remain hopeful that the dawn of AI will lead humans to develop even greater compassion, lead us to value emotional labor highly, and teach emotional intelligence, it’s not quite so simple. The idea of the human as the emotional layer of the operating system has its own limitations to overcome. But one thing is clear—we need emotionally intelligent humans to develop emotionally intelligent technology.