Negative emotions are an everyday phenomenon that are both pervasive and difficult to avoid. Everyone will experience feelings of anger, sadness, or anxiety at some time. However, what differentiates between individuals is the frequency of those emotional experiences and the severity of each episode. Most of us have an intuitive sense that negative emotion is detrimental for health and well-being in the long term, particularly the health of the cardiovascular system, but the significance of this connection may be underestimated by the general population. For example, one large-scale, international study concluded that psychosocial factors (including depression and stress) were stronger predictors of myocardial infarction than diabetes, obesity, smoking, and hypertension [21]. Similarly, the tendency to direct anger and aggressive feeling toward another person has a linear relationship with hypertension. Everson and colleagues [22] reported that an increase of 1 point on the Anger Out scale resulted in a 12% increased likelihood of hypertension over a 4-year period. An increased incidence of coronary heart disease has also been associated with higher levels of anxiety [23].

The experience of negative emotion appears to have a detrimental effect on cardiovascular health that is both cumulative and attritional. This pattern of long-term “wear and tear” on human physiology was described by McEwen [24] in connection with the experience of stress. He argued that major physiological systems in the human body, such as the autonomic nervous system, the endocrine system, the immune system, etc., could be locked into a pattern of sustained activity due to chronic exposure to stressful stimuli. This type of chronic stress has significant consequences for biological ageing, as well as the long-term health of the individual and mortality [25].

The pattern of sustained change in major physiological systems associated with stress are also found in connection with negative emotional states. People suffering from depression are characterized by increased heart rate and reduced heart rate variability due to alterations in the way in which the autonomic nervous system is controlled [26]. The sinoatrial node, which regulates the beating action of the heart, is controlled jointly by inputs from the sympathetic nervous system and the parasympathetic nervous system; the former increases the heart rate, whereas the latter has the opposite effect. Depression tends to impair input from the parasympathetic nervous system, leading to a faster heart rate. This pattern is enhanced in depressed patients by increased circulation of adrenaline and cortisol in the bloodstream, both of which reinforce the influence of the sympathetic nervous system [26]. As well as the autonomic nervous system and the endocrine system, the immune system is also influenced by episodes of depression; levels of pro-inflammatory cytokines are increased, which has consequences for both mood (e.g., a tendency toward negative emotion) and behavior (e.g., reduced appetite and physical activity) [27]. Like the allostatic load model, advanced by McEwen [28], repeated and sustained periods of depression have a corrosive impact on cardiovascular health via a pattern of sustained physiological disruption.

The deleterious impact of anger on cardiovascular health stems from an overstimulation of the sympathetic nervous system. This alteration in autonomic activity leads to increased vasculature constriction, which increases blood pressure and heart rate; there is also evidence for elevated levels of adrenaline and cortisol in the bloodstream well as alterations in the ventricular function of the heart [3]. Although the physiological manifestation of anger within the cardiovascular system is straightforward, estimating its impact on cardiovascular health is complicated by individual differences with respect to the expression and experience of anger. A prospective study of almost 8000 participants, studied over 10–15 years, indicated that high levels of cynicism and low levels of control over the expression of anger were the strongest predictors of subsequent CVD [29].

Anxiety is a negative emotional experience that is commonly linked to situations that provoke fear, apprehension, and uncertainty in the individual. This negative state is associated with over activation of sympathetic nervous system and elevated adrenaline and cortisol. Anxiety has also been linked to abnormal patterns of cardiac control and coronary vasospasm (i.e., rupturing of atherosclerotic plaques on the arterial wall) [26]. The experience of anxiety has been linked to increased incidence of coronary heart disease [23] and elevated incident hypertension [30]; however, the link between anxiety and hypertension has been disputed [31]. It has been argued that the psychological mechanism of rumination (i.e., uncontrollable, repetitive, and intrusive thoughts about a distressing topic) is fundamental to the experience of anxiety and the influence of this negative emotion on cardiovascular health [32].

It should be noted that all three negative emotional states contain significant overlap both with respect to their conceptual definition and associated measures [33]. For example, anxiety and depression are often positively correlated, and one wonders whether a general factor of negative affectivity (i.e., the propensity to experience negative emotion) would be the most appropriate psychological concept on which to base the study of cardiovascular health. Camacho [34] have explicitly described a continuum from anxiety to depression as an inflammatory emotional state; in other words, an experience of negative emotion that is strongly associated with inflammation and dysregulation of the major physiological systems. A prospective study reported a strong link between anxiety/depression and CVD, but this association was not replicated for anger [35]. It is also important to consider the time course of the relationship between negative emotion and cardiovascular health. Episodes of negative emotion that are short-term and acute may cause specific cardiovascular events, such as an arrhythmia, whereas the process of atherosclerosis is cumulative and gradual and takes place over a long period of time [36].

This short review has emphasized the convergence of three negative emotional states on the physiological systems of the human body. Depression, anger, and anxiety all create a shift toward sympathetic dominance in the autonomic nervous system. This increase of catabolic activity creates a series of coordinated changes across the heart, lungs, vasculature, viscera etc. The same pattern is also associated with elevated adrenaline and cortisol in the bloodstream and a rise in levels of pro-inflammatory cytokines. Hence, negative emotions promote a sustained increase of metabolic activity and an inflammatory process, which can be highly damaging for the long-term health of the individual. It should be noted that many of the physiological changes described above occur without any conscious awareness on the part of the individual. It has also been argued that negative emotions may exert significant effects on the cardiovascular system even when the person is sleeping [37]. Objective measurement in combination with pervasive monitoring technology may be the only way to unveil the influence of negative emotional states on physiological systems and could act as an important data source for interventions that are designed to improve health in the long-term.

The effective use of advanced technological systems represents one approach to reducing the profound economic and societal costs of disease on the global population. Lifelogging is a technology that supports the necessary self-reflection required to develop an effective coping strategy via the continuous recording of behavior from data from the body. Data are continuously gathered from multiple sources (e.g., from wearable cameras and body sensors) to capture patterns of daily activity in a permanent and comprehensive format. Although this practice of recording objective data on daily patterns of behavior is known as lifelogging, the outcome of this process is often referred to as a lifelog or human digital memory (HDM). As technology advances, and sensors become more prevalent within our environment, we have access to an increasing range of data. Through unique addressing schemes, the Internet of Things (IoT) has enabled a range of mobile devices to interact with each other and cooperate with their neighbors in order to provide a comprehensive source of information [41]. Currently, there are more connected devices than people on the planet: approximately 25 billion connected devices compared to the world’s population of 7.2 billion, with this number set to increase twofold to 50 billion by 2020 [42] as shown in Figure 2.1.

Furthermore, over the next decade, the IoT is projected to generate $14.4 trillion of value from factors including reduced costs, employee productivity, waste elimination, customer experience, and innovation (reducing time to market) [43]. This availability of a myriad of connected sensing devices has enabled lifelogs to become richer with information, and their use in various application domains is also increasing.

The data arising from a lifelog can be consulted and used as the basis for self-reflection and insight. For instance, cross-referencing the daily occurrence of negative emotional episodes with physiological markers (e.g., electrocardiogram [ECG] and pulse velocity) and lifelogging data, (e.g., time of day, location, and photographs) enables great insight into behavior. For example, during the context of driving, the occurrence of traffic congestion at a particular junction, raises blood pressure without any awareness on the part of the driver [44]. The repetition of this daily behavior over a sustained period would contribute to the development of CVD. However, if the individual could review those moments when anger increased and view the magnitude of change in the cardiovascular system, this would lead to a profound insight into the context of this behavior and the significance of psychophysiological pathways. Armed with this insight, the driver may adapt his behavior via a change in behavior, such as avoiding a particular location at a particular time of day in the future. The availability and range of measures that are available greatly increase the sophistication of self-monitoring, and the availability of such data can be used to increase self-awareness and self-knowledge. However, this benign influence of lifelogging technology depends on a number of factors, including (1) the confidence users have in the sensitivity and validity of the resulting data, (2) the insight provided by these systems and the additional benefit or utility that is provided by the availability of these data, and (3) the interface/visualization of data, which determines the ease with which insight can be obtained.

This chapter will explore lifelogging technologies, the effectiveness of technological coping strategies to alleviate anger and anxiety and the human–computer interaction (HCI) issues that the area faces. The chapter will conclude with a summary of the key challenges facing this category of technology.

The use of mobile phones and cameras for the collection of data related to the surroundings of an individual have been explored in previous research [49,50]. This work indicated that the use of such visual imagery is the preferred representation for autobiographical memories because it is an efficient form of data representation [51]. One central argument of the current chapter is that lifelogging can support positive self-regulation of health via representation and feedback of implicit behavior. This position is supported by Doherty et al. [52], who found that the act of reviewing lifelogging data prompted their participants to make changes to their lifestyle, for example, increased level of exercise, spending more time with their children. However, these hardware platforms are limited in the sense that the physiological changes underpinning the experiences were not included. These data are important because they provide insight into those unconscious physiological processes that support everyday behavior and experience. Furthermore, these data can yield an objective index of emotional experience and promote positive strategies for coping by bridging the gap between conscious experience and the unconscious physiological activity.

Recent research into the area of affective computing has demonstrated that emotional states can be recognized from physiological signals [53-55]. These data are highly personalized and unique to every individual. The incorporation of physiological data into a lifelog would allow users to access an objective representation of how an emotional episode, such as an outburst of anger, is manifested in the body. The development of smaller sensors and wireless communications is revolutionizing the ubiquity of monitoring and the availability of physiological data for personal use [56]. One such approach has been the Affective Diary [57,58], which is a digital diary where users can write notes while the application allows “body memorabilia” to be recorded from sensors [58]. This system has been used to explore the emotional aspect to creating diaries and is designed to support self-reflection [59]. However, manually uploading logged data into the application, and visualizing data in the form of “somewhat ambiguously shaped and coloured figures” both represented disadvantages for uses of the Affective Diary [58]. The AffectAura [55] takes a different approach and represents itself as an “emotional prosthetic” that allows users to reflect on their emotional states over long periods of time. The AffectAura has been designed as a technology probe to explore the potential power of a reflective process based upon a pairing of affective data with knowledge of workers’ information and data interaction artifacts. Data are collected from a variety of devices, including a webcam, Kinect, and microphone. Supervised machine learning was used to develop an affect recognition engine based on the resulting database. These categorized data are displayed in a timeline format, which captured the ebb and flow of affects, represented by a series of bubbles. The results of an evaluation indicated that users were able to leverage cues from AffectAura to construct stories about their days, even after they had forgotten these particular incidents or their related emotional tones [55]. Using the k-nearest neighbor (kNN) classifier, AffectAura predicted emotional status across the emotional states of valence, arousal, and engagement and achieved a modest prediction accuracy of 68% across all three states [55]. Other related research includes the lifelogging system developed by the Hernandez et al. [51] lifelogging mirror system that enabled an individual to capture and reflect on their daily activities. This system was composed of a number of wearable devices that capture physiological signals, acceleration, and photographs. These data are subsequently displayed in an interactive digital mirror interface to aid reflection. This interface consisted of two components: a digital mirror, which was used to display the information and a “gesture recognition system” to browse the data. In order to interact with the system, via the gesture system, a Kinect camera has been attached below the mirror, which allowed the user to navigate through their data by waving their arms (up, down, left and right) in front of the camera [51]. As such, photos are linked to the physiological data and the user may search the resulting data on a daily timeline.

There are a number of HCI issues pertaining to the design of lifelogging interfaces, and a number of studies have been conducted in this area. For example, Impact used a website to visualize step counts to represent physical activity and self-annotated notes of the context of this behavior to represent the activity level of the user [61]. UbiFit Garden combined a mobile sensing platform (MSP) with a visual metaphor via a smartphone interface that was designed to increase users’ levels of physical activity [61,62]. Feedback was provided by the visualization of a garden, with the flowers in this digital garden growing and blooming as the user became more active. Furthermore, a large yellow butterfly appeared when a weekly goal had been completed and a series of smaller butterflies have been used as indications of recent goal attainments, which acted as a reward and a reminder of past successes [62]. Fish n’ Steps also attempted to increase levels of physical activity by measuring the step count of the user via pedometers and linking these data with an interactive game that involved caring for a virtual pet. The daily step count from the users was used to grow and sustain the emotional well-being of a fish in an aquarium, and this visualization was used to motivate users to become more physically active [63].

Other types of lifelogging interfaces have moved away from physical activity and toward more traditional journal-keeping activities. Footprint Tracker is a lifelogging tool that combines photographs, location, temporal, and social context cues into a two-panel display. A timeline interface was used to highlight the presence of data within a particular period, and this was supplemented with a data panel to display the relevant information [64]. A study of Footprint Tracker concluded that this kind of diary format increased participants’ ability to recall and reflect on their daily activities. The emphasis of Footprint Tracker on realism and the requirement for unobtrusive devices should be noted for diary or journal applications [64].

Other types of lifelogging applications emphasize psychological well-being as well as physical fitness. The DStress system is designed to reduce stress by setting goals and activities for the user to undertake [65]. The goals of the system are achieved by providing feedback to the user with respect to:

The system was used by 77 participants over a period of 1 month to determine the effectiveness of the application with respect to stress reduction. A survey technique was used to collect information before and after the study to evaluate its effectiveness as an intervention. The system included an element of adaptive personalization by modifying the level of difficulty for goals for the user, with respect to their previous performance. The study found that being adaptive was beneficial as it was able to maintain a suitable level of difficulty for users to get the greatest benefit [65]. The EmoTree represents a similar application [66] that incorporated data from an ECG into the data stream. This system was designed to detect emotional triggers for overeating and was based on the rationale that increased stress was associated with undesirable behavior. The application included an intervention that was delivered via the mobile application, which was designed to assist participants in adjusting from a state of stress to a state of relative calm. The interface is designed as a tree that displays the collection of physiological data and the deliverance of emotion to the user. The user’s emotional state is aggregated as the background color to indicate positive or negative valence of affect [66]. Evaluation of this system was undertaken via three studies that (1) gathered emotional eating patterns via participants self-reporting emotions and food intake every hour, (2) investigated intervention techniques by including a deep breathing intervention exercise into the app that is triggered when the user self-reports stress/anxiety emotions, and (3) detected emotional states using wearable ECG, electrodermal activity (EDA), and a three-axis accelerometer and a two-axis gyroscope body sensors [66]. The authors reported that all participants became more aware of their eating habits, and 87.5% became more aware of their emotions via their interaction with the application. However, only 37.5% reported a change in behavior as a result of the first study. This work is interesting because of the low conversion from recording behavior to implementing changes, as “logging made participants more aware, it seemed that most needed something extra to incentivise real change” [66]. It also seems that in the case of the second study, this intervention worked relatively well, but it was reported that personalization would have been more beneficial. The third study aimed to move toward this idea with the introduction of wearable technology. Using a machine learning approach of Gaussian process regression (GPR) produced an accuracy of 75% for classifying arousal and 72.62% for valence [66].

As individuals collect more personal information, the topic of HCI becomes more intrinsic to lifelogging applications. This area offers as a way of visualizing our data to derive knowledge and understanding, as well as promoting positive behavioral changes. However, the creation of such applications and tools poses new challenges in HCI and enables diverse disciplines, such as ubiquitous computing, data visualization, and psychophysiology, to collaborate on the design of these systems [67].

Body sensor networks can be created for the capture of medically relevant data that are composed of multiple wearable sensors [72]. Technological advances in the design of wearable devices have allowed sensor apparatus to become smaller, weigh less, and consume less power; therefore, physiological data can be collected unobtrusively [73]. For instance, pulse and respiratory rates have been successfully collected using different sensors in Google Glass [73]. Such systems consist of three main entities: (1) the hardware that is used to collect such information, (2) the communication protocols that are used to transmit data to a central location, and (3) data analysis techniques to extract clinically relevant information from the raw data [72]. However, each element in this data collection pipeline incorporates a number of challenges. For instance, multiple pieces of hardware are required, such as a number of wearable sensors/devices and cameras, to capture signals from the body and the environment. Furthermore, synchronizing data streams from multiple devices, in real time and in a mobile environment, is challenging. Physiological data must be filtered, and artifacts identified in order to isolate the contribution of such factors on cardiovascular physiology. However, in principle, the inclusion of multiple sensors should make it easier to detect such artifacts. Additionally, transmitting these data securely, efficiently, and reliably over wireless networks presents further issues to ensure that the data packets are not intercepted and modified [74]. The reliable transmission of personal physiological data over public networks is a cause for concern as outsiders can gain access to such networks and potentially comprise the privacy and the reliability of the data [74]. These data need to be protected securely as any system that uses physiological data requires data to be free of errors, which could occur by intentional manipulation, for processing and diagnosis [75].

Smartphone technology has revolutionized the development of lifelogging systems that are used to remotely monitor the user [72]. The global smartphone market has experienced tremendous growth in recent years, with shipments increasing by 40%, to more than 1 billion units, in 2013 [78]. These devices are seen as the preferred platform to capture context-related data as they are seen as a “ready to use” platform that does not require significant configuration but is capable of gathering a wealth of contextual information about the user, including call/message logs, captured images, and music preference [72,79]. Additionally, a number of sensors, including Global Positioning System (GPS), Bluetooth and motion sensors (e.g., accelerometer), permit location, proximity of other devices, and activity to also be recorded [79]. Such sensor-based systems are integral in forming awareness of context because they provide (1) insight into physiological changes over time, (2) the ability to correlate activity with physiological data, and (3) prediction of future trends of the user’s health [75].

One of the fundamental challenges of implementing a context-aware system is the problem of identifying the relevant context for data capture [75]. Supervised machine learning can be used to address this issue by training the system on a specific set of criteria [75]. For instance, once the markers of anger have been identified, the system can be trained to recognize those physiological signs and correlate them with other pieces of data (e.g., location, photos). However, the heterogeneous and distributed nature of these data represents a challenge for the integration of multiple streams into a single timeline. These data must be synchronized to capture specific episodes of emotion through a number of different sensors using a number of frequencies. Further research is required to determine the level of synchronization required to achieve the level of data integration in order to describe an episode of anger as one example. Once anger has been determined, these algorithms require training in order to distinguish between episodes that yield high or low levels of inflammation.

The current chapter is concerned with a lifelogging system whose design goal is to provide a platform that supports self-reflection to reduce the onset of CVD. In order to achieve this goal, data must be processed efficiently and displayed in an informative and clear manner. The system interface is a powerful tool that can generate either a positive (e.g., tolerant/content) or negative (e.g., anger/disgust) effect on the user’s experience of the entire system [1]. For instance, if an interface is overly complicated, fails to perform its task (e.g., crashes regularly) or if the appearance is not adaptive can lead to the generation of negative emotions [1]. These are important points to consider when designing a system that is meant to reduce anger and stress. Data must be presented in a way that intuitively supports self-reflection, without being overwhelming, causing alarm or inciting further negative emotions. As such, the area of big data visualization poses several types of challenges. In order to convey information and derive knowledge from complex and large data sets, it is important to employ techniques that are both aesthetically pleasing and functional [83]. However, trying to achieve this often leads to tools that have poor performance in terms of functionality, scalability, and response time [83]. It is important to recognize that effective visualization techniques may not transfer across different tasks [84]. For instance, Epstein et al. [84] have explored displaying location and activity data that was generated from a smartphone in a number of ways, including tables, graphs, captions, maps, Sankey diagrams, and as a timeline. Their study concluded that participants generally preferred tabular and graphical presentations, which summarized data in an easily consumable format and that these methods were more useful at “discovering factors that influence their behavior (p<0.01) and for finding patterns (p<0.01)” [84]. However, these findings are fairly preliminary. In order to support self-reflection, reducing CVD requires more complex items of data that need to be displayed effectively to encourage this task.

This approach will enable the promotion of positive coping strategies by bridging the gap between conscious experience and data representing unconscious processes. Utilizing such technologies to monitor behavior provides a very powerful graphical illustration of our health and well-being. Bringing together lifelogging and physiological data capture enables the context of situations to be derived and markers for anger to be determined. This technology enables the user to quantify the impact that everyday periods of anger has on the markers of cardiovascular health by reflecting the triggers for associated behavior. The individual is subsequently empowered to reevaluate the situation and change his or her behavior accordingly.