Supporting Privacy in Mobile and Pervasive Computing
How can future mobile and pervasive computing systems properly take personal privacy concerns and needs into account? How can we ensure that a world full of mobile and pervasive computing will not turn into a dystopian future of mass surveillance? These are hard questions with no simple answers. In this chapter, we discuss key approaches and challenges for respecting and supporting privacy in mobile and pervasive computing. However, none of these approaches alone will “fix” all privacy problems. The previous chapters showed that privacy is a complex topic and protecting privacy in socio-technical systems is a complex challenge. There is no simple technical solution that can fully address all privacy concerns. Nor can technology do so by itself: laws and social norms influence what data practices are deemed acceptable and what should be prevented. Yet, laws and social norms in turn need to be supported by technology, so that they can be implemented in practice. Understanding how mobile and pervasive computing works on a technical level is essential for being able to shape the legal and political privacy discourse, just as it is essential to understand, say, behavioral economics in order to understand decision-making practices of individuals.
The previous chapters provided three key insights.
1. Privacy provides both individual and societal benefits. Privacy is not a “nice to have” that we might want to trade in for better shopping experiences or higher efficiency—it is a core requirement of democratic societies that thrive on the individuality of their citizens. Neither is privacy just sought by criminals, delinquents, or deviants, while “honest” citizens have “nothing to hide.” Privacy supports human dignity, empowers individuals, and can provide checks and balances on society’s powers, such as the government and corporate entities. However, supporting and enforcing privacy comes with costs that individuals often do not or cannot bear—Solove calls this the “consent dilemma” of privacy self-management. We need structural support—legal, technical, and social—in order to enable privacy in mobile and pervasive computing.
2. Understanding privacy issues and their implications is challenging. Nissenbaum identifies “contextual integrity violations,” Marx describes “privacy border crossings,” and Altman describes privacy infringements as a mismatch between an individual’s desired and actual “levels of privacy.” The “smart” sensing technology at the core of mobile and pervasive computing is particularly bound to violate those boundaries and levels: it is invisible, large-scale, and inherently data-centric. Mobile and pervasive computing signifies a shift in scale of what information is being collected about individuals, and how long it is being stored. Almost everything is stored permanently by companies and the government. Mobile and pervasive computing often further implies the continuous and implicit collection of information through sensors integrated into personal devices and the environment. Continuous and invisible data collection makes it not only difficult for individuals to know when their activities are being recorded, but also makes it difficult to understand the respective privacy implications, e.g., what higher-level information can be inferred from sensor data. The resulting detailed yet opaque user profiles may lead to discrimination and are prone to manipulation and inaccuracies.
3. Traditional privacy approaches are difficult to implement in mobile and pervasive computing systems. Take for example the concept of “notice and choice:” the scale and implicit nature of mobile and pervasive computing systems would place a huge burden on individuals when asking them to individually take a decision on each and every data exchange. However, while new privacy approaches exist, such as “privacy by design,” it is not yet clear how these can actually be implemented in mobile and pervasive computing systems.
Following-up on the last point above, the rest of this chapter summarizes seven key approaches for supporting privacy in mobile and pervasive computing: privacy-friendly defaults, integrated privacy risk communication, assisting with privacy management, context-adaptive privacy mechanisms, user-centric privacy controls, algorithmic accountability, and privacy engineering. All of them are active areas of research. We discuss some established solutions, promising directions, and open issues in those areas.
5.1 PRIVACY-FRIENDLY BY DEFAULT
A popular approach to privacy is what Solove [2013] calls privacy self-management: through “rights of notice, access, and consent regarding the collection, use, and disclosure of personal data […] people can decide for themselves how to weigh the costs and benefits of the collection, use, or disclosure of their information.” This approach has clearly reached its limits years ago already, even without the additional challenges posed by mobile and pervasive computing. Numerous studies and surveys have shown that there are “severe cognitive problems that undermine privacy self-management” [Solove, 2013]. A study by McDonald and Cranor [2008] estimates that users would need to spend 80–300 hs per year if they were to simply skim each privacy policy they encounter while browsing the Web. Solove [2013] cites work from 2004 that found only 4.5% of users stating they would always read a website’s privacy policy [Milne and Culnan, 2004]. These numbers clearly represent a lower bound on the efforts that would be required today, given the proliferation of online services, smartphones and mobile apps over the last decade [Schaub et al., 2017].
Not surprisingly, default settings have a significant impact on consumers’ privacy choices and their level of protection. According to Acquisti et al. [2017], individuals are subject to a status quo bias—most individuals stick with default settings and very few make changes to their privacy settings. Citing Dhingra et al. [2012], they note that people not only have a “propensity to keep the default,” but that defaults also influence “which alternatives they choose, should they deviate from the default.” Willis [2014] additionally identifies transaction barriers, choice bracketing, endorsement effects, and endowment effects as contributing factors in why people stick to defaults. Here, “transaction barriers” refers to the cost of understanding (and executing) the opt-out procedure, potentially repeatedly on a multitude of devices. “Choice bracketing” [Willis, 2014] refers to the fact that an individual decision to share data would be considered trivial to one’s privacy, yet the cumulative effect of such sharing decisions could substantially expose private details. The “endorsement effect” describes “the interpretation of a default as a form of implicit advice by a more knowledgeable party as to what most people prefer or ought to prefer” [Willis, 2014]. In behavioral economics, the “endowment effect” further describes the phenomenon that people attribute a higher value to things they already own, i.e., they would sell them for a much higher amount than what they are willing to pay for them if they needed to acquire these in the first place. In a privacy setting, this means that people ask for up to five times as much compensation for giving up presumably private data than they would be willing to pay in order to have otherwise shared data become private.
Whether or not a certain data collection setting is “opt-in” or “opt-out” thus has profound implications for a user’s privacy choices. Janger and Schwartz [2002] cite a 2001 survey that found only 0.5% of banking customers had exercised opt-out rights given to them by newly introduced legislation that forced banks to provide such an opt-out choice to their customers.
These findings have two important implications. First, a simple notice-based approach is simply not feasible for users—even more so in a world of mobile and pervasive computing. Second, given the strong bias to keep default settings, in particular for users who are unsure of their own preferences [Acquisti et al., 2015, Willis, 2014], this default choice plays a key role in setting the overall “baseline” of privacy protection.
So what is the correct privacy baseline? As discussed in Section 2.1.2, Europe’s General Data Protection Regulation [European Parliament and Council, 2016] explicitly requires “Data Protection by Design and by Default” (Article 25, GDPR):
The controller shall implement appropriate technical and organizational measures for ensuring that, by default, only personal data which are necessary for each specific purpose of the processing are processed.
Here, “privacy by default” is a specific implementation of the data minimization principle, which requires personal data to be “collected for specified, explicit, and legitimate purposes and not processed in a manner that is incompatible with those purposes” and be “adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed” (cf. GDPR Art. 5.1.(b) and (c)).
While the principle of data minimization has received increased attention since the ratification of the GDPR, data minimization is not a new concept. Data minimization was already defined in almost identical wording in Articles 6.1(b) and (c) of the 1995 European Data Protection Directive [European Parliament and Council, 1995] and can be traced back to the “Collection Limitation” and “Use Limitation” principles described in the 1980 OECD guidelines on privacy protection (updated 2013) [OECD, 2013]. Despite this long history, current privacy and data protection regulation is not prescriptive in specifying what these defaults should be—instead the GDPR requires data controllers to only collect what is “adequate and relevant” for the specific purpose of data processing.
Another inspiration for finding the appropriate defaults may come from analyzing the potential of a particular data collection to violate what Nissenbaum [2004] calls “contextual integrity,” i.e., the norms and expectations of privacy of a particular context. Nissenbaum’s multi-step process, described briefly in Section 2.3.3, focuses on identifying potential violations of an individual’s privacy expectations, and thus can help uncover contextually adequate defaults.
Even if personal information is collected adequately, i.e., within the limits of an individual’s privacy expectations and “adequate and relevant” for the envisioned purpose of data processing, it is also important to ensure that this data is not used for purposes beyond what it was initially collected for. This so-called concept of “purpose binding” is one of the core principles of the Fair Information Principles (see Section 2.1.1), formulated as part of the 1973 United States Department for Health Education and Welfare (HEW) report [HEW Advisory Committee, 1973]: “There must be a way for an individual to prevent information about him that was obtained for one purpose from being used or made available for other purposes without his consent.” Today, the concept of purpose binding is an integral part of current privacy legislation (e.g., Article 5.1(b) of the GDPR [European Parliament and Council, 2016] quoted above). Privacy policy specification languages such as P3P [Wenning et al., 2006], EPAL [Ashley et al., 2003], XACML [Parducci et al., 2010], and others [Backes and Markus, 2008] offer an automated way of enforcing such purpose bindings within an information system. However, Koops [2011] questions whether the rigidity of a rule-based system will ever be able to handle the fluidity, flexibility, and context dependency inherent in our legal system, which may help explain the lack of adoption of machine-readable privacy specifications, such as P3P [Cranor, 2012].
5.2 PRIVACY RISK COMMUNICATION
In current privacy regulation and practice, a large emphasis is placed on transparency of data practices. Privacy notices, privacy policies, and mobile permission dialogs aim to make transparent what information about an individual is being collected under what circumstances; how the collected information is used; when and how collected information is shared with other parties; how long it is stored; and privacy controls available to the individual. This notion of transparency is historically rooted in the OECD privacy guidelines [OECD, 2013], the Fair Information Practice Principles [Federal Trade Commission, 1998, HEW Advisory Committee, 1973], as well as the notion of informational self-determination [Westin, 1967].
Unfortunately, privacy notices often fall short of providing transparency due to long and complex privacy statements [Cate, 2010], which people tend to ignore [McDonald and Cranor, 2008]. Furthermore, the current approach toward privacy transparency often falls short in another crucial aspect: even if individuals are aware of data practices, i.e., what information is collected and why, many struggle to accurately estimate the implications and risks for their personal privacy of those data practices. As we discussed in Chapter 4, seemingly innocuous sensor data, such as heart rate or activity level, can reveal a lot about an individual’s habits, health and personality, especially if collected and analyzed continuously. Uncertainty about consequences, framing of privacy-related information, cognitive biases, and decision heuristics detrimentally affect people’s privacy decision making [Acquisti and Grossklags, 2005, Acquisti et al., 2015, 2017, Smith et al., 2011]. Behavioral economists refer to this phenomenon of individuals making suboptimal decisions based on incomplete or asymmetric information as bounded rationality [Acquisti et al., 2015, Simon, 1982].
Addressing this issue requires us to rethink what constitutes relevant and useful information for people to make effective privacy decisions. Rather than just listing what a system’s data practices are, privacy notices should emphasize how those data practices may affect an individual’s privacy. Making privacy risks and implications more explicit would make it easier for indviduals to consider the privacy cost and impact of agreeing to a certain data practice, or when they might want to opt-out or deny a certain practice.
The communication of privacy risks and implications needs to strike a balance between those risks and potential benefits. The goal should not be privacy fearmongering. Instead, a nuanced and balanced presentation of how a specific privacy decision—such as allowing an app access to your location or using a fitness wearable—might impact you, both in terms of direct or intended effects (e.g., the app can provide location-based recommendations; the wearable can track your steps), as well as indirect, less likely or long-term implications (e.g., the app’s advertising partners may analyze your location to infer where you live, work and whether you should be targeted with high-priced advertisements or fast food discounts; your fitness wearable could estimate your risk for certain medical conditions and share this information with insurance companies or your employer).
On a more basic level, transparency about privacy risks and implications might help some individuals recognize in the first place that a specific information sharing decision has privacy implications (e.g., that an app shares your location with third-party advertisers). Thus, risk-centric privacy communication can help enhance privacy awareness and support both people with low digital literacy and higher tech saviness.
One way to create awareness for privacy risks is to transparently communicate the implications of a privacy decision at the time individuals are asked to make a decision. For example by laying out what happens if one says “yes” to a sharing decision and what happens if one says “no.” An important aspect of this is to also communicate what happens when one denies a certain data practice. If the app is denied access to your location, does that prevent the app from working? Is the utility degraded? If yes, in what way? Is there a manual option to provide information that would otherwise be collected manually (e.g., selecting a location on a map instead of using location information)? Communicating consequences and options in case of deny or opt-out decisions could help mitigate certain default effects, i.e., the system default being interpreted as a recommendation on what action is preferable [Acquisti et al., 2017, McKenzie et al., 2006].
Furthermore, creating awareness of data practices and potentially associated risks does not only benefit individuals but also the company. Being transparent about data practices and risks helps reduce surprise for users and can help shape a user’s understanding and mental model of less expected or seemingly privacy-intrusive practices by explaining the purpose and need for certain data collection [Schaub et al., 2015, 2017]. Thus, effective privacy and risk communication may foster consumer trust in a new mobile and pervasive computing technology.
One area where we are starting to see a privacy communication approach more focused on the consequences of a decision are mobile app permissions. Increasingly, mobile apps sandwich just-in-time permission dialogs (e.g., “This app wants to access your location—Allow / Deny”) with additional privacy communication. Figure 5.1 shows an example from Google’s Material Design Guidelines for Android permissions [Google, 2017]. Before the system’s permission dialog is shown, the app explains why access to the respective permission is needed and sometimes how a “deny” decision might impact app functionality. If the user denies the permission, a subsequent dialog may explain how functionality is now limited, how the user can work around those limitations, or how the user can “allow” access if they change their mind.
Of course, privacy risk communication—if provided by the data processor—might often try to persuade users to choose “allow” and accept a data practice. Yet, the privacy communication around mobile permissions demonstrates that explanations of a privacy decision’s consequences can be easily integrated into an application’s interaction flow and user experience. What is needed is research, guidance and best practices on how to leverage such opportunities to make privacy risks and implications more transparent. At the same time, there’s a need to objectively describe and quantify privacy risks, as well as an opportunity to augment privacy (risk) communication by the data processor with privacy risk communication provided by the system (e.g., the underlying mobile operating system or pervasive computing middleware) or third parties (e.g., researchers, journalists or activists).
5.3 PRIVACY MANAGEMENT ASSISTANCE
The case of mobile app permissions demonstrates another important aspect of privacy: individuals should be given real choices to control their privacy, i.e., they should be able to say “no” to certain data practices, and “yes” to others. For example, a person should be able to agree to getting local weather updates based on their location, while disallowing the detailed analysis of their location data to infer where they live, work, or travel.
Figure 5.1: An example of privacy explanations around a mobile permission dialog from Google’s Material Design Guidelines: an on-screen dialog educates the user why the Notes app wants to record audio (left) before the audio recording permission request is shown (center); if the user has denied the permission, the app provides information on how to enable the feature if it is desired later on (right). Image source: Google [2017].
Many of today’s privacy notices do not provide this opportunity. For instance, privacy policies are lengthy documents written in legal language with very little or no choices regarding specific data practices. Individuals are confronted with a take-it-or-leave-it choice: accept the whole privacy policy or do not use the service. In reality, this does not provide a meaningful choice to users [Cate, 2010]. People are forced to accept the privacy policy or terms of service in order to gain access to the desired service—regardless of their actual privacy preferences. If there are no granular choices associated with a privacy notice, individuals have little incentive to read it [Schaub et al., 2017]: it is not worth investing the time to read and understand a privacy policy if there are no real actions one can take, if the policy can change anytime (most privacy policies include a provision to that extent), and if the policy is intentionally abstract and ambiguous about what data practices a user is actually subject to [Bhatia et al., 2016, Reidenberg et al., 2015, 2016]. Abstract and ambiguous descriptions in privacy policies are a consequence of the trend that many companies try to consolidate all their services under a single privacy policy. While this makes it easier for the company to provide its privacy-related information in one place and apply consistent practices across its services, it also means that the privacy policy has to be necessarily abstract and generic in order to cover all services’ data collection, use, and sharing practices. As a result, it is often not clear how a specific service covered by the privacy policy actually collects, processes or shares personal information.
Vague privacy notices and policies leave individuals helpless and resigned [Madden, 2014, Turow et al., 2015]. While individuals might care about their privacy, the choice architectures they are presented with force them to accept practices they do not necessarily agree with or are even aware of, because the only other option is to completely abstain from the benefits the service or technology might provide to them [Acquisti et al., 2017, Cate, 2010, Schaub et al., 2017].
The importance of providing actionable privacy information in order to obtain informed consent has been recognized by researchers [Cate, 2010, Cranor, 2012, Schaub et al., 2017] and policy makers [Federal Trade Commission, 2012, 2015, President’s Concil of Advisors on Science and Technology, 2014]. Europe’s General Data Protection Regulation therefore places strong requirements on how consent has to be obtained, mandating that consent must be freely given, explicit, and specific [European Parliament and Council, 2016]. However, the challenge is to provide individuals with real and granular privacy choices without overwhelming them with those choices. Just exposing more and more opt-ins, opt-outs, and privacy settings to users will not scale, as it will just overwhelm individuals with choices and place the burden for privacy management solely on them [Solove, 2013]. Obtaining meaningful consent without overwhelming users is particularly challenging in mobile and pervasive computing, given the often implicit and imperceptible nature of data collection via sensors embedded into devices and the environment [Luger and Rodden, 2013b], as well as the fact that privacy is commonly not the user’s primary task or motivation in using a system [Ackermann and Mainwaring, 2005].
For instance, mobile permissions already require individuals to make dozens if not hundreds of privacy decisions for all the resources the apps on their smartphones want to access. Based on a sample of 3,760 Android users, Malmi and Weber [2016] find that Android users in 2015 had used 83 smartphone apps at least once per month, on average. Given that on average each app requests five permissions [Pew Research Center, 2015], this results in over 400 permission decisions a smartphone user would have to make on average. Expanding the smartphone permission approach to online services, wearables, smart home systems and other pervasive computing systems, as well as all data collection, use, sharing, and retention practices would result in a plethora of privacy settings, choices, and permissions. Even the most interested user would not be able to consistently manage their privacy across all those choices and contexts [Liu et al., 2016, Schaub, 2014, Schaub et al., 2015], as well as over time [Luger and Rodden, 2013a].
A potential way forward are privacy management assistance solutions that aim to help users more effectively manage their privacy. For instance, machine learning can be leveraged to create personalized privacy assistants or privacy agents which learn from an individual’s privacy decisions and either provide recommendations for future decisions in the same context or across contexts or even automate privacy decisions for the user [Kelley et al., 2008, Knijnenburg and Kobsa, 2013, Liu et al., 2016, Sadeh et al., 2009, Schaub et al., 2015]. The personalized prediction of privacy settings and preferences has received considerable attention [Cornwell et al., 2007, Cranshaw et al., 2011, Kelley et al., 2008, Lin et al., 2012, 2014, Sadeh et al., 2009]. There are two general approaches: leveraging an individual’s prior privacy decisions to predict preferences in new decision contexts or assigning an individual to a profile, a cluster of people with similar privacy preferences. Such profiles could be based on clustering privacy settings of other users, learning from privacy settings made by the user’s “friends,” or be curated by experts [Agarwal and Hall, 2013, Toch, 2014]. These two approaches can also be combined: an individual can be assigned to a profile initially to bootstrap a privacy assistant and avoid coldstart issues, followed by experiential refinement and privacy preference learning from an individual’s actions over time [Knijnenburg and Kobsa, 2013, Schaub, 2014, Schaub et al., 2015]. For example, through a set of questions, the system might learn that a person is highly concerned about location data. By assigning the person to the “location privacy” cluster, the privacy assistant may determine that for this person location sharing should either be blocked by default or the user should be asked when location is being accessed. Based on the person’s privacy settings adjustments over time, the assistant might learn that while the user prefers not to share location in general, the user regularly grants location access to transportation-related apps (e.g., navigation, ride sharing, bus schedule) and update the user’s privacy preference profile accordingly.
Reasoning results can either be provided as privacy settings recommendations to users or settings can be automatically adjusted for the user. Typically, the level of confidence in the reasoning result determines the level of automation or user involvement in making the privacy decisions [Bilogrevic et al., 2013, Kelley et al., 2008, Knijnenburg and Kobsa, 2013, Schaub, 2014, Schaub et al., 2015, Toch, 2011]. Determining the appropriate level of automation—with multiple stages between fully manual and fully automated configuration [Parasuraman et al., 2000]—for privacy decision making and privacy management assistance is a topic of active research. Enabling auditing of system decisions and giving users controls to adjust and tweak preference models, for instance through overviews of what apps were allowed or denied access to location and other resources [Schaub et al., 2014, Tsai et al., 2017], both improves the privacy assistant and provides the users with agency over their privacy assistant.
While researchers have made progress in accurately predicting individuals’ preferred privacy settings in specific situations, it often remains a challenge to help individuals make those settings or automate settings changes for them. Current privacy assistance solutions are either confined to their application context, in which they learn from users’ behavior within the application and may adjust recommendations and settings within that context [Das et al., 2018, Knijnenburg and Kobsa, 2013, Schaub et al., 2014], or require modifications to the underlying system layer, e.g., research prototypes with superuser rights hooking into Android’s permission framework [Liu et al., 2016, Wijesekera et al., 2018].
There is a call for the need to expose privacy settings APIs which would allow privacy assistants to function across contexts [Liu et al., 2016], positing that otherwise we will end up with siloed and less useful privacy assistants for different systems and services, such as Facebook, your smartphone apps, or (parts of) your smart home. Such disjoint privacy assistants would not be able to leverage and benefit from privacy decisions made in other contexts to improve prediction accuracy, requiring the re-learning of privacy preferences in different application contexts and for different privacy assistants. However, given the context-dependent nature of privacy perceptions and preferences [Acquisti et al., 2015, Nissenbaum, 2009], highly specialized privacy assistants might be able to provide more relevant support in the context they are focusing on (e.g., smartphone apps) than general privacy assistants that aim to comprehensively model an individual’s privacy preferences, which may be fraught with uncertainty and context dependency as behavioral economics research suggests [Acquisti et al., 2015]. Nevertheless, further research on more deeply understanding factors that affect privacy decisions and modeling privacy decision making processes is essential to further improve privacy management support.
5.4 CONTEXT-ADAPTIVE PRIVACY MECHANISMS
A particular challenge in developing “smart” privacy assistants and controls for mobile and pervasive technologies is the dynamic nature of both privacy preferences and privacy implications [Acquisti et al., 2015, Nissenbaum, 2009, 2011]. Mobile and pervasive computing exacerbate those aspects due to the inherent and often continuous collection of sensor and context data enabling further inferences about an individual’s behavior, as we discussed in Chapter 4. Privacy solutions need to account for the context-dependent nature of privacy needs, otherwise they may shine in a constrained context but may falter in another. Adapting privacy assistance solutions to a different context or application domain often requires complete re-training and reconsideration to fit the approach to the new circumstances.
Context can affect and shape privacy expectations and needs on multiple levels [Nissenbaum, 2009]. Privacy expectations, preferences, and needs may differ across application domains. For example, physiological data streams may be acceptable and desired for fitness and health—both for personal use and in collective settings—but may create privacy concerns in employment and health insurance contexts, where activity data could be used to estimate work performance, insurability and risk factors. Expectations and the conceptualization of privacy are also shaped by sociocultural context. What may be considered an invasion of privacy in one culture may be acceptable in another; or what may be considered common privacy-seeking behavior in one cultural context, may be less acceptable in another. For example, in the Netherlands most homes do not have window curtains or blinds and curtains are rarely closed, whereas it is common in other cultures to close curtains to create a more private sphere for domestic life.
For individuals, privacy preferences, expectations, concerns, and requirements may also change over time. Personal experiences, knowledge gained, or changes in circumstances affect the awareness of privacy risks and the needs for privacy. Privacy perceptions and expectations may also change on a societal level over time. For instance, when new privacy-related knowledge pervades the collective consciousness. A recent example are Edward Snowden’s revelations about large-scale and persistent monitoring and analysis of personal internet communication and other digital traces by intelligence agencies from the U.S. and other countries [Landau, 2013, 2014, Madden, 2014]. Whereas comprehensive surveillance of online behavior was considered a possibility by security experts, it is now established fact that intelligence agencies gather and analyze digital traces of large parts of the population. This knowledge can result in chilling effects on the expression of government-critical sentiments or engagement in controversial activities. Indeed, studies have shown chilling effects on online searches after the Snowden revelations: searches for privacy-sensitive terms declined significantly both in Wikipedia [Penney, 2016] and Google Search [Marthews and Tucker, 2017] after the extent of U.S. government surveillance became public. Another example is the collective realization, spurred by the debate about Russian meddling in the 2016 U.S. election, that behavioral information gathered by Facebook and other online services can be used to predict personality traits and can be exploited to influence political sentiments and even elections. Privacy experts had warned about how revealing online behavior can be for many years, but the 2016 U.S. election as well as the news that a third party, Cambridge Analytica, had obtained large amounts of Facebook user data [Valdez, 2018] transferred this knowledge into the public consciousness.
Contextual factors influencing privacy can be manifold. Considering and anticipating privacy-relevant context aspects a priori in the design of privacy mechanisms can be challenging. Rather than requiring manual adjustment and tuning to each new context, privacy mechanisms can be designed to dynamically adapt to changes in context. Privacy mechanisms and privacy assistants should be able to dynamically adapt to certain context changes by accounting for context in how privacy assistants function. Furthermore, privacy assistants need to provide tools for individuals to reflect on their privacy settings and preferences in certain contexts and support them to make adjustments.
While the inherent context awareness of mobile and pervasive computing systems poses many privacy challenges, this context awareness also provides an opportunity for privacy in enabling such context-adaptive privacy mechanisms [Schaub, 2018, Schaub et al., 2015]. Similar to how contextual aspects play into how an individual makes privacy decisions [Altman, 1975, Nissenbaum, 2009], a context-adaptive privacy system can analyze and react to privacy-relevant changes in context in order to support its users in managing their privacy across changing contexts. Schaub et al. [2015] operationalize this approach by structuring how context-adaptive privacy mechanisms function into three consecutive phases: privacy context modeling to provide situational privacy awareness, privacy reasoning and adaption to support privacy decision making, and privacy preference realization to translate a privacy decision into actual privacy configurations in a given context and environment.
Privacy context modeling requires an understanding of how and at what granularity privacy-relevant context factors can and need to be represented in computational context models. For instance, Palen and Dourish [2003], Lehikoinen et al. [2008], and Romero et al. [2013] applied Altman’s boundary regulation theory to pervasive computing in order to identify situations in which information exposure and privacy implications change. Furthermore, privacy-relevant context information needs to be collected either by deriving privacy-relevant context features from sensor information, e.g., by sensing people nearby [Könings et al., 2016, Schaub et al., 2014], or by communicating privacy information in machine-readable formats. For instance, Langheinrich [2002a] early on proposed a privacy awareness system (pawS), in which privacy beacons send out privacy policy information encoded with the P3P standard [Wenning et al., 2006] which is received by the user’s privacy proxy and matches a system’s data practices against the user’s privacy preferences. This approach has been adopted and expanded to facilitate context-based privacy negotiation in pervasive and IoT environments [Das et al., 2018, Kwon, 2010, Yee, 2006], as well as enable users to communicate their preferences to a pervasive environment, e.g., through a mobile app sending out privacy preference beacons [Könings et al., 2014].
When privacy-relevant context changes are detected, context-adaptive privacy mechanisms need to match or predict what the user’s privacy preferences may be for the new situation. Reasoning about the user’s privacy preferences determines whether a user should be warned or informed about the change, whether privacy settings can be adapted to the new situation automatically, or whether a set of recommended actions should be provided to the user. Privacy systems might be rule-based (e.g., “clear this display when a new person is detected”), for instance assigning disclosure rules to physical objects [Roesner et al., 2014] or adjusting displayed content based on context rules [Davies et al., 2014, Könings et al., 2016, Schaub et al., 2014]. While effective for specific contexts, rule-based approaches struggle with the uncertainty and fuzziness in both context detection and user’s privacy preferences in substantial context changes. More advanced approaches rely on case-based reasoning leveraging prior decisions of an individual [Bernsmed et al., 2012, Sadeh et al., 2009, Saleh et al., 2007, Schaub et al., 2014] or extract privacy preference profiles from a large number of individuals’ privacy settings [Knijnenburg and Kobsa, 2013, Liu et al., 2016] or their expressions of privacy preferences and concerns [Lin et al., 2014, Wijesekera et al., 2017].
All these approaches require substantial amounts of privacy preference or behavioral data. In collecting such data, especially when eliciting privacy preferences and concerns through self reports, one has to be cautious of the privacy paradox [Norberg et al., 2007]: people’s stated privacy preferences and concerns are not always reflected in their actual behavior. Stated privacy preferences and concerns are usually aspirational—they describe a desired level of privacy, whereas actual behavior is affected by uncertainty, cognitive biases, decision heuristics, and is malleable by framing and offered choice architectures [Acquisti et al., 2015, 2017]. Context-adaptive privacy reasoning approaches should account for uncertainty about sensed context as well as a user’s privacy preferences. Similar to privacy assistance solutions in general, the level of confidence in the context-specific privacy reasoning outcome should determine the required level of human participation in an adaptation decision: systems might automatically adapt privacy settings in high confidence cases with or without informing the user, whereas the user might be offered a set of recommended privacy actions in lower confidence cases [Schaub et al., 2015]. The ten levels of human interaction with automation by Parasuraman et al. [2000] can help guide the appropriate level of user involvement in the privacy adaption.
A particular challenge is enabling context-adaptive privacy mechanisms to effectively realize adaptation decisions in heterogeneous environments. Different systems, infrastructure, sensors, services, or devices may be controlled by different stakeholders [Ackerman and Darrell, 2001]. How a privacy reasoning result can be realized in a specific environment depends on the privacy control capabilities available, i.e., the level of control and trust concerning other devices, infrastructure, and entities [Könings and Schaub, 2011]. Context-adaptive privacy mechanisms may have to consider different adaptation strategies within the same context as well as across contexts and systems. Furthermore, privacy controls available within a specific environment should be discoverable in order to facilitate privacy management and adaptation in previously unknown environments and contexts [Das et al., 2018, Langheinrich, 2002a].
5.5 USER-CENTRIC PRIVACY CONTROLS
On a more fundamental level, we also need to rethink how privacy settings and controls are structured on a systems level and how they are made available to users. Current privacy controls are rooted in access control. Almost always privacy settings are binary access decisions. Once access to a resource has been granted to an app or system, the resource can be used without further restriction until access is revoked again. For example, once you grant a mobile app access to your location, the app can access location information whenever it wants to and for whatever purpose until the permission is revoked. This is inconsistent with both how people make privacy decisions and certain legal requirements (e.g., GDPR [European Parliament and Council, 2016]) and privacy protection principles (e.g., the OECD’s purpose specification and use limitation principles [OECD, 2013]) prescribing that consent for data processing has to be limited to a specified purpose. Context information can be used to infer and contrain the legitimacy of mobile apps’ data access [Chitkara et al., 2017, Tsai et al., 2017, Wijesekera et al., 2017]. New data and permission management approaches, such as PrivacyStreams [Li et al., 2017], aim to change the underlying permission model to make it easier to constrain and audit data flows.
Furthermore, continuous data collection through sensor streams creates a need for potentially excluding certain moments or situations from collection. For instance, user should be able to proactively pause collection, e.g., for a certain period of time, adjust the granularity of collection, or retroactively remove collected data. Such controls could both be available for manual control by the user or through automated support. For example, as discussed in Section 5.4, systems could detect potentially sensitive contexts and suggest to the user when to “tighten” privacy settings, as well as when to “loosen” them again. There is substantial opportunity for innovation in designing privacy controls that facilitate more nuanced decisions but are also easy to use. Such controls also provide an interesting opportunity for integrating privacy risk communication by estimating privacy implications of data collection in specific contexts and automatically flagging sensitive or interesting situations the user might have different privacy preferences for.
In mobile and pervasive systems, users should also be given opportunities to reflect on and adjust privacy settings over time. Privacy settings and retroactive privacy controls in current systems typically give users the option to revoke access permissions and limit future collection; data access tools enable users to review and possibly remove collected data. What requires more attention is how to design such retroactive privacy controls in a way that supports users in reverting previous privacy decisions, i.e., if a user changes their mind about a permission previously granted, they may be able to relatively easily deny or limit future data processing, but it is often difficult and tedious to revoke permissions or consent for data previously collected. What is missing are usable rollback solutions that support users in retroactively and selectively limiting the use of previously collected data. Such approaches should not just support deletion of directly collected data but extend to any inferences made from that data, as well as facilitate distributed data deletion and revocation of consent for data that has been shared with other entities. Such retroactive privacy controls are gaining in importance due to the GDPR’s right to erasure, also called the “right to be forgotten” (Article 17, GDPR). Considering and investing in the usability of such mechanisms will also benefit system developers because it may allow for more differentiated and selective data deletion by users than a simplistic “delete my account and all data” option.
Another worthwhile consideration is whether users can be not only supported to delete their data but also retroactively share more data. If at some point the user decides to activate or reactivate data collection that was previously denied, can—if so desired—past information be shared? While this may sound counterintuitive—how can you share information that hasn’t been collected?—middleware solutions for context-aware computing increasingly act as gate keepers for context or sensor information. For example, Apple’s iOS devices continuously collect activity data from motion sensors and, through the Apple Health API, apps can request access to both future and past sensor information. However, those solutions are currently still binary—either you grant access to all past and future data or to nothing. Yet, context awareness provides opportunities to group and categorize sensor data into events, time spans and situations which may allow for more specific and more intuitive sharing settings.
Overall, context awareness does not just pose privacy challenges but also provides exciting opportunities to design and develop novel privacy controls that are better aligned with people’s contextualized privacy preferences, while also facilitating better integration of privacy controls into the user experience of a mobile or pervasive system. Current privacy controls are often hidden in privacy settings disconnected from the primary user experience, and are therefore under-utilized. Context-aware privacy mechanisms can be better integrated into the user’s interaction flow with the system without disrupting the user experience by providing information and controls relevant to the user’s current context and activity.
5.6 ALGORITHMIC ACCOUNTABILITY
One of the key issues of building comprehensive data collections on individuals is the risk of profiling [Hildebrandt, 2008, Schermer, 2011], as we discussed in Section 4.3. Profiling can be defined as an act of algorithmic decision making on user profiles. Based on collected personal data, an algorithm is responsible for the prioritization and ultimately filtering of people (e.g., applicants), the association of people into similar groups, and ultimately their classification into categories (e.g., as preferred customer) [Diakopoulos, 2016].
At the outset, an individual profile may be inaccurate and thus prevent its subject (i.e., the person the profile is about) from receiving certain benefits they would otherwise receive. Unfortunately, there are many real-world examples for the dangers of relying on algorithmic decision making. In 2013, the Michigan Unemployment Insurance Agency (UIA) replaced a 30-year old mainframe system for deciding unemployment benefits. The new system, dubbed MiDAS (Michigan Integrated Data Automated System), quickly seemed to pay off: in its first few weeks of operation, MiDAS not only uncovered a fivefold increase in fraudulent unemployment filings, but also triggered fraud proceedings against the offenders without any need for human intervention (eliminating one third of UIA’s staff in the process). Less than two years later, and only after coming under immense public pressure, UIA halted this automated fraud assessment program: it turned out that over 92% of its automated decisions were wrong [Charette, 2018]. Nearly 21,000 individuals not only had significant parts of their income and/or assets wrongfully seized by the state, but were also criminalized by a rogue computer program without any information on what precisely they had done wrong, nor were given the chance to defend themselves. Charette [2018] provides a list of other such cases (e.g., Australia’s Centerlink benefits system).
Even an accurate profile can be problematic, as it may expose its subject to the threat of identity theft due to it being a comprehensive collection of private information about a person, which an attacker could use to impersonate that individual. Profiles that are applied to groups (e.g., to people from a certain geographic region) may equally entail individual disadvantages, as group attributes are uniformly applied to all group members (e.g., expectation of higher crime rate). Thus, profiles ultimately carry also societal risk, as their widespread use makes identity theft more likely and can contribute to discrimination in society.
At the same time, some form of “profiling” and algorithmic decision making are integral for the functionality of many mobile and pervasive computing services, ranging from recommending locations and stores near-by that are relevant to the user to automatically adjusting a home’s temperature to the preferences of those present. Therefore, privacy-aware systems will need to find a way to minimize the negative impacts of profiling and algorithmic decision making. This can be achieved using three main approaches: transparency, redress, and data minimization.
At the outset, individuals subject to an automated decision taken on the basis of an individual profile should be made aware of this process. Without this baseline level of transparency, an individual might not even be aware of any disadvantages their profile might cause them. The dangers of automated profiling already feature prominently in European data protection law. The General Data Privacy Regulation [European Parliament and Council, 2016] states in Article 22 on “automated individual decision making, including profiling” that individuals “have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her” except when necessary for a contract, required by law, or with explicit consent. Further restrictions apply if sensitive data (as defined in Article 9, GDPR) are being processed. Furthermore, the GDPR requires data controllers to “provide the data subject with … information” about “the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject” (GDPR Art. 13.2(f)).
Providing meaningful transparency is not trivial, given the complexity and sophistication of today’s profiling and machine intelligence algorithms [Knight, 2017]. At the outset, this requires informing subjects about the classification results of the system (e.g., “high-income spender,” “credit risk”), though such human-readable descriptions seem increasingly quaint when more and more processing is directly handled by self-learning systems: a classification system may dynamically generate such classes of users and label them in a generic fashion that may be difficult (if not impossible) to map onto meaningful human descriptions. Secondly, the actual reasons for classification should be communicated, yet again the sheer number of data points used for today’s profiling systems makes this difficult to achieve in practice. For example, research has been able to infer sexual orientation from one’s social network [Jernigan and Mistree, 2009], Facebook “Likes” [Chen et al., 2016], or profile pictures [Wang and Kosinski, 2018]—hence communicating to a data subject why, for example, their profile picture causes them to fall in one category or another would at best mean disclosing the trained weights that led the employed neural network to come up with its decision. This would hardly be “transparent” for most people.
The second approach, redress, clearly relies on sufficiently transparent processes in order to offer meaningful ways for data subjects to seek help in case of misclassified information. If one does not even know that a certain outcome (e.g., a denied loan) is the result of an automated assessment based on a (potentially) inaccurate profile, seeking redress is difficult. Crawford and Schultz [2014] propose to provide citizens with a new legal right—procedural data due process. Similar to existing procedural due process rights, which give individuals certain rights to review and contest the evidence against them, the authors propose that companies would need to allow individuals to “review and contest” the profile and data upon which an automated decision was based. McClurg [2003] argues for a new approach to U.S. tort law (corresponding to the civil law legal system in contiental Europe) that would make the unauthorized selling of consumer profiles by data brokers actionable under an appropriation tort.1
Others have proposed technical approaches that would allow data processors engaging in data mining to minimize the risk of their algorithms violating anti-discrimination or privacy laws, e.g., by flagging sensitive attributes [Fule and Roddick, 2004], replacing discriminative factors (e.g., ethnicity) with implicitly present factors (e.g., education) [Schermer, 2011], or by detecting rules ex post that produce significantly different outcomes for particular subgroups of a population [Ruggieri et al., 2010]. Note that simply adding human oversight to intransparent systems alone is not a sufficient solution. As Cuéllar [2016] points out, even systems that incorporate human review are seldom challenged in their automatically generated decisions, given the often-assumed “objectivity” of computer algorithms [Marx, 2003].
Data minimization, the third approach for addressing potential negative aspects of profiling, has been a tenet of privacy laws. Already the 1980 OECD guidelines [OECD, 1980] stipulated Data Quality as its second principle: “Personal data should be relevant to the purposes for which they are to be used”. The 1995 EU Privacy Directive [European Parliament and Council, 1995] was much more explicit, asking in its Article 6.1(c) that the collected that was not only relevant, but also “not excessive in relation to the purposes for which they are collected and/or further processed”. The same principle, sometimes called the “proportionality and necessity principle,” can now be found in Article 5.1(c) of the GDPR [European Parliament and Council, 2016] in slightly revised form: personal data shall be “adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimization’).” As we pointed out before, determining what data collection and processing is “relevant” and “not excessive” is far from trivial—in particular when collected with the intention of building a general profile of a data subject. It is worth noting that the principle applies to both the collection and the processing of personal data, highlighting the fact that data, once collected, may see multiple “uses” when a profile is used to infer a range of attributes.
Several authors have pointed out that “classic” privacy principles such as data minimization and consent may not be adequate for systems that rely heavily on profiles and data mining [Rubinstein, 2013, Tene and Polonetsky, 2012, 2013]. Instead, systems should favor accountability principles such as transparency, access, and accuracy:
The shift is from empowering individuals at the point of information collection, which traditionally revolved around opting into or out of seldom read, much less understood corporate privacy policies, to allowing them to engage with and benefit from information already collected, thereby harnessing big data for their own personal usage [Tene and Polonetsky, 2013].
It is obvious that data that has not been collected (data collection minimization) cannot be misused. However, given the likelihood that data processors will incentivize data subjects to consent to broad data collection practices (and thus will be able to build comprehensive profiles), ensuring subject-controlled use of the data (data processing minimization) is critical.
A pre-requisite to most privacy approaches discussed so far is a system’s ability to support such privacy-friendly behavior in the first place. Thus, an important question is how to effectively integrate privacy-friendly features into a system.
The systematic approach to such an integration is called privacy engineering. Privacy engineering comprises the “design, implementation and integration of privacy protection” in products and services [Gürses et al., 2015]. This goes beyond what is typically called “privacy-enhancing technologies” (PETs)—specialized privacy techniques or mechanisms that address a specific privacy problem. Those PETs—while often powerful—do not make it into the majority of products due to associated technical complexity. Privacy engineering thus is not only about technology, but about a process to integrate privacy protections into actual systems. Spiekermann and Cranor [2009] define two key aspects of such a system privacy integration:
1. ensuring that users can exercise immediate control over who can access their personal data (or, alternatively, who can access the person themselves, e.g., with a call or a message); and
2. ensuring that personal data, once disclosed, is protected so as to minimize subsequent privacy risks.
Depending on the actual system, these two functionalities can be located across three “domains” [Spiekermann and Cranor, 2009].
1. The user sphere: data located on a user’s device, or—in the case of physical privacy—the reachability of a user through this device.
2. The recipient sphere: data located in a third party’s backend infrastructure.
3. The joint sphere: data directly accessible to a user but technically hosted on third party infrastructure (often free of charge), e.g., free Web-based email or cloud services.
At the outset, privacy engineering seeks to enumerate protection goals, identify threats and vulnerabilities, and catalog technical solutions that fit the corresponding requirements in order to support engineers in building privacy-aware systems across IT processes. However, to be effective, privacy engineering also must include an understanding of end-user behavior and expectations, in order to provide the right level of support at the right time.
Spiekermann and Cranor [2009] point out that one can either integrate privacy protection into a system’s software architecture (privacy by architecture), or use legal and organizational means (privacy by policy) to ensure such protection. Privacy-by-architecture approaches represent protection services, usually cryptographic algorithms, such as zero-knowledge proofs, that prevent adversaries from making unintended inferences, as well as appropriate access control and security. Privacy-by-policy approaches represent deterrence services that use threats of disciplinary or legal consequences if individuals and organizations do not follow accepted or agreed-upon practices for the collection and use of personal data. Ideally, such privacy-by-policy approaches stipulate the creation of respective internal privacy policies, practices and processes within organizations.
Gürses and del Alamo [2016] add a third approach, dubbed privacy by interaction, which situates privacy supporting mechanisms at the socio-technical level, i.e., directly addressing the conceptualization of privacy as a contextual process [Nissenbaum, 2009]. When seen as a security service, privacy-by-interaction approaches work as resilience services that lower the impact of a data breach by virtue of taking user concerns and wide-scale impacts into account when designing such services in order to limit potential exposure of personal information. Gürses and del Alamo [2016] point out that successful privacy engineering requires all three of these approaches to be applied together, rather than individually (e.g., software developers addressing architectural problems, lawyers considering policy issues and UX designers investigating interaction challenges).
How system design and development can best integrate these three approaches is an area of active research and emerging professional practice. The basic idea of this approach—considering and integrating privacy at the outset of a system’s design—was suggested by Cavoukian [2009] with the concept of Privacy by Design (PbD). While Cavoukian’s principles suggest consideration of privacy at a high-level (e.g., “retain full functionality” and “respect user privacy”), they lacked concrete privacy-preserving tools and techniques as well as a systematic methodology to integrate them into a system’s lifecycle. The concept of privacy by design, however, has since been integrated into EU privacy law, notably Article 25 of the GDPR [European Parliament and Council, 2016] called “Data Protection by Design and by Default,” which requires data controllers to “implement appropriate technical and organizational measures […] designed to implement data-protection principles […] in an effective manner, and to integrate the necessary safeguards into the processing […].”
Danezis et al. [2015] offer eight practical privacy “design strategies,” derived from Hoepman [2012], as summarized in Table 5.1. The first four strategies—MINIMIZE, HIDE, SEPARATE, and AGGREGATE—are called “data-oriented strategies,” roughly corresponding to the concept of privacy-by-architecture and implementing data minimization. The latter four strategies—INFORM, CONTROL, ENFORCE, and DEMONSTRATE—are called “process-oriented strategies” and correspond to both the concepts of privacy-by-policy and privacy-by-interaction.
While these strategies offer a more operationalizable approach to privacy-by-design—in particular when combined with illustrative examples of technical and methodological approaches of each strategy, as provided by Danezis et al. [2015]—they still do not fully address how to integrate privacy engineering in the overall system design process. Kroener and Wright [2014] propose to combine best-practices, privacy-enhancing technologies (PETs), and privacy impact assessments (PIAs) in order to operationalize PbD.
Table 5.1: Design strategies for implementing privacy-by-design [Danezis et al., 2015]
# |
PbD Strategy |
Description |
1 |
MINIMIZE |
The amount of personal data that is processed should be restricted to the minimal amount possible. |
2 |
HIDE |
Personal data, and their interrelationships, should be hidden from plain view. |
3 |
SEPARATE |
Personal data should be processed in a distributed fashion, in separate compartments whenever possible. |
4 |
AGGREGATE |
Personal data should be processed at the highest level of aggregation and with the least possible detail in which it is (still) useful. |
5 |
INFORM |
Data subjects should be adequately informed whenever personal data is processed. |
6 |
CONTROL |
Data subjects should be provided agency over the processing of their personal data. |
7 |
ENFORCE |
A privacy policy compatible with legal requirements should be in place and should be enforced. |
8 |
DEMONSTRATE |
A data controller should be able to demonstrate compliance with the privacy policy and any applicable legal requirements. |
According to Wright [2011] PIAs started to emerge since the mid-1990s in countries around the world. In Europe, the UK privacy commissioner, the Information Commissioner’s Office (ICO), published the first handbook on privacy impact assessments in 2007 [ICO, 2007]. The handbook saw a first revision in 2009 and has since been integrated into the more general Guide to the General Data Protection Regulation (GDPR) [ICO, 2018], which the ICO regularly updates. PIAs draw on the tradition of environmental impact assessments and technology assessments, which have a long tradition in both the U.S. and Europe. Clarke [2009] offers a comprehensive history and Wright [2011] provides an overview of PIA requirements around the world, as well as a detailed description of typical PIA steps [Wright, 2013].
PIAs set out a formal process for identifying (and minimizing) privacy risks by drawing on input from all stakeholders. The key steps are [ICO, 2018]:
1. identify the need for a PIA;
2. describe the information flows;
3. identify the privacy and related risks;
4. identify and evaluate the privacy solutions;
5. sign off and record the PIA outcomes;
6. integrate the outcomes into the project plan; and
7. consult with internal and external stakeholders as needed throughout the process.
PIAs have since become a mandatory part of privacy legislation in Europe, the U.S., and other countries. For example, the GDPR [European Parliament and Council, 2016] requires a “data protection impact assessment” (DPIA) in Article 35, at least for processing that “is likely to result in a high risk to the rights and freedoms of natural persons.” While the GDPR does not mandate a formal methodology for conducting such a DPIA, its expected outcomes largely overlap with those of a PIA. Several authors offer advice on how to integrate PIAs into system development, e.g., Kroener and Wright [2014] and Oetzel and Spiekermann [2014]. In the United States, Section 208 of the E-Government Act of 2002 requires government agencies to conduct and publish Privacy Impact Assessments when developing or procuring information technology for the collection and processing of personally identifiable information, or when considering new data collection processes [United States Government, 2002]. Similar laws with PIA requirements exist in many other countries.
A more comprehensive privacy engineering methodology was proposed by Notario et al. [2015] as the outcome of a research project called “PRIPARE—Preparing Industry to Privacy by Design by supporting its Application in Research.” The authors propose a two-pronged approach that uses both risk-based privacy requirements analysis and goal-based privacy requirements analysis (see Figure 5.2). The authors map PIAs onto the risk-based requirements analysis, while legal requirements and codes-of-conduct are part of a goal-oriented requirements gathering process. Both sets of requirements can then drive a design methodology using either a top-down approach that instantiates an architecture from those requirements (e.g., following the CAPRIV framework), a bottom-up approach that works off existing code, or a “horizontal” approach (e.g., CBAM [Nord et al., 2003] or ATAM [Kazman et al., 1998]) that starts with an initial architecture and iteratively refines them based on the identified privacy requirements.
These approaches are just examples of the many methodologies for integrating privacy into the engineering process. However, this multitude does not mean that this is easy, but rather that the best solution has yet to be found. For current research on privacy engineering, the Workshop on Privacy Engineering (IWPE) series2 might offer a good starting point. In 2018, the International Association of Privacy Professionals (IAPP) also established a Privacy Engineering Section, dedicated to advancing privacy engineering practice.3
Figure 5.2: The PRIPARE Privacy-by-Design Methodology considers both goal-based and risk-based privacy requirement analyses (base on Notario et al. [2015]).
In this chapter, we provided an overview of paradigms, mechanisms and techniques to design and develop privacy-friendly mobile and pervasive computing systems. While there’s no silver bullet to remedy all privacy implications of any mobile and pervasive system, the presented approaches constitute an essential toolbox for building privacy into mobile and pervasive computing systems. An important first step for any system is the integration of privacy-friendly defaults. Data collection and use should be minimal—no more than necessary for the system to function—and potentially privacy-invasive or unexpected data practices should not be active by default but rather be activated by the user in order to help users in constructing a more accurate mental model of what data a system collects and how that data is used. Just-in-time mobile permission dialogs are a common example of this approach.
Rather than just informing users about data practices, associated privacy risks and implications should further be communicated to users at such occasions, in order to minimize surprise and help users make meaningful decisions regarding their privacy in the use of mobile and pervasive technologies. Furthermore, provided privacy information should always be actionable in order to assist individuals both in forming and expressing privacy decisions through privacy settings and options. At the same time, individuals are faced with an increasing number of privacy decisions. Machine learning can help provide personalized privacy assistance by learning from people’s privacy preferences and recommending likely preferred settings to individual users. Context-adaptive privacy mechanisms extend this notion to providing context-specific privacy assistance by leveraging context-aware sensing infrastructure and computing approaches to dynamically adjust privacy settings to context changes.
A privacy challenge that arises with automated systems, is that algorithmic decision making can incidentally codify discriminatory practices and behavior in algorithms. Thus, a substantial challenge for artificial intelligence research is providing transparency about how automated decisions are made in an intelligible and correctible fashion. Designers and developers of mobile and pervasive computing systems—which derive much of their benefits from ingesting sensor information and inferring state and required actions—need to consider how they can detect and eliminate undue bias in their decision engines, provide algorithmic transparency to users, as well as offer opportunities for correcting errors as well as provide redress processes for harmed users.
The privacy challenges and approaches discussed put a lot on the plate of a designer or developer of a mobile or pervasive system or application. Engaging in privacy engineering and considering privacy from the beginning, can help surface privacy issues and challenges early-on in the design, as well as facilitate the consideration of creative privacy approaches and solutions. Similar to other engineering practices and considerations, relying on established frameworks, such as privacy impact assessments, can provide structured guidance in the process and enable reliable and consistent privacy outcomes.
1An appropriation tort provides for “liability against one who appropriates the identity of another for his own benefit” [Mc-Clurg, 2003]. It is typically used by celebrities for safeguarding their “brand value” when others try to sell a product or service based on the celebrity’s likeness (e.g., image).
2Workshop on Privacy Engineering Series: http://iwpe.info/.
3IAPP Privacy Engineering Sec.: https://iapp.org/connect/communities/sections/privacy-engineering/.