Centaur Health

Most people know the story of a Russian chess grandmaster Garry Kasparov, former world chess champion playing a machine in chess. Although Garry would win the first match in 1996, he lost the rematch, to IBM’s Deep Blue, a year later. The symbolic significance was enormous. It signaled to many that machines, computers, AI were catching up to humans. After all, Garry previously exclaimed he would never lose to a computer. The story of Deep Blue and Kasparov tells us a lot about machines and intelligence, and how artificial intelligence will manifest in healthcare and why it must be human-centered.

Out of Kasparov’s defeat, he developed a new kind of chess tournament and a new type of chess player illustrating reciprocity between man and computer, centaur chess. Introduced by Gary Kasparov, after his defeat by Big Blue, Kasparov set out to demonstrate that if you combine human and computer, you achieve better results than either alone. Teaming machines and humans in chess create a better chess player than either humans or computers can manage on their own.¹ Centaur chess is all about augmenting and improving human performance. That is, you increase the level of performance and reduce the number of errors, thereby improving the quality.

In a like manner, AI combined with clinicians will amplify the skills and capabilities of human doctors, other clinicians, and providers of healthcare services. In his 2013 book, "Smarter Than You Think,” Clive Thompson describes a new style of intelligence where we can augment human minds in innovative ways. Today is the age of centaurs for healthcare. Intelligent systems and AI systems of care can transform healthcare.

Deep Blue exhibits AI², because it beat a grandmaster at chess, something previously only humans could do. Watson demonstrated intelligence because it best the most winning Jeopardy players in a trivia game, already something we thought could only be done with human knowledge. AlphaGo showed intelligence and is an AI system beating an 18-time world champion Lee Sedol in a 2500 old strategy game, Go.

In each win by AI over humans, we have some common AI characteristics, which by the way also describes the limits of AI today, the key to implementing human-centered AI:

1. AI performed a task, previously only ever done by people and a responsibility we attributed to human intelligence

2. AI performed a specific, well-defined task, a single job, and that job was not to show human cognitive capabilities but simply to win the game at hand;

3. If humans don’t think about it, AI can’t do it; that is, each intelligent entity was purpose-built to show human “intelligence” for a particular task

4. AI has enough evidence, data, about a subject to know it has a high probability of being correct but does not have enough knowledge, data to understand it’s wrong

5. Humans were required to make the AI system work and perform their single task

Let’s compare these AI characteristics with healthcare which helps us understand the symbiotic relationship between humans and AI, human centered AI healthcare.

AI Characteristics: Examples in Games (e.g., Jeopardy)	AI Characteristics: Examples in Healthcare
AI was tasked with a single goal (e.g., win the Jeopardy match or best Kasparov at chess) AND it was a goal previously only ever done by people AND we attribute achieving the goal as exhibiting human intelligence	AI is tasked with a single goal, diagnosing a specific disease based on a medical image. The goal might be detecting a specific disease, or cancer. In many cases AI outperforms humans.a Often, unique AI entities are necessary per goal; the AI detecting cancer is different from the AI detecting diabetes.
AI performed a specific, well-defined task, a single job, and that job was not to show human cognitive capabilities but simply to win the game at hand	The AI diagnostic tool does a single task, detects a specific disease state using a medical image, or other factors in an algorithm. AI doesn’t show any human cognitive capabilities like making treatment decisions based on the diagnosis.
If humans don’t think about it, AI can’t do it; that is, each intelligent entity, AI, was purpose-built to show human “intelligence” for a very specific task	The AI diagnostic tool detects disease states, purpose built, but since the AI tool was not designed to make treatment recommendations it has no such capability. That is, if humans don’t instruct or tell AI what it has to do it does not get engineered to perform what it hasn’t been designed to do.
AI has enough evidence, data, about a subject to know it has a high probability of being correct but does not have enough knowledge, data to know its wrong	The AI diagnostic tool cannot deal with missing data and it may not be able to accommodate conflicts in clinical data from a doctor patient interaction. So the AI tool doesn’t know it doesn’t have enough COVID-19 pneumonia images to distinguish bacterial pneumonia from COVID-19 associated pneumonia. Or the missing data that would change the diagnosis based on data from a clinical interaction.
Humans were required to make the AI system work and perform its single task	Clinicians must ensure the AI tool is as unbiased as possible and that the datasets used by the deep learning model represent current standards of care. Subject matter experts must make sure a medical image dataset used to train a machine learning model has the proper amount of medical images accommodating both the obvious and not so obvious.
a https://www.medicalnewstoday.com/articles/326460

The Age of Centaurs, Centaur Healthcare, keeps humans in charge of decision making, the agency is always with the doctor, and now AI augments physician’s capabilities. An example of Centaur Healthcare is a true story of a patient we describe as Patient K. He visited his primary care physician (PCP) because of a lump growing on the left side of his neck. His PCP examination concluded it was a lipoma, fatty tissue, and harmless. Patient K, a few days later, in the presence of a doctor trained in internal medicine, an endocrinologist, saw the lump and expressed a concerned comment. Patient K and his wife said, “Oh, it’s nothing. Our other doctor felt it and concluded it is harmless fatty tissue.” The endocrinologist asked Patient K if he would mind if she touched the lump on his neck, and he obliged. To her, it felt hard like a tumor and not rubbery and mobile, like a benign fat mass. She suggested Patient K get a CAT scan, but he and his wife were not in favor after all their PCP had assured them the lump was harmless. After much prodding by the endocrinologist, they reluctantly decided to get the CAT scan, and it found that it was in fact, likely a cancerous tumor. Early detection and surgical removal meant this cancer was able to be resected/removed while it was still localized and Patient K did not need more systemic therapies, such as chemotherapy. The couple, to this day, are grateful to the endocrinologist for her insistence on getting the CAT scan. The story informs us that we need to raise the capabilities of primary care physicians so that they, too, see what a specialist sees. Doctor’s have a common saying, “if you don’t think about it, you can’t find it.” The story tells a tale of why augmentation of doctors with AI plays a role. The marriage of a PCP with AI tools, centaur healthcare, can make a huge difference.

Centaur Health is essential, as in some countries, like the USA, the confidence patients have in their primary care physician may not always be warranted. In China, just the opposite as the population at large recognizes the difference between their general practitioners and specialists, as the differences in professional training between the two are vast. China is looking to use AI to correct these imbalances of training.³ China has a shortage of general practitioners and knows they must improve effectiveness without having all general practitioners undergoing another three or four years of training.

The same challenge is real in the US, although not widely known by the general population. In the US, a PCP generally completes medical school and a three-year residency. A specialist like an endocrinologist completes the same education, including residency, plus an additional three to four years of training. As a result, the knowledge base difference between a PCP and specialist can be huge. Patients like Patient K, think the training is the same. Common ailments are typical, but specialists see rare things more often than a PCP. For them, unusual things are ordinary. Because specialists tend to have more training, they “think about it,” so they can “find it.” We need AI to lift and raise the knowledge base and capabilities of doctors. The next section defines more clearly human-centered AI.

Human-Centered AI

J. C. R. or “Lick” is not a well-known figure in the world of artificial intelligence. He has been called computing’s Johnny Appleseed for planting the seeds of computing in the 1950s and 1960s. Lick, Joseph Carl Robnett Licklider, wrote a paper in 1960, Man-Computer Symbiosis, where he wrote about the symbiotic relationship between man and machines, writing “... men will set the goals, formulate the hypothesis, determine the criteria, and perform the evaluations. Computing machines will do the routine work ….” Lick saw a world in the 1960s where humans and machines were not competing like playing chess or AlphaGo but working together to amplify the strengths of both, intelligent amplification (IA). Today, we would most likely think differently about artificial intelligence if IA was the moniker of choice versus AI. Lick saw a cooperative interaction between humans and machines, the basic premise of human-centered AI.

Computer scientist, professor, and co-director of Stanford’s Human-Centered AI Institute,

Fei-Fei Li makes the point that AI is very task-focused but lacks contextual awareness and lacks the flexible learning seen with humans. In a 2017 MIT Technology Review article, Fei-Fei quoted a phrase from the 70s. “The definition of today’s AI is a machine that can make a perfect chess move while the room is on fire.” She goes on to say that if we want to make AI more helpful, we must bring back contextual understanding, and this is especially true in healthcare.

The quote from the 1970s stresses the importance that AI integration and adoption in healthcare must be human-centered. What does human-centered AI mean? As we integrate and connect with AI on a more routine basis, human-centered AI focuses on creating AI systems for human consumption. AI must focus on the impact such technology will have on humans. From this perspective Mark O. Riedl in Human-Centered Artificial Intelligence and Machine Learning⁴ breaks human-centered AI down into two components:

1. AI systems that take into account the human sociocultural perspective, and

2. AI systems that facilitate human understanding of these systems, i.e. explainable AI.

There is growing awareness that advances to AI alone are insufficient when building applications designed to augment human tasks. For example, if I am developing an AI system of care to work with elderly patients and my AI engineering team comprises a brilliant team of engineers and scientists. Still, my average team’s age is in their mid-thirties. The team’s educational background, work experience, and body of work are impressive. However, they’re still lacking a key, active ingredient for developing a human-centered AI system of care for seniors. Not a single team member has ever walked in the shoes of an older adult. Human-centered AI finds a way to ensure that we build systems accounting for sociocultural perspectives such as age, ethnic background, socioeconomic status, and more.

Our energies and design of AI systems should start with the intersection of people and machines. That is, understanding the symbiotic relationship between AI and humans helps us build better AI applications.

AI and Human Socio-Cultural Values

Humans can be equally confusing to intelligent systems, AI. Natural language processing (NLP) and speech processing, using automatic speech recognition systems (ASR), show understanding of the English language. However, it may not understand all English language speakers. Stanford Engineering researchers show that the AI embedded into many ASR systems has twice as many errors understanding the exact words spoken by African Americans as when interpreting words spoken by whites.⁶ An example of socio-cultural values not being accommodated by AI.

The negative effects to people where companies use ASR technology, can be devastating. The impact of this disparity of not properly recognizing nonwhite speakers may have significant negative consequences on people’s job prospects and lives. Does the company know the ASR is not reliable? Do they know its error rates depend on the socio-economic variables like class and race? Disabled citizens accessing healthcare services through voice suddenly may not have access to working healthcare services. A likely remedy is adding a lot more voice data for training of the speech recognition systems with the diversity of dialects from a diverse set of English speakers and not just white Americans.

AI also needs to understand human motivation or behaviors underlying their actions, the sociocultural perspective of humans. Our rearing, generation, and geography impact our social practices and decisions. Human-centered artificial intelligence means building AI applications and corresponding algorithms with the understanding that AI is part of something bigger, comprising societies with humans. If the ASRs employed human-centered AI thinking and practices, the technology would have been tested using voice data from a broad spectrum of citizens. Ensuring it worked or placed a warning label, “works only for native English speaking white Americans with no accents.” Of course, the latter would be absurd, but it points out that we must build AI systems with a broader and more precise understanding of their usage.

Human-centered artificial intelligence in healthcare requires us to design and build AI tools and applications with the awareness of the impact on stakeholders who come in contact with the AI. In healthcare, that connection could be direct (the doctor using an AI diagnostic tool), or indirect (a patient who receives a diagnostic result not knowing it was the output of AI versus an actual doctor examining a test or procedure result). AI design must accommodate an awareness that the AI is part of a more extensive system consisting of human stakeholders: nurses, patients, clinicians, operators, clients, and other people. Some AI researchers and practitioners use the term human-centered AI to refer to intelligent systems designed with social responsibility in mind. They are addressing issues of fairness, accountability, interpretability, and transparency.

If AI can account for sociocultural realities, then decision making and a problem solution may be more understandable to humans. When we ignore these sociocultural beliefs, AI misses or mistakes occur. For example, when we leverage AI to detect people with high potential for comorbidities, we must understand the AI goal. Is the purpose of the AI to make people healthier, or is it to reduce costs? In the former case, the AI would prioritize those at highest risk to receive healthcare. In the latter case, the AI would appear discriminatory if it prioritized on the low end those at highest risk with likely poor outcomes, to receive the least amount of healthcare.

Consider the elderly patient with the newly diagnosed cancer that we introduced in Chapter 1 as an example of where AI fails. In that example, we had an otherwise healthy elderly patient with a new diagnosis of easily treatable cancer. AI without knowledge of human values would not take into account that human autonomy in decision making would take priority over the logical decision to treat the disease. That is, AI cannot recognize that the patient’s age or quality of life may want them not to consider cancer treatments. The scenario is not technically a miss on AI’s part. Decision and logic would argue for treatment, but a gap appeared, because AI did not take into consideration the very human concept of autonomy in patient choice. Human-centered AI would factor in socio-cultural values, such as patient autonomy, in the creation of an intelligent structure addressing patient decision support.

In this patient example, clinical knowledge of cancer treatment and factoring in known prognoses and treatment strategies encompass the explicit understanding of AI. What was missing was the implicit recognition of the importance humans give a patient’s right to decide on treatment. Human-Centered AI works to bring that tacit knowledge into the creation of our intelligent structures. Currently, AI is capable of tasks associated with explicit and implied human knowledge. Explicit instruction is generally accepted wisdom and could be considered “book learning.” Implicit knowledge often derives from intuition. Intuition relies on experience both in time and of varying factors that lead humans to have an instinctive understanding of a situation or problem. Explicit learning informs the physician that the patient warrants clinical treatment. Sociocultural values and implicit learning leads to withholding treatment. But, AI is quickly gaining a tacit understanding of humans.

The growth of affective computing is a complementary field to AI addressing the study of systems and devices which can recognize and interpret human experiences of feelings, emotions, or moods. The growth of affective computing combined with AI facial recognition, biometric monitors, may, in some cases, substitute for markers of human emotion, the ability to interpret facial gestures or expressions without the spoken word.

This newer generation of AI makes it tempting to state that AI has an understanding of humans. But this would not be accurate. AI demonstrates human-like implied knowledge. AI must be constructed and trained using data that encodes this implicit knowledge. Data based on human interactions and experience behind the scenes. While big data and AI algorithms, along with deep and machine learning, enable the creation of AI processes simulating human-like understanding, this is different from originating meaning. That is, AI does not have an inherent knowledge of humans. Humans create the expertise and constructs for AI to understand, which is why we must recognize that AI must understand people.

AI Understanding Humans

Several AI associated failures are attributable, at least in part, to a lack of human-centric design. One study estimated that 60% of “big data” projects fail to be operationalized, which is put into production for clinicians or others to experience and use. For example, in 2013, there was great excitement as Watson for Oncology (IBM) partnered with MD Anderson in using Watson’s cognitive computing to help eradicate cancer by identifying the best treatment options. Patient and research databases were used by Watson to recommend cancer treatment. What happened? The joint venture between the University of Texas MD Anderson Cancer Center and IBM collapsed, and the reasons differ based on who’s talking. This collaboration is a case study on why human-centered AI is required upfront in enabling AI systems to delight and meet stated goals. Human-centered AI requires a partnership with all parties as they succeed or fail together.

Physicians look at individual patients, initially through the lens of the immediate underlying disease process but also then expand this to a holistic analysis as all parts of an individual can be involved in and impact disease management. Holistic patient analysis examines among other things, medical factors, medications, safety, social stability, and support. Physicians consider behavioral health factors, baseline health before disease onset, socioeconomic background, and values. The patient relationship/trust in their physician partner, access to treatment facilities and location, and numerous other variables doctors include in the physician’s holistic analysis. These factors will have varying significance and weight in the treatment decision based on the explicit and implicit understanding of the patient and the disease process by the physician. When so many variables are not taken into account by AI, as perhaps in the MD Anderson IBM Watson collaboration, how could humans have imagined a functional system? Constructing AI accounting for these variables requires cooperation and meeting of the minds between engineering and stakeholder. Stakeholders must consider the user’s limitation of understanding AI and ensuring AI had the constructs relevant to a human-like decision that was lacking.

Because humans create AI constructs, human bias is inevitable in AI models. Let’s contrast two differing goals, using the same AI model: (1) AI goal is to help diabetes management in a self-funded population, a population where the employer assumes all health care costs for their membership population, and (2) AI goal is to help diabetes management for Medicaid or Medicare where the government pays for a majority of that member’s healthcare. The underlying population types vary significantly in characteristics, and this impacts their management and how the model gets used.

A self-funded population is employment aged, generally younger, and assumed to be healthier. In contrast, a Medicare population would be older, retired, and more likely to have chronic medical conditions. So an AI strategy addressing one community would potentially be inappropriate for another population. As an example, AI solutions detecting and recommending treatment plans will be different. Nurses might need to be more proactive and plan in-home patient visits for the more senior population. Dealing with comorbidities, aging at home, or even the primary modality of engagement might vary. Some age groups will be more comfortable with human interaction when addressing healthcare issues by phone than say an intelligent AI virtual assistant. People born post-2010 growing up with AI may respond differently than people lacking trust in AI. Again, AI solutions cannot be divorced from human understanding and sociocultural variables.

The medical community knows that average laboratory values, or test results, are not based on the typical findings of the national population. Instead, an average laboratory value range reflects the average values for a middle-aged caucasian male as this was determined to be the norm during the initial laboratory standard setting. AI use of these lab values in any construct, even a human-centric construct, would be limited by this inherent bias in the underlying data. That is, ethnic differences are often not reflected in the widely adopted average laboratory ranges, which would potentially result in lower-quality healthcare.

Ethnic-specific laboratory ranges may improve the diagnosis of disease states resulting in improved monitoring of patient’s health, better clinical decision making, and improved healthcare in general. Why is it so crucial that AI tries to work beyond these biases and apply socio-cultural human relevance? One main reason is goal relevance. AI applications are most valuable when they are relevant to human end users.

For example, let’s take a simple automated system where AI is used to identify and flag abnormal White Blood Cell Count (WBC) lab values for clinicians to address. WBCs help to fight infection and can also be a sign of inflammation. Typical values are usually 6,000-11,000 WBCs per microliter of blood. If I were an average Black male, then my White Blood Cell Count (WBC) of 3,000 WBCs per microliter of blood could be within the normal range for my racial group. However, if the example AI construct were to apply the general laboratory norm for White Blood Cell Count, then as a Black male, my lab value would be considered low and abnormal. Taking this further, as an individual patient, my historical WBCs may always run between 3,000-3,500 WBCs per microliter of blood. If I were to be ill with pneumonia (which typically causes an elevated WBC), my WBC of 11,000 WBCs per microliter of blood would be considered within the normal range, when in actuality, this laboratory finding would indicate an infection and a significant rise in my WBCs from baseline.

The relevance of the result is essential. In the first example, as a Black male, I could be subjected to expensive and potentially injurious further testing, including possible bone marrow biopsy. In case two, as an individual with “low” WBCs at baseline, we might overlook a significant marker of severe infection. Given that specific context is needed to interpret laboratory findings in the examples above, similarly, AI cannot provide relevant solutions without human regard.

Another reason AI’s understanding of human context is essential is in the case where human handoff must occur. However, when human-centric AI is applied, AI is aware of its limitations. We design AI to trigger handoff to humans for further contextual analysis. That is an application of implicit knowledge to obtain an adequate solution for the patient. Consider the second example above. In the example, the intelligent lab system is given the additional information that normal WBCs ranges may differ by ethnicities and gender. Handoff to a clinician for further review would show that this was ordinary, and the patient would not undertake unnecessary testing. Human-centric AI can help to determine based on the human context where handoffs for human intervention should occur and when.

As another example of where AI can go wrong without the inclusion of human norms, in 2016, Microsoft made headlines announcing their new chatbot, Tay. Tay utilized teenage voice and jargon and could automatically reply to people and engage in casual “fun” conversation on Twitter. In less than 24 hours, internet trolls corrupted Tay’s personality by flooding Tay with racist, misogynistic, and anti-semitic tweets. Chatbot Tay went from altruistic human loving chatbot to full-on racist, based on learnings from trolls, and so Tay was deactivated. Tay, through automated algorithmic decisions, reflected and amplified undesirable responses or patterns based on conversations with Tay’s users. If Tay’s design addressed the inclusion of societal norms (rules of human society), then Tay may have been highly successful. Human-centric AI could address these types of issues by creating feedback loops to inform the intelligent system of potential underlying behavior and societal bias encoded in the responses.

Humans Understanding AI

Humans understanding AI is critical to AI success and adoption in transforming healthcare, and we describe this under the umbrella of explainable AI. When a plane goes down unexplainably, the Federal Aviation Administration will look at the “black box” data to help determine where a process or system went wrong. Likewise, in our automated intelligent systems, AI, when a breakdown occurs, we as humans want to know why. Often AI neural networks show up as un-interpretable, just as in a black aviation box. It may take a tremendous amount of time and energy to understand an autonomous AI decision-making process or why a specific outcome came about. It is usually the AI experts who perform this type of analysis, and their goal is generally to debug or improve a particular AI system. The work of making AI understandable to non-expert humans is called explainable AI.

As in other AI applications, the issue of transparency is a human problem and a technical one. Currently, many of the algorithms used in AI are not able to be examined after the fact to understand how and why the AI results were achieved. From a healthcare perspective, the goal of explainable AI is to provide enough education about these intelligent systems to understand how they work in general. This understanding, as described previously, must be to all stakeholders affected or using the AI. In practice, providing an understanding of the underlying hypothesis, assumptions, goals, datasets, and scope of the model helps with model usage. Once the AI establishes trust, it is easier for humans to adopt the AI. There is natural skepticism for acceptance of any system that is not understandable to the human end-user.

There are various options in helping humans understand intelligent systems. All focus to some extent on the amount of information that must be shared to create understanding, and therefore, acceptance. One option in helping to explain AI is to develop descriptions of how the algorithm processes input. Describing the goals of the model, its intended use cases all help explain the model. Another is developing rationales for why decisions come about. We produce rationales through experiences collected by explanations of humans performing similar tasks to intelligent systems. Intelligent systems can then take these human explained examples and translate them into automated reason that is similar to social rationales, including culturally specific idioms. However, some intelligent systems defy a natural explanation. In a sense, this is not unlike human processing. Doctors’ clinical decisions may be apparent at times and almost automated as they follow clinical practice guidelines for a specific disease condition, such as hypertension. Once complexity is added to that patient story, then our human thinking may not be so transparent or “explainable.”

Consider this AI system example, where AI is used in determining the site of discharge from a hospital setting, that is, home, nursing home, or rehab facility. A 65-year-old patient with known hypertension gets hospitalized with a stroke. AI and human decision making dictated that while the member was still neurologically impaired to a significant extent, he should have remained hospitalized for his safety. However, the patient left the hospital against medical advice. While at home, the patient then called to be admitted for further care at a rehabilitation facility. The standard process is to move from acute inpatient hospitalization straight to a rehabilitation facility with no intermittent stop at home. It is exceedingly difficult, perhaps nearly impossible, to admit a patient from home to an acute inpatient rehabilitation facility. However, the patient was admitted that day to such a rehabilitation unit--this was the benefit of human intervention.

This example focuses on diverse human-centered AI needs. Performing optimal decision making requires factoring the entire human context or holistic experience. Knowing when a human should intervene is part of human-centered AI. Lastly, how much information and in what format would information sharing occur to help the human end users feel educated on why this decision resulted?

To account for why a direct admission to a rehabilitation facility was allowed, one has to understand the patient context. This patient had witnessed the violent death of his son, causing his blood pressure to skyrocket out of control, which led to a major stroke. The patient left the hospital against medical advice because of his son’s scheduled funeral that day, and the patient refused to miss attending his son’s funeral.

Human intervention looks at the context, including events in the patient’s life. Human intervention understands and would override AI allowing the irregular direct admission from home to a rehabilitation facility. The AI system had never seen an outlier such as this case; it is not built into the AI. The human analysis allowed for an irregular decision outcome. The human socio-cultural values were given precedence, and this unlikely outcome was allowed to occur, which ended up being the optimal treatment decision for this patient. Human decision making focuses on a careful weighing of all available evidence to come to the best conclusion. Ignoring the way humans make decisions would have resulted in a significantly more costly outcome, both financially and emotionally, for this patient. The patient avoided a repeat emergency department evaluation, ambulance transfer, possible repeat inpatient hospital stay, and emotional toll from repeating a process that occurred and was associated with his son’s death.

Explainable AI is at play in this scenario because we did not allow AI to be in decision making, and it was apparent to the clinicians why AI made its recommendation adhering to clinical guidelines or hospital guidelines, but the doctor could override this decision because of context. Doctors understanding why AI made its initial recommendation and supporting evidence helps doctors override when necessary.

Humans understanding AI also focuses on the ethical issues related to a lack of knowledge of AI applications. If human end users do not understand at some level, how decisions regarding social care or therapy are determined, then how can we ensure that ethical decisions are made, without bias, or commercial motivation (how would one know their insurer has not created AI systems focused on lowest cost versus best care). In the above example, how much information would have needed to have been shared with humans to make the outcome understandable? Given that this information was highly sensitive, what are the ethical responsibilities in sharing this information to enhance end-user understanding?

Human-Centric Approach

We are seeing a significant trend where a majority of the workforce is composed of people of color.⁷ Most cities in the U.S will no longer have a racial majority by 2030, according to U.S. Census projections. This demographic shift makes lack of diversity in AI systems more obvious. There is a lack of race and gender diversity in creation of intelligent systems. Though this bias may be unintentional, a majority of engineers are non-diverse and the systems they are creating do not reflect the cultural diversity of the social systems their solutions are meant to address. Another way in which racial bias can be injected into AI is that algorithms are designed on available norms,such as, our normal range of White Blood Cell counts mentioned previously, which are based on caucasian middle aged male findings. This creates an inherent bias when intelligent systems utilizes these norms in its processes.

Several corporations champion the examination of implicit bias training and have numerous diversity initiatives. The Algorithmic Accountability Act, legislation introduced to both houses of Congress in 2019, charges the Federal Trade Commission with the assessment of algorithmic bias and allows the federal agency to issue fines based on company size. Nkonde states that even with the steps made to address diversity, we “have failed to move the needle on creating a tech workforce that looks like its users.” Mutale Nkonde, co-author of the legislation, argues for racial literacy and the concerted effort by industry and tech to create a framework to support diversity in its engineers and products. Although at the time of publication this bill has not been enacted into law, it raises an interesting issue of holding algorithms to different standards than humans and enforcement presumably would be through existing practices by the federal government.

Further ethical issues arise when individual autonomy or privacy and AI intersect. Harvard Business School professor, Shoshana Zuboff, in her book The Age of Surveillance Capitalism describes a new age of capitalism. Capitalism combines many data points, including security surveillance cameras, smart home devices, smartphones, biometric devices, and social media used to obtain data on us as individuals and to make predictions about our lives and how we will behave as consumers. Zuboff writes this is to “know and shape our behavior at scale.”

This form of capitalism would consider personal data as a free raw resource to be translated into behavioral data. The ethical threat is that when our behaviors can be predicted and shaped by a handful of top corporations, we humans will face a sense of hopelessness and no longer be in control of ourselves and our behaviors. Humans risk being manipulated by those who control the data for their own economic advantage. Furthermore, we no longer have privacy. All our personal data is used as a raw material without any individual autonomy in its use or access.

Further ethical considerations focus on power. Vladamir Putin said that the “nation that controls AI would control the world.” Elon Musk said, “the AI arms race will lead to World War III.” Musk and 4500 AI, and robotics researchers have signed a Fight for the Future open letter in opposition to autonomous weapons that act without human intervention. There has been a proliferation of AI among national militaries, including China and Russia. The National Security Council on AI was created by Congress in 2018. Ethical issues under consideration include how AI will shape the future of power in our countries. To address some of these issues, a board made up of technology executives and AI ethicists and experts created recommendations for AI ethics principle recommendations for the Department of Defense.

The complexity of the human context that AI must work with and for is challenging. These ethical considerations and potential fears must be addressed proactively through human-centered AI to ensure limitations and enhancement with collaboration with humans occurs. To this end, the result is the ultimate collaboration between humans and AI rather than the extreme world of domination by the few data hoarders and powerlessness for the rest. Retaining the essential aspects of work like empathy, touch, and more in healthcare is essential to making human-centered work.

Making Human-Centered AI Work

There is a general agreement that maintaining the human element in the way AI is designed, delivered, used, and improved will make it more successful. We want AI to be understandable by non-expert users and be designed with social responsibility in mind. To go about this, a true collaboration between humans and AI must occur.

Research on 1500 companies found that performance indices were most enhanced when AI and humans worked collaboratively. Humans and AI have a greater value together more significant than their separate parts, and their strengths and capabilities augment performance when combined. Harvard Business Review (HBR) makes this point in describing collaborative intelligence, humans, and AI joining forces.⁸ Wei Xu takes this even further and states that the first challenge is to move beyond initial interaction and to take it to “human-machine integration and human-machine teaming.” To overly simplify, the basic concept would be humans and AI working together in a dynamic relationship where dynamic data inputs result in dynamic goals and ongoing micro-adjustments to the relationship where each player takes priority in their role addressing the function they are most adapted to handle.

An example of how this type of human-centered AI occurs currently is through retinal scanning. AI performs the initial assessment of normal versus abnormal findings. The vast majority of scans will be normal. The remaining 30% or so of abnormal findings are reviewed by a human. In this case, human-centered AI focused on where the limitations to its capabilities occurred, the 30%, and then handed off to its human partner. Out of this is a partnership between humans and AI, where AI performs quickly, efficiently, and accurately the task it is assigned, and the human performs their more complex nuanced evaluation taking in additional human factors that may impact the patient solution. Clinicians benefit in that they are no longer reviewing all of the scans and can focus on the scans that require their human capabilities. One can imagine future models where AI is used to learn to assess other types of abnormal scans as well, and as its fluency in this area increases, then dynamic goals and shifts in the human-AI partnership will occur. As human experience and clinical medicine are dynamic, AI will not replace the human, rather through this symbiotic relationship, the best outcomes will be achieved.

Call centers or customer service centers in healthcare leverage AI using chatbots or virtual assistants. When chatbot use replaces human agents, the adoption and acceptance by customers often fail. The implementation of AI without the human-centric focus appears to be a root cause. Chatbots designed to replace humans, goes against the principles of human-centered AI, not to replace humans but replace or augment human tasks. Several technological challenges must be overcome for chatbots to mimic human behavior, and we will explore this in more detail in Chapter 6, Emerging Applications of AI. However, virtual assistants and chatbots for call centers or customer support have been successful. Still, often, these compliment call center agents or operate as an alternative, not a replacement means of engaging.

Human-centered AI utilizes AI’s strength of pattern recognition to address changing the way hospitals triage. AI can sift through a patient’s data to assess which patient is most in need of care within the next ninety minutes. This enables clinicians to focus on the care they are providing and less on logistics regarding who requires attention next. One can envision an ongoing dynamic relationship between humans and AI here. As clinical medicine advances and fewer people require actual hospitalization, a sicker population ends up presenting to emergency departments.

Consequently, a large number of resources triaging patients, determining who is sick versus not that sick gets harder. The current practice focuses on biometric variables to determine who is most unstable and therefore requiring more immediate care. As additional data inputs arrive interpretable by humans, such as pain intolerance or risk of a dangerous outcome, AI accommodating these signals makes a difference. AI will be able to hone its analysis, and caregivers will be able to focus on the sickest of the sick. In the ideal state, feedback loops exist where other implicit variables, including socio-cultural values impacting outcomes, are added to the AI knowledge base. The identification of AI limitations would result in both human handoffs as well as oversight.