CHAPTER THIRTEEN

E-HEALTH CARE TECHNOLOGY MANAGEMENT

A Multifactorial Model for Harnessing E-Technologies

George Eisler, Sam Sheps, Joseph Tan

I. Learning Objectives

II. Introduction

III. Multidimensionality of the E-HCTM Concept

A. The Strategic Role of E-Technology

B. E-Health Care Technology Management Strategy

IV. E-Surveying Health Executives

A. Defining Health Care Technology Management

B. Pilot Test and Field Test, Using a Delphi Approach

C. Validity and Reliability Issues and the Gap Score

D. Web-Based Survey Design

E. The National Survey

V. Research Findings with Relevance for E-Health Care Technology Management

A. Factor Analysis

B. Cluster Analysis

C. The Hay Group Study

VI. Conclusion

VII. Chapter Questions

VIII. References

IX. Evidence-Based Medicine Case

Learning Objectives

1. Conceptualize e-health care technology management
2. Recognize the benefits and challenges of e-surveying health administrators and executive team members
3. Identify the perceptions of senior health care executives on technology management issues and interpretations of expert opinions and ratings
4. Understand the relationships of HCTM research findings to the results of the Hay Group Study
5. Associate HCTM research findings with an e-HCTM context

Introduction

Human history and development have always been linked dynamically to the technology inherent in tools and means of production. The survival of individuals, clans, tribes, organizations, societies, and empires depends on the power of their technology to harness nature and their environment. The evolution of human civilization from the hunter-gatherer stage to the industrial stage took almost two million years. Amazingly, the evolution of computing and automated information processing technology has taken no more than a few decades, following the Industrial Revolution, two World Wars, the Cold War, and the race to the moon. In the last decade, this trend of accelerating change has been further fueled by instant access to worldwide information, global competition, and the pervasive power of converging advances in computing, information and telecommunication technology, and biotechnology.

The e-health paradigm shift, the topic of this text, is another revolution in the human history of technological developments. The view that e-technology is just an implementation issue or just another operational requirement vying for resources may be one of the key reasons for the current poor coupling of e-technology and e-health care. This view sees e-technology merely as a tool to implement e-health care strategies. It assesses e-technology in terms of return on investment or in terms of satisfying current e-market needs, covering such aspects as identification, selection, acquisition, exploitation, and protection of e-health product or process technologies. Although such tactical e-technology plans are useful (Gregory, Probert, and Cowell, 1996), more compelling is the potential and power of e-technology to radically change clinical and business strategies in health care, not just support e-health systems that mimic traditional systems. Indeed, e-technological innovation has already shifted the competitive balance within the health care industry and is creating more new opportunities for growth, as previous chapters of this text have discussed.

Economists such as Tapscott and Caston (1993) have pointed to technology as an important change agent in the structure of industries and competition. Andersen, Belardo, and Dawes (1994) confirm that the issues are similar in the public service sector arena: “Public expectations for the level and quality of government services were formed in better economic times. Those expectations have grown while satisfaction with their fulfillment has steadily declined. In the past few years, it has become evident that cutting fat, eliminating waste, and preventing abuse is not nearly enough. Government needs to rethink its methods and restructure its approach to public services.” Around the world, countries are recognizing that the competitiveness of their health care products and services in the global marketplace depends on their focus on e-technology management. In the e-health environment, the task of managing applications and services is particularly complex. It requires that health care executives master many different skills, including government relations, community liaison, employment of human resources in e-work, financing of e-health business initiatives, e-patient care, research on e-technologies (for example, research based on linked databases), and on-line education. E-health care technology management (e-HCTM), therefore, adds one more dimension to the challenge of harnessing IT for health care in the new economy.

In recent years, e-HCTM and mainstream health care technology management (HCTM) have been receiving attention in developed countries (for example, Japan and countries in Europe and North America) as well as in developing nations (for example, Southeast Asian countries). The World Health Organization (WHO), for example, proclaimed that there were serious shortcomings in the performance of health systems in virtually all countries (World Health Organization, 2000). In the late 1980s, the WHO admitted that its attempts to introduce components of an HCTM system around the world had not been very successful (World Health Organization, 2000). The lack of a working HCTM model or framework and a shortage of technology management skills, expertise, and knowledge among workers in those countries were identified as serious limitations. Without a functioning HCTM or e-HCTM system (incorporating, for example, technology planning, technology life cycle management, and technology assessment and evaluation), long-term support for technology applications and health initiatives is unsustainable. These deficiencies with respect to the management of technology point to the crucial need to align technology strategy and e-business strategy. In other words, the strengthening, linking, and aligning of technology planning and e-business planning in the e-health care context is the essential purpose of e-HCTM. In light of this development, the discussion of this chapter will focus on drawing lessons for e-HCTM from previous research on health care technology management in traditional health care organizations—specifically, large teaching hospitals (Eisler, Sheps, Satuglu, and Tan, 2002).

Multidimensionality of the E-HCTM Concept

Transferring lessons we have gathered from technology management in other industry sectors—particularly the concept of strategic HCTM and the importance of innovation—to health care and e-health care is the beginning step in exploring the concept of e-HCTM.

The complexity of the e-health care environment, the multitude of forces that shape technology decisions, and the uniqueness of the e-health care environment are all justifications for applying e-HCTM to overcome challenges of sustainability, cost, and quality of care. Compared with other industry sectors, such as banking and transportation, the e-health care environment is not only more complex but also more turbulent. The environment is challenging not only because of the complexities inherent in the development and maintenance of a seamless system spanning the continuum of e-health care delivery but also because of the complexity of relationships among stakeholders, including providers, vendors, payers, investors, insurers, patients, the general public (consumers), policymakers, regulators, researchers, and educators.

The Strategic Role of E-Technology

The health care industry is in transition, driven by such changing factors as economic trends, technology products and services, and population demographics. These pressures have resulted in changes in the structure and process of care, financing, and human resource management. The challenge in health care can be summarized as ensuring timely access to high-quality and cost-effective health care services. Health care systems in Canada, the United States, and other developed countries are expected to continue on the road of cost reduction and quality improvement through the growth and diffusion of e-health business models and services (see Chapter Twelve). Reforms in health care have been intended to increase efficiency, flexibility, and integration, as well as to improve health outcomes, community participation, and cost control. Given these sometimes conflicting pressures, a debate about the role of technology as the problem or as an important part of the solution is taking shape. Indeed, e-technology can play a vital strategic role in health care, as it does in other knowledge-based service industries, including banking and entertainment. This is particularly true for information and communication technologies and e-technologies, which can contribute significantly to improved management, cost-effectiveness, customer service, and support. These applications can create opportunities for new e-health services or for new delivery methods for existing services. For these reasons, some governments (for example, the government of British Columbia) have maintained information and communication technologies and e-health applications on their list of priorities even during a period of severe cost reduction (British Columbia Ministry of Health Planning, 2002). Only after a thorough economic evaluation will questions about comparative costs and benefits of various e-technologies, including the status quo, be answerable.

The emergence of e-technology as a lever of economic competitive advantage has created a demand for personnel who can help enterprises take advantage of such technological innovation (Raghupathi and Tan, 2002). In the past, many industries have seen technologies such as computer and telecommunication networks as playing a supportive role, contributing to overhead costs. In other words, these technologies are not seen as central to corporate objectives. Today, e-technologies are beginning to be recognized as significant core enabling assets with major strategic implications for an organization's survival and success. In addition, the power of converging e-technologies is blurring the boundaries between administrative and core technology tools. Many CEOs now believe that such enabling technologies, if managed appropriately, can contribute significantly to the achievement of e-business strategy and new organizational objectives. At the same time, these e-technologies may fundamentally change the way an organization functions as well as the way it relates to its industry sector, sponsors, suppliers, and, most important, its customers or clients. For example, the availability of e-health care through the Internet and related Web services is transforming mainstream health care.

From a marketing perspective, creative and rapid technological evolution generates a volatile technology push on the input side of organizations. Many companies, including giant retailers like Sears and CVS Pharmacy are going on-line to prevent their chain stores from losing customers to a growing list of on-line competitors. On the output side, customers expect reliable, consistent, safe, effective, and efficient service. The convenience of on-line shopping means that they can change their loyalties easily and quickly. They are looking for seamless technology and applications. The challenge, then, is for executives, including health executives, to enable their organization to continually transform the turbulent technology input into a customer-focused and appropriate output in the face of increasingly difficult internal and external constraints (Tapscott, 1996).

“Where change used to occur periodically, it's a way of life now,” said Charles Webb Edwards, executive vice president of the Technology and Operations Group at Wells Fargo and executive vice president and chief technology officer at Norwest Corporation prior to its merger with Wells Fargo. “There is real value in being able to manage … change.” Strategic planning horizons for most companies are shortening from ten or twenty years to five and, more recently, to three years. “The new approach to strategic planning recognizes that the New World is not predictable, linear, or deterministic. Rather, it is unpredictable, nonlinear, and full of surprises.” Rapid technological change is partly responsible for this nonlinearity (McCallum, 1996). Technology strategy is an integral strand in the strategic management fabric of an organization (Badawy, 1998; Husain and Sushil, 1997).

According to McGee and Thomas (1989), what has been missing “is a comprehensive view of how technological change can affect the rules of competition, and the ways in which technology can be the foundation of creating defensible strategies for firms.” Restructuring programs, takeover campaigns, and the unprecedented trend toward joint ventures are indications of the new way of doing business, “driven by the need to compete more aggressively and efficiently in world scale markets” (Perrino and Tipping, 1989). Studies have shown that levels of companies' investments in technology explain international differences in productivity and in shares of world markets.

Geisler and Heller (1996) argue that because of economic pressures, our health care system is in crisis. What's more, technology, especially medical technology, has played an increasing role in creating the crisis. They claim that proper and better management of medical technology provides some hope for dealing with the forthcoming challenges.

The World Health Organization defines health as a state of total physical, mental, and social well-being, not merely the absence of disease and infirmity. It is now recognized that population and individual health has many determinants not traditionally associated directly with the health care system—for example, air quality and socioeconomic status. Accordingly, e-HCTM includes managing applications of technology that influence the environment, information dissemination, health protection, and disease prevention. It goes beyond just applications of medical technology found in modern acute care systems or for direct medical care. In this context, the term e-health care technology applies, in the broadest sense, to more than just e-health information. It includes hardware and firmware devices, software and business processes, health products such as drugs and home care health products marketed on-line, as well as e-prescription and e-home care services. E-technologies that may contribute to quality or sustainability of health care systems (see Chapters One and Two) could be associated with virtual communities (see Chapter Three), e-clinical care (see Chapter Four), e-public health systems (see Chapter Five), e-network infrastructure (see Chapter Six), various e-health domains and applications (see Chapters Seven through Eleven) or other e-health business processes (see Chapter Twelve).

E-Health Care Technology Management Strategy

In this chapter, we review the literature across several industry sectors and combine the results of that review with the findings of our research on technology management in traditional health care organizations. This approach yields general agreement about the basis for e-HCTM strategy; the characteristics of an e-HCTM-focused business; and the responsibilities and capabilities of the e-technology officer, who is equivalent to the chief technology officer (CTO) in traditional corporate settings.

E-HCTM strategy is based on the competitive and turbulent e-health environment, the nature of the e-health business, and the state of e-technology development. Other factors in e-HCTM strategy include considerations of business-specific factors, environmental factors, and customer preferences; creation of strategic advantage and differentiation; development of e-technological expertise, e-business decision-making and problem-solving skills, and human resource capabilities; and readiness for a comprehensive rethinking and readjustment of job descriptions, information systems, governance structure, incentives, and decision-making processes. One of the most important issues is e-health business structure and its value propositions, as has been noted throughout this text. E-health policies hold together a decentralized, virtual workplace with rapid access to global information. In addition to flexible governance structures, management of e-health systems must emphasize seamless information flow, appropriate incentives (for example, for focusing on customers), and innovative performance assessment schemes.

E-HCTM strategy needs to be characterized by managerial vision, foresight, and entrepreneurial spirit. Strong leadership is one of the most critical aspects of success. This entails commitment to knowledge acquisition rather than just product development. Management personnel must know what they want, given the difficult-to-quantify costs and benefits of newer e-technologies and the need for flexibility. Managers must set realistic goals, match the supply of products and services to market demands, and be clearly aware of resources, constraints, and risks. Decisions and attitudes of management must be based on an analysis of competitive position, market intelligence, technical preferences of e-consumers (customers), and internal capabilities. The e-HCTM strategy focuses on the customer, replacing organization-centered approaches with an emphasis on market pull rather than e-technology push.

Management systems must focus on an internally integrated enterprise. These systems must coordinate across functional boundaries; in other words, cross-functional approaches must facilitate convergence of the historically divergent views of technically oriented and market-oriented individuals. Full and meaningful worker and customer participation in the production and delivery process is key to e-health success. Moreover, process management has to replace product management; this shift in focus to flexibility, adaptability, responsiveness, and effectiveness rather than efficiency and costs is necessary mainly because competitive advantage comes from achieving greater customer satisfaction and enterprise knowledge integration by deploying the appropriate e-HCTM strategy, not just from labor cost savings. Above all, the ability of the management team to change, adapt, and avail itself of new opportunities is critical in an environment as turbulent as the e-health marketplace.

E-consumers', e-providers' and the government's expectations have increased because of advances in technological capabilities. For the e-technology officer to move e-HCTM strategy forward, he or she must demonstrate thinking and visionary leadership, the ability to create new ways of funding, and a commitment to the alignment of e-technology with clinical objectives. Thus, the e-technology officer must bridge gaps between virtual team members and engage in continual planning, active resource allocation, development of standards, rapid reorganization when necessary, and adoption and implementation of fundamental changes in the e-business system. He or she must also be a steward of networked leadership, be close to the front line, and build an invisible enabling infrastructure.

The e-technology officer must ensure that promises made on behalf of e-technology applications are kept. He or she must build a viable, productive, and flexible e-technology asset base that can deliver goods and services on time and with a competitive pricing scheme. Moreover, he or she needs to take responsibility for managing technology-driven change and act as a change champion. Overall, this individual must be able to manage in an environment of decentralized decision making with a high level of interfunctional coordination; be conversant in e-business issues and challenges; have a focused commitment, empowered with applicable technical information; and have the skills to effect and manage change. Such an individual will also need relevant technical competence and an understanding of the importance of e-technologies and systems that provide a competitive edge, as well as the need for e-technologies and systems that support the goals of the virtual enterprise.

E-Surveying Health Executives

In this research, we electronically surveyed (e-surveyed) executive team members of the forty largest Canadian teaching hospitals on the topic of HCTM. Our target sample was representative of senior administrators across the Canadian health care system. Selected individuals from this group responded to a newly developed instrument. Factor analysis and cluster analysis were applied to the resulting data set. The research methodology involved the development of a new measurement structure or abstraction ladder (Geisler, 2000) that addresses criticisms of past technology management (TM) research.

Defining Health Care Technology Management

A rigorous content analysis was conducted to develop an acceptable definition of technology management and to identify related critical TM capabilities and attributes. A database of 255 articles was searched and 47 related dissertations were read for the sampling unit “defin*” (for “definition”) in the abstract or full text, to locate discussions of the definition of technology management. Scanning and integrating the information found in related articles and dissertations and eliminating those that were not relevant to our purpose, we generated the following comprehensive definition of the technology management construct:

From this definition, a schematic diagram for the technology management framework emerges, as shown in Figure 13.1. The final step in the framework, organizational success, can be defined differently in each organization. Business success in health care could, for example, be expressed as improved patient care within given resources. Alternatively, the performance goal may be, for example, the protection of the five health care principles of the 1984 Canada Health Act: comprehensiveness, universality, portability, accessibility, and public administration.

HCTM is a complex, multifaceted process with interlinked activities that clearly encompass more than a single indicator of successful performance. Guided by the notion of an abstraction ladder approach to metric development (Geisler, 2000), the goal was to create a hierarchy of major dimensions, including constructs, dimensions, and variables that describe these dimensions and their measurable indicators. The instrument was constructed from the exhaustive list of individual attributes that resulted from the literature content analysis. The attributes were grouped and regrouped to form indicators in such a way until an intuitively sensible and comprehensible list of indicators emerged. The list of indicators was far more detailed than similar attempts reported in the literature, although it soon became apparent that the indicators could be grouped into variables comparable to those referred to in the literature.

Our study did not develop a new definition for the technology construct. Rather, the perspective of technology as an extension of human and organizational ability was adopted from the literature. But based on an extensive literature content analysis, we have proposed a more complete definition of technology management to use in studying the concept in a health care setting as well as in generating a model of health care technology management. The proposed definition acknowledges the multidisciplinary and integrative nature of the TM construct. It captures the need to manage technology throughout its life cycle, from conception to replacement. Inherent in the definition of TM given earlier is the broadest view of technology. The two most critical aspects of the proposed definition of TM are (1) that technology both shapes and supports organizational strategy and (2) that its objective is to contribute to organizational success, however success is defined for a particular organization.

FIGURE 13.1. TECHNOLOGY MANAGEMENT FRAMEWORK

As we have discussed, the literature content analysis yielded a list of critical attributes for HCTM practice. These attributes, which are summarized in Table 13.1, formed the basis of a metric and thus of a testable HCTM model.

The model consists of three dimensions, which are subdivided into variables and indicators. The collective package constitutes the proposed HCTM model that is to be empirically validated. The individual building blocks are not new to engineering, business, or social science. A body of literature, research, and empirical evidence informs each dimension, variable, and indicator.

Pilot Test and Field Test, Using a Delphi Approach

Using a modified Delphi approach (consultation of experts), a pilot group of senior health care administrators and management consultants were asked to rank and validate the attributes of the HCTM model from the perspective of large-scale health care systems—that is, Canadian teaching hospitals. The focus was on responsibilities at the corporate or executive level. In order to qualify as an expert, an individual had to have been involved in health care management at senior levels for more than ten years. Positions held by these experts during their careers included deputy and assistant deputy minister; CEO, vice president, and chief information officer of a health organization; and director and executive director of a health industry organization. These experts were shown a draft survey and asked to critique the instrument with respect to clarity, comprehension, and ability to complete it. The fifteen pilot participants were also asked to recommend unambiguous descriptors for the state of implementation of each statement category that appeared in the survey.

TABLE 13.1. CRITICAL ATTRIBUTES FOR HEALTH CARE TECHNOLOGY MANAGEMENT FROM CONTENT ANALYSIS

It was envisaged that the outcomes of this e-survey would lead to a new research agenda for HCTM and e-HCTM. It was also hoped that the outcomes would influence curriculum development in health care technology management training in Canada and other countries. Therefore, the Canadian College for Health Service Executives, based on its own survey on desirable attributes of future health service leaders, published this statement: “Health care leaders carefully and successfully adapt some features of the private sector model for use within the public sector infrastructure. Technology conscious leaders are increasingly able to guide its infusion to their best advantage” (Canadian College for Health Service Executives, 2000). The statement was used to motivate respondents to fill out the questionnaire. Pilot participants were informed about the national character and the expected target audience of the study. They were presented with the prototype questionnaire, which included forty-two indicators, and asked to do the following:

• Complete the survey, keeping any large health care organizations with which they were familiar in mind
• Identify whether and how specific statements should be altered
• Add any missing statements that they thought were important
• Express concerns they might have about the appropriateness of the indicators being grouped into the identified domains and the categories within each domain
• Identify examples of additional real and practical evidence or indicators that could be used to support statements about the extent of implementation in the various categories (for example, strategic plans, job descriptions, budgets)
• Comment on the comprehensiveness and comprehensibility of the instrument

In the follow-up field test, senior managers of four regional referral organizations, representing a variety of administrative responsibilities, participated; specifically, the CEOs of four medium-sized health service organizations in British Columbia provided e-mail addresses of their executive teams (vice presidents, CEOs) and individuals reporting to the executive team members (senior managers). These sixty-four executives and senior managers represented eight different areas of responsibility With thirty-three responses from sixty-four individuals on the e-mail list, overall response rate was 52 percent and ranged between 43 percent and 64 percent for the four participating institutions. As in the pilot test, respondents were asked to comment on content, wording, and structure of the instrument; however, no changes to the instrument were suggested. The importance of the topic of technology management was confirmed by a consistently high average rating from each organization. On the six-point Likert scale in which 1 meant “of least importance” and 6 meant “of greatest importance,” average ratings from each organization ranged from 5.6 to 5.9.

Feedback from the pilot study as well as from the follow-up field test was used to further refine the questionnaire. Following the pilot test and the field test, the survey instrument was reshaped and reformatted before it was used in the national survey.

Validity and Reliability Issues and the Gap Score

The literature supports our approach to the evaluation of validity and reliability concerns about our research instrument (Lewis, 1993). Content validity was optimized through the iterative process by which the instrument was developed. Content analysis informed the instrument, and expert opinions from the pilot test and field test stages refined the instrument.

The following steps were used to establish different types of validity with respect to the instrument:

• Face validity and content validity of the instrument development process was supported by existing literature (see Table 13.1).
• Refinements were made based on initial expert suggestions from academic researchers and on comments and suggestions received from experts in the field in the pilot test and field test to achieve significant content validity.
• The field test stage further tackled content as well as structure, clarity, and comprehensibility of the survey instrument.
• Construct validity was addressed through a factor analysis of all responses.
• Concurrent validity was eventually addressed through a comparison of our research findings with a performance review of the same institutions that was completely independent of our research; this review, conducted by the Hay Group Study, uses a completely different metric.

The internal consistency coefficient, Cronbach's alpha, was computed with SPSS statistical software and applied to each of the three major dimensions (factors) resulting from the factor analysis of the field test data. An alpha of .8 is desirable. Reliability of the instrument was assessed by calculating this coefficient for each of the indicators according to the factors established by the factor analysis. For all indicators, Cronbach's alpha was found to range from .79 to .94.

In addition to generating Likert ratings for each indicator, a single measure was established to capture each respondent's perception of the gap between the ideal extent of implementation (as reflected in a rating of 6 on the Likert scale) and the perceived extent of implementation (which could be represented by a score from 1 to 6 on the Likert scale) for twenty-six HCTM indicators. Gap scores were weighted by using the respondent's assessment of the importance of the respective indicator. This resulted in a gap score, a composite variable given by the following formulation:

GS = (6 − E) × B, where

GS = gap between ideal and perceived implementation

6 = ideal implementation rating

E = rating of perceived extent of implementation

B = rating of perceived benefit or importance of implementation

A gap score (GS) was derived for each respondent. The term (6 − E) can be interpreted as an indication of room for improvement, because an E score of 6 would indicate full implementation. The factor B is a weighting factor based on the perceived importance of the item. For example, perceived full implementation of an indicator would yield a gap score of 0—that is, GS = (6 − 6) × B = 0, whereas deeply perceived need for an indicator to be implemented in the responding organization would yield a very high gap score—for instance, GS = (6 − 1) × 6 = 30. Here, an implementation rating of 1 (meaning “not at all”) would yield the highest gap score if the indicator's importance was also rated as 6 (meaning “very great”). Gap scores, therefore, range from 0 to 30.

An average gap score (the average of responses from a particular organization) of, say, less than 8 for an indicator can then be interpreted as “This indicator has been implemented to a high level and would not require additional attention for improvement.” This could be either because the indicator has been well implemented (rating of 5 or 6) or because the importance was rated lower than other indicators (4 or less). Field test results indicated that the instrument and the gap score meaningfully and consistently capture differences in perception among senior managers from different organizations with regard to known HCTM factors.

Altogether, responses from the organizations that participated in the field test can be grouped into major clusters. Our analysis of the field results uncovered three primary dimensions, or clusters. Cluster A, the low gap cluster, represented responses from executives in organizations achieving average gap scores of between 6 and 10. Cluster B, the moderate gap cluster, comprised organizations with an average gap score of more than 10 and less than 15, while Cluster C, the high gap cluster, consisted of organizations receiving average gap scores of 15–19.

One could interpret this finding as an indication of poorer-performing clusters (Clusters B and C) versus better-performing ones (Cluster A) in the eyes of their senior managers. Moreover, some respondents whose responses generally fell into Clusters A or C appear to have a more consistent view of their organization's performance than those whose answers generally fell into Cluster B. Finally, Cluster A organizations seem to have reached a high level of HCTM performance. Consistent with this finding, the organizations found in Cluster A are among those nationally recognized for their quality management systems. Our next step was to verify results of this field test in a wider national survey.

Web-Based Survey Design

The survey instrument was designed on the basis of the list of attributes and administered via a Web-based delivery system. This strategy was adopted because our target audience comprised very busy executives. The questions posed in the survey were ordered in a logical sequence that was based on the overall model, which was laid out for participants in a cover letter. Given that the target audiences for the survey were knowledgeable senior executives, it was deemed unnecessary and probably unhelpful to scramble the sequence of the statements; this was verified by our pilot-testing of the instrument. As in the Lewis study on the information resource management construct (Lewis, 1993), respondents used a Likert scale to rate their perception of the importance of various HCTM factors and the extent of the implementation of these factors in their organization.

Obviously, a survey can only be employed if the intended audience can be assumed to have access to appropriate technology. While our intended participants were believed to have Web access, other major reasons for using e-surveying in our case included the following:

• Senior health administrators are too busy to take the time to answer mail-in questionnaires.
• The speed and convenience of e-surveys would encourage a high response rate and result in faster return of the surveys.
• E-surveys ease data handling and permit convenient data analysis.

Developing and posting the e-survey on the Web involved a number of critical steps. The most time-consuming aspect of the process was determining the correct e-mail addresses of hundreds of potential participants. Repeated exchanges of notes and information to and from representatives of the executive offices of the participating organizations took place. The survey content, including all associated notes and rating scales, was developed as a Microsoft Word document; Web pages with forms for accepting data input were generated with Dreamweaver 4.0 Web design software. Rather than using purchased Web survey software, the Web survey tool was custom-designed in order to achieve a more professional look. A thank-you page was automatically generated and sent to the respondent on receipt of a survey response. The survey was sent electronically to hundreds of senior managers whose e-mail addresses had been identified. Data files of submitted responses could easily be created and exported directly into an Excel database and spreadsheet program.

Anonymity of respondents and their organizations was ensured. Even so, the process allowed reminders and requests for responses to individuals from whom none had been received within the specified time period. Distribution of the e-survey, receipt of responses, and creation of the data set for statistical analysis were all accomplished within a six-week period; in contrast, gathering and correcting compiled e-mail addresses took eight weeks.

The National Survey

In February 2001, the Executive Committee of the Association of Canadian Academic Health Organisations (ACAHO) enthusiastically endorsed the research reported here. The Executive Committee members, the CEOs of some of Canada's largest teaching hopitals, shared the view that HCTM approaches in the Canadian health care system could be and needed to be strengthened. Their support and active promotion of this project presented an invaluable opportunity to attract participation from senior managers across Canada's teaching hopitals. The list of ACAHO member organizations was compared with the list of organizations ranked by number of technology-intensive beds from the Canadian Health Association guide. This list indicated that ACAHO member organizations included most of the major and most technology-intensive health care organizations. It was felt that the senior managers of these organizations were best positioned to assess HCTM practices in Canadian health care agencies.

Following the decision of the ACAHO's Executive Committee, which included the CEOs of six of Canada's largest health care organizations and ACAHO's executive director, to sponsor this national survey, letters were sent to the CEOs of member organizations to ask for their organization's participation. With the aid of ACAHO, requests were issued to the CEOs for the e-mail addresses of their executive teams and their senior management. The collection, validation, and verification of the e-mail addresses were a considerable undertaking that had to be completed prior administration of the e-survey.

Factor analysis was used to explore the data for patterns and to reduce the number of variables to a more manageable size. Survey responses were clustered by individual organizations and further analyzed using analysis of variance (ANOVA) to identify systemic differences in HCTM practices. However, this analysis was inconclusive, suggesting the possibility that senior managers with direct patient care responsibilities were responding differently from those who did not have direct patient care responsibilities because of differences in the pattern of observed responses. Executive team members (vice presidents, CEOs) were responding differently from senior managers reporting to VPs. Cluster analysis was used to investigate the underlying data groups and patterns more thoroughly. In this sense, it was used to optimize the comparison and identification of differing HCTM practices and capabilities.

Research Findings with Relevance for E-Health Care Technology Management

Thirty-three ACAHO member organizations were invited to participate in this national e-survey project. In the end, approximately 850 individuals in twenty-eight organizations received the survey electronically. Of these, 324 responded, representing an average response rate of approximately 38 percent per participating institution (range of 24 percent to 58 percent).

Confidentiality of the respondents and the organizations were protected through the assignment of an organization code between 1 and 33, which will be used for the remainder of this discussion. For example, organization 1 submitted thirty-six acceptable responses, reflecting a 33 percent response rate from the executives and senior managers who received the survey. Of these thirty-six respondents, six executive members had patient care responsibilities (for example, the vice president of nursing and vice president of nursing), while seven did not (for example, the vice president of finance and vice president of support services). Of the responding managers reporting to vice presidents, eight had patient care responsibilities and fifteen did not. The respondents were first asked to rate the importance of the topic of the survey (1 = not at all, 2 = very little, 3 = little, 4 = some, 5 = great, and 6 = very great). The overall result was that 317 of 324, or 98 percent, rated the topic's importance as great or very great. No respondent rated the importance at less than 4.

In addition, when respondents were asked to rate the importance of the individual indicators, every indicator scored an average rating of 5 or greater on the six-point rating scale. This implies that, on average, the 324 respondents agreed on the great importance of each indicator. The results validated both the instrument and the applicability of our HCTM model to the health care setting.

Factor Analysis

Factor analysis was performed to identify the degree of separation and aggregation of underlying variables, or latent variables, on which the twenty-six measured indicators loaded (for specific variables, see headings under each dimension such as “Technology Strategy,” “Chief Technology Officer,” and so on). The resulting model is presented in Table 13.2, which shows the apparent loading of the twenty-six indicators onto three primary dimensions, or factors. Thus, it was no longer necessary to treat the indicators separately for further analysis; these three dimensions, or factor loadings, define the primary units for analysis. For the purpose of our discussion and to ease comparison, the numbering of the revised indicators has been retained in Table 13.2.

TABLE 13.2. REVISED MODEL OF INDICATORS

Developing a technology strategy that is aligned with business strategy to meet the highest possible order of customer needs represents dimension 1 of the model. The more sophisticated the organization's ability to become aware of relevant new technologies, the more timely its ability to adapt or adopt emerging technologies will be. If those emerging technologies cannot be supported or sustained because of environmental factors, their usefulness will be limited. Thus, dimension 1 also includes technology and environmental scanning and further suggests that the development, implementation, and administration of the technology strategy needs to be driven and managed from the executive level.

The model further implies that in addition to the development of a technology strategy, the organization has to be nimble in order to make necessary changes in business direction or adapt to new ways of meeting customer needs. Dimension 2 therefore focuses on change management. Key elements of the change management dimension are a customer service attitude that permeates the organization and a change management process driven by the executive team. Celebrating technology as part of the solution, keeping an eye on the future of the organization, and celebrating integrated multifunctional teams are other components of this dimension of the model. A human resource strategy that fosters innovation and creativity in line with strategic technologies is considered another critical aspect of this dimension.

Dimension 3 of the model represents the organizational and operational management component of HCTM. Once strategic direction has been established and the organization has been prepared for change, the success of the implementation and operation depends on organizational and operational management. This continuous and iterative process should encompass every stage of the technology life cycle and should be well supported by ongoing evaluation and assessment of performance and risks at the technological level and the organizational level.

These new dimensions were used to test for significant differences in responses between and among various categories of respondents. This was necessary to establish whether all responses from individual organizations could be treated as coming from the same population for the purpose of response comparisons within and between organizations. The resulting p-values (p = .25 for patient care versus non–patient care managers; p = .5 for executive versus non-executive managers) indicated no significant differences in gap scores between the categories of respondents within organizations. Thus, all responses from the same organization could be grouped for the purpose of testing for significant differences between organizations.

One key question to be answered in our research was the ability of the instrument to differentiate between organizations in regard to their HCTM approach. The twenty organizations that supplied five or more responses were included in this part of the analysis. Based on ANOVA and the resulting p-values (p < 0.001), significant differences were found in the way in which senior managers across the twenty organizations perceived the state of HCTM in their organization.

Cluster Analysis

A cluster analysis approach was used to determine if the organizations grouped around similar mean gap scores for each of the three dimensions. Cluster analysis of the twenty organizations with more than five responses yielded an apparent three-cluster taxonomy:

• Cluster 1: Organizations 1, 2, 3, 5, 6, 7, 9, 11, 13, 14, 17, 18, 19
• Cluster 2: Organizations 4, 8, 10, 12, 16
• Cluster 3: Organizations 15, 20

Testing for significant differences between and among these clusters indicated a statistically significant difference (p < 0.001) between the mean gap scores of clusters of health care organizations along each of the three dimensions identified. Since Cluster 3 consisted of only two institutions, with comparatively few responses, further analysis focused primarily on the differences between Cluster 1 and Cluster 2 organizations. The average gap scores for every indicator were lower for Cluster 2 than for Cluster 1 organizations. In addition, the significant differences between the two clusters of organizations, based on the perceptions of their senior managers, are influenced more by some variables than others.

The importance of implementing each indicator was not rated as highly as the importance of the indicator itself. This was consistent for every indicator. The mean scores for implementation range from 3.15 to 4.46, demonstrating greater variability than those for indicators with a standard deviation range of 1.01 to 1.62. The mean gap scores ranged from 8.4 to 14.66, with a standard deviation range of 5.45 to 8.27. This considerable variability raised questions about the factors contributing to the different perceptions. Statistical analysis showed that the key factors contributing to the variability were significantly different responses from senior managers representing different organizations. In fact, the participating organizations fell into two main groups of perceived HCTM performance—that is, Cluster 1 and Cluster 2 organizations.

Table 13.3 lists the twenty-six variables in the survey in the order of the percentage difference between Cluster 1 and Cluster 2 mean gap scores. Percentage differences range from 6.54 to 56.42 percent. On a national basis, the two indicators (indicator 3 and 4) with the highest percentage difference were also the ones with the highest variability.

Based on the analysis, the major differences between the two clusters from the perspective of senior managers could be identified. Cluster 2 managers rated their organizations very highly (low gap score) in the following three areas:

• Indicator 3: Executive member formulates technology strategy.
• Indicator 4: Executive member administers and manages technology strategy.
• Indicator 12: Senior management promotes organizational vision and direction.

TABLE 13.3. GAP SCORE DIFFERENCES BETWEEN CLUSTER 1 AND CLUSTER 2

In contrast, Cluster 1 managers rated their organizations particularly poorly (high gap scores) in the following areas:

• Indicator 8: Technology is linked to customer needs.
• Indicator 19: Organization has strategies for responding to change.
• Indicator 26: Performance due to technology management is evaluated.

The distribution of indicators in the four quartiles of the gap score range is shown in Table 13.4.

At one end of the spectrum, Cluster 1 managers rated none of the indicators in the lowest quartile of gap scores, compared with Cluster 2 managers, who placed four indicators there, the top two of which were indicators 3 and 4. On the other end, seven indicators were rated in the highest quartile for Cluster 1 as opposed to none for Cluster 2. These seven Cluster 1 indicators were led by indicators 8, 19, and 26, which also were deemed to need the most attention according to the overall national averages.

TABLE 13.4. INDICATOR DISTRIBUTION

The largest differences in reported perception based on implementation ratings and gap scores between Cluster 1 and Cluster 2 managers related to indicators 3 and 4 of our HCTM model.

These are the two indicators that focus on the function of a chief technology officer (CTO). This result strongly confirmed the message from the literature about the necessity of executive attention and leadership for HCTM. As a member of the executive team, the CTO typically reports directly to the CEO. Indicators 3 and 4 imply that this individual is responsible for staying abreast of present and potential future technologies and for leading the strategic decision-making process about when to adopt, support, or abandon technologies. Along with providing leadership, coordination, and facilitation, the responsibilities of the CTO include such activities as gatekeeping, advocacy, funding, sponsorship, policy and procedure development, promotion, capacity building, and overseeing the technology management system.

Not all hospitals have a CTO, in which case the responsibilities discussed here are either taken up by the CEO or, more likely, delegated to someone else such as the chief financial officer or chief operations officer. The CTO's job is to understand the strategic business issues, the customers, and the technology. The CTO should be an effective leader and command the respect of his or her employees, managers, and peers. The CTO should demonstrate vital communication skills. This finding implies that a good CTO probably can translate technical issues so that they can be readily understood by nontechnical personnel.

Cluster 1 hospitals were shown to be weakest in the indicators that also presented the highest need for improvement on a national, systemwide basis—namely, indicators 8, 19, and 26.

A high gap score on indicator 8 implies a lack of ongoing access to information about current and future needs and priorities of customers and staff. Indicator 19, similarly, addresses how quickly the organization responds to shifts in technological trends. A high gap score here implies that the organization ignores “market pull” strategies as part of its customer relations policy, essentially going along with “technology push.” Product development times, innovation cycles, and overall cycle times for putting ideas and innovations into practice are longer than expected relative to industry norms. Finally, a high gap score for indicator 26 implies a lack of systematic performance reviews for different aspects of HCTM. These might include their impact on overall organizational goals, objectives, and customer service, including, for example, annual performance evaluations of the executive fulfilling the CTO role.

From our analysis, the instrument (the e-survey) is able to distinguish between senior managers' differences in perceptions of their organization's standard of HCTM. Significant differences (p < 0.001) between Cluster 1 and Cluster 2 organizations can be consistently identified across all three dimensions of HCTM. In particular, the perceived presence of chief technology officer roles in the organization seems to contribute strongly to the differences in mean gap scores between the clusters. Other strategic management and change management variables are perceived as being implemented to a greater extent in Cluster 2 institutions. Improvements seem to be possible along all three dimensions.

Overall, our results and statistical analysis indicated that there are significant differences in HCTM sophistication among Canadian teaching hospitals. The major differences occur in areas of strategic technology management (dimension 1), followed by change management (dimension 2), and, to a lesser extent, organizational management (dimension 3). The perceptions of senior managers are not significantly influenced by their area of responsibility relative to patient or non–patient care or by their position in the reporting structure. Improvements are generally needed in all areas addressed by the indicators identified in our HCTM model.

The Hay Group Study

As pointed out earlier, the purpose of the entire HCTM exercise is to improve performance in terms of achieving the strategic goals and objectives of the organization. For a private sector organization, this may be expressed as competitive advantage, market share, profitability, return on investment, or other such measures. What would be the performance measures for public sector agencies such as the hospitals that participated in this study? What difference does it make that one cluster of hospitals seems to manage technology better than another cluster?

Fortunately, answers to these questions have been provided independently by ACAHO information about operational efficiency and clinical efficiency that is generated on a confidential basis. The Hay Group in Toronto compiled the data in an annual report entitled Benchmarking Comparison of Canadian Hospitals. The Hay study was commissioned by the hospitals in an effort “to improve the efficiency, effectiveness and quality of their care processes.” The comparisons were based on Canadian hospital separation data, on accounts and statistics reported to Ministries of Health, and other data provided by the participating hospitals.

We made use of the Hay study to validate the results of our own study. To determine whether there was any association between HCTM performance of the two clusters of hospitals identified in our study and the measures of performance used in the Hay study, we examined the outcomes provided by the Hay study on the following two summary measures:

• The measure for the overall clinical efficiency of a hospital is the percentage of inpatient days that could be reduced if a hospital were to achieve benchmark levels of performance. The smaller the percentage, the more efficient the hospital.
• The measure for the overall operational efficiency of a hospital is the potential reduction in operating cost, including direct care, administrative, and support functions. Subject to some caveats, the hospital with the smaller potential reduction can be considered more efficient with respect to the areas examined by these comparisons.

If an average gap score of 6 (over all responses and all indicators for each organization) was considered the benchmark target, the potential HCTM improvement could be calculated and expressed in terms of potential percentage reduction. We can use this measure to compare our results to those of the Hay study.

Four of the five Cluster 2 hospitals and eleven of the thirteen Cluster 1 hospitals participate in annual benchmark comparison studies. Confidential summary statements provide participating organizations with feedback about the room for improvement if they operated at benchmark levels on specific performance indicators. From the summary statements, it was clear that Cluster 2 hospitals had considerably less room for improvement, suggesting a strong correlation between their high HCTM performance and their clinical and operational efficiency.

Conclusion

Our research generated a technology management conceptualization consisting of a definition, its attributes, and a metric. Development of a validated and reliable instrument amounted subsequently to the formulation of a first-order HCTM model. This model represents both a theoretical perspective that can be further tested empirically and a framework for applying HCTM practice in the real world.

A key question in our research relates to the impact of HCTM sophistication on health care organizational performance and success. Our study was able to explore to some extent the relationship between clinical and operational efficiency and technology management performance, which was discussed in terms of a comparison of our study results with those of the Hay Group study. Unfortunately, our study did not address the more strategic impact of HCTM performance on customer satisfaction and customer service levels relative to customer needs; this is a topic for future HCTM research.

Our purpose in this concluding section is to conceptualize how the findings from our HCTM research challenge us to view the importance of the customer (consumer, patient, or client) in health care more critically. Specifically, the need to understand in detail the various consumer groups and their needs, priorities, and values can be applied meaningfully in an e-health context. We will come back to this topic a bit later in the section, but first we will consider some of the implications of our findings.

We believe that further HCTM and e-HCTM research is clearly needed to test the generalizability of our three-factorial model. Our analysis suggests that this three-dimensional HCTM model can also be used to serve not only as the basis for creating a HCTM training curriculum, but as practical guide for the establishment of a HCTM strategy in a health care organization. It may also be the basis for an e-HCTM curriculum, as well as a practical tool for generating an e-HCTM strategy by providing benchmarks for future research.

In terms of e-HCTM curriculum, we have seen how the e-health paradigm components in this text can be divided into three major themes. First, the e-health foundation (Chapters Three through Six), which corresponds to dimension 3 of the HCTM model (organization management) because of the concentration on how organizational perspectives and infrastructures are to be reshaped through e-health thinking and the management of virtual organizations. Second, e-health domains and applications (Chapters Seven through Eleven) correspond to dimension 2 of the HCTM model (change management) because of the focus on how different sectors of mainstream applications of health services must now be changed and managed into new domains and alternative forms of e-health services. Finally, e-health strategies and impacts (Chapters Twelve through Fifteen) correspond most clearly to dimension 1 (strategic management) because of the need to focus this paradigm shift at a strategic management and policy level, rather than at operational or lower levels in order to achieve organizational success and high system performance.

In terms of generating an e-HCTM strategy, our study recognizes the increasing power of technology (in our case, e-technology) to meet the needs of e-consumers and e-providers in revolutionary and unpredictable ways. In this sense, our focus is on harnessing the power of e-technology through e-health care technology management. Like other potentially strategic resources such as health, human resources, and financial resources, e-technologies should and can become more commonly understood assets that must be managed well if high payoffs are to be expected. Executive-level attention is critical in facilitating the alignment of an organization's overall business strategy and its e-technology strategy in order to achieve high-performing e-health systems. The empirical evidence from our study is clear on this matter: to be successful in the current turbulent and unpredictable technology environment, organizations need to anchor technology management at the vice presidential level. Both traditional health care and e-health care systems have efficiency goals and consider innovation and technology to be tactical tools to accomplish existing services at lower cost or to produce more services at a given cost. The judgment of what constitutes strategic technology needs to be made with full knowledge and information about the customers.

Strategic technology improves the manner in which customer needs are met. It is aimed at the highest order of customer needs achievable under prevailing circumstances. One key research area emerging from our study is the definition and identification of customers, customer groups, and customer needs in health care or in the case of e-health systems, e-consumers, e-consumer groups, and e-consumer needs. Organizations like teaching hospitals are faced with competing customer interests (provincial services, community services, research and education services), not to mention internal “customer” interests, including those of doctors, nurses, and technicians as well as patients. Resource planning of any kind—human, technological, or financial—is difficult unless the primary customers and their needs are well defined.

With the emergence of e-health, a whole new scenario of e-customer and e-consumer relationships is unfolding, with significant implications for providers and agencies of health care services. In e-health, e-consumers and e-patients must take center stage, and their needs must be considered very carefully. Definition of e-consumers, e-stakeholders, and their hierarchy of needs is essential. Everyone in the e-health system must be clear on whose needs are to be met by a particular technology and be prepared to adjust if better ways to meet those needs are discovered. Groups of e-customers or individual customers may prioritize their needs differently. Most expressed needs are simply a means to another end, intermediate needs that can sometimes be leapfrogged with e-technology. Without a thorough understanding and explicit statement of this hierarchy of customer needs, no effective e-technology strategy can be developed.

While most organizations attempt to tackle various aspects of the e-technology management model, the lack of an integrated framework limits organizations and individuals, preventing them from realizing the full potential and benefit of those attempts. Without an e-technology management focus and framework, there is no basis for relating technology management ideas; thus, the response to those ideas is fragmented. An example might be the often-lamented difficulty of bringing research results to bear on practice environments through a linkage and exchange process (see, for example, Robson, 1993). Using a fragmented approach may lead to only limited success in changing environmental conditions; for example, individuals may be trained in new technologies but then may return to environments that do not have a suitable support framework in place. A cohesive framework would optimize the benefit of individual actions. Applying the HCTM model to e-HCTM would at the very least provide a foundational perspective from which ideas about e-HCTM can be generated and shared.

E-health care systems should not overlook the importance of e-technology management responsibility. There seems to be some opposition to the idea of a central executive coordinator of technology strategy and implementation, primarily due to the breadth of e-technology in health care. It is important to remember that inherent in the HCTM model is the concept of multidisciplinary and multifunctional teams with representation from both research and operations departments. In a similar vein, e-technology management training and research needs to be strengthened from a multidisciplinary and multifunctional perspective. E-technology has made the critical leap from being a tactical resource to also being a strategic resource. While a broad view of potentially strategic e-technology is important, an e-HCTM framework focuses efforts on strengthening the managerial aspects of an organization for e-technology use and application.

Finally, a number of issues relate to the application of the HCTM model and its implications for e-HCTM. The HCTM model consists of a hierarchy of dimensions, variables, and indicators. As mentioned earlier, there is a body of literature dealing with the basic and applied science fundamentals underlying each indicator. While knowledge could undoubtedly be expanded in each of these facets, some examples of priority research areas for HCTM and for e-HCTM are listed here:

• How are the customers in e-health care different from those in traditional health care? What are their differing needs, and what order of priority may be attached to those needs (that is, what is their hierarchy of needs)? What are the implications for HCTM versus e-HCTM?
• Aside from health administrators and executive team leaders, who would be key targets for an e-survey on HCTM and e-HCTM? Are the views of these groups potentially the same or different in traditional and e-health care systems?
• What concretely differentiates clusters of high-performing and low-performing health care organizations in terms of HCTM? How would this information apply to e-HCTM, a system with apparently quite different e-technology management perspectives and performance capabilities?
• How significantly do HCTM and e-HCTM contribute to clinical and operational efficiencies, quality and levels of care, and customer (or e-consumer) satisfaction?

Organizations should be guided by market pull as opposed to technology push considerations. The public's unlimited demand for customized, high-quality, rapidly delivered, and low-cost health care is well documented. E-providers and e-customers today are demanding ready access to global e-health information to achieve the highest level of health care. This, and the rapid rate of e-technological development, require both public and private sector organizations to manage e-technology and innovation strategically, albeit for different combinations of economic and altruistic motives. The perspective of e-technology as a strategic resource implies willingness to change the business fundamentally if the needs of customers can be better met with different technology.

Chapter Questions

1. How is e-HCTM strategy defined on the basis of the extant literature? What emerging dimensions define HCTM and can in turn be applied to e-HCTM?
2. What is the significance of e-surveying health administrators and executive team members on a topic such as health care technology management?
3. What characteristics do you expect to find in well-managed e-health care systems? Why? Who are the best people to manage e-technologies?
4. What are some of the areas for future research in e-HCTM?
5. Imagine that you have been asked to oversee a new e-health initiative on moving a large pharmacy chain store on-line. What steps would you take with respect to e-technology applications and management of these applications in the longer run to ensure sustainability?

References

Andersen, D. F., Belardo, S., & Dawes, S. S. (1994). Strategic information management: Conceptual frameworks for the public sector. Public Productivity and Management Review, 17(4), 335–353.

Badawy, M. K. (1998). Technology management education: Alternative models. California Management Review, 40(4), 94–116.

British Columbia Ministry of Health Planning. (2002). A new era for patient centred health care: Building a sustainable, accountable structure for delivery of high-quality patient services.: Author.

Canadian College of Health Service Executives. (2000). Survey of leadership skill requirements.: Author.

Eisler, G. (2002). Health care technology management (HCTM): An assessment of its application in Canadian teaching hospitals. Unpublished doctoral dissertation, University of British Columbia, Canada.

Eisler, G., Sheps, S., Satuglu, S., & Tan, J. (2002). Health technology management in Canadian teaching hospitals: An empirical investigation. Paper presented at the Hospital of the Future Conference, Chicago, Illinois.

Geisler, E. (2000). The metrics of science and technology. Westport, CT: Quorum Books.

Geisler, E., & Heller, O. (1996). Managing technology in healthcare. Boston: Kluwer.

Gregory, M. J., Probert, D. R., & Cowell, D. R. (1996). Auditing technology management processes. International Journal of Technology Management, 12(3), 306–319.

Husain, Z., & Zafar Husain, S. (1997). Strategic management of technology—A glimpse of literature. International Journal of Technology Management, 14(5), 539–578.

Lewis, B. R. (1993). The information resource management concept: Domain, measurement, and implementation status (Technology planning). Doctoral dissertation, Auburn University, Auburn, AL. (Available from UMI Dissertation Services)

McCallum, J. S. (1996). Changing at warp speed: Managing technology. Business Quarterly, 60(31), 87–89, 92–93.

McGee, J., & Thomas, H. (1989). Technology and strategic management progress and future directions. R & D Management, 19(3), 205–213.

Perrino, A. C., & Tipping, J. W. (1989). Global management of technology. Research Technology Management, 32(3), 12.

Raghupathi, W., & Tan, J. (2002, December). Strategic IT applications in health care. Communications of the ACM, 45(12), 56–61.

Robson, C. (1993). Real world research: A resource for social scientists and practitioner-researchers. Cambridge, MA: Blackwell.

Tapscott, D. (1996). The digital economy: Promise and peril in the age of networked intelligence. New York: McGraw-Hill.

Tapscott, D. and Caston, A. (1993). Paradigm shift: The new promise of information technology. New York: McGraw-Hill Inc.

World Health Organization. (2000). The World Health Report 2000: Health systems: Improving performance. Geneva, Switzerland: World Health Organization.

Evidence-Based Medicine Case

Barry P. Markovitz

A second-year medical student hears a psychiatry professor lecture about Freud's theories. He asks the professor what evidence there is to support these theories and receives this answer: “I don't think there is any. I certainly don't believe this stuff; I was asked by the department chair to give this lecture.” On rotation in internal medicine, a medical student is required to present evidence to support her treatment plans for every new patient she has admitted. She finds this exciting and challenging compared with the usual routine of following the attending physician's preferences with-out supporting evidence. A student is asked to read an article to help determine the proper course of treatment for a particular patient. The attending physician provides this disclaimer: “No one really does this in real life.”

These three experiences are noted as defining moments in the careers of three authors (Brian Haynes, Sharon Straus, and Scott Richardson, respectively) of a very popular handbook called Evidence-Based Medicine: How to Practice and Teach EBM (Sackett and others, 2000). First formally introduced in a 1992 article in the Journal of the American Medical Association, (Evidence-Based Medicine Working Group, 1992), evidence-based medicine has become known as “the means by which current best evidence from research can be judiciously and conscientiously applied in the prevention, detection, and care of health disorders” (Haynes and Haines, 1998). The term research in this context refers almost exclusively to clinical research. So was the concept of using scientifically derived information to guide medical care just invented in 1992? Of course not, but even today, health care is far too heavily influenced by the individual practitioner's beliefs, experiences, and preferences. Indeed, the correlation between the strength of evidence to support a practice and the frequency of its use remains poor (Rich, 2002).

The explicit process known as evidence-based medicine or, more generically, evidence-based health care, truly represents a paradigm shift, as defined by Thomas Kuhn (1970). Prior to the explosion of research findings available in the past few decades, physicians for generations had little more than their mentors and personal experience to rely on in deciding what was best for their patients. With the advent of the controlled clinical trial, therapeutic interventions could be rigorously tested for efficacy. The principles of clinical epidemiology were developed and applied to interpret patient-oriented research. The old paradigm was based on the assumption that clinical experience and understanding of pathophysiology were sufficient to diagnose and treat most patients. That paradigm has faded away. The new paradigm still calls for clinical experience and mechanistic understanding but also calls for a knowledge of how to use the rules of evidence to properly interpret the medical literature for direct patient care. Although publications on clinical research trials have increased dramatically, many studies have substantial flaws that may limit their application. A critical eye is necessary to evaluate such reports.

Why evidence-based medicine, and why now? With exponentially increasing numbers of outcome-oriented clinical trials, systematic reviews, and evidence-based guidelines being published, clinicians need a consistent approach to search for, appraise, and apply this evidence to the care of their patients. Classic continuing medical education programs do not necessarily improve physicians' skills; pathophysiological treatment often fails; and traditional textbooks are typically out of date with regard to recent innovations by the time they are published.

Evidence-based medicine (EBM) comprises five stages:

• Developing the clinical question relevant to the patient at hand, typically formatted by the PICO acronym: In a given patient population, does a particular intervention, compared with controls or standard therapy, result in an improved outcome?
• Searching for the evidence, usually in bibliographic databases such as the National Library of Medicine's MEDLINE
• Critically appraising the evidence for its validity, results, and applicability
• Integration of the appraised evidence with the patient's values and preferences
• Self-evaluation of the process to continually improve its efficiency and effectiveness

It becomes obvious early in this process that many relevant clinical trials may address one's patient's problems. Systematic reviews can be structured and concise summaries of two or more original clinical studies designed to address a focused question; when quantitative analysis is performed, the term meta-analysis applies. The methodology and results of a systematic review can be critically appraised in a manner analogous to the appraisal process of a primary clinical investigation. When systematic reviews are analyzed on the basis of individuals or organizations, evidence-based clinical practice guidelines may be published. As one might expect, the guidelines, like the studies, can and should be viewed through the prism of critical analysis.

Each clinical situation requires a unique set of questions to filter the studies and interpret the results. The Users' Guides to the Medical Literature are a series that has appeared in the Journal of the American Medical Association since 1992, offering structured methods of critical appraisal of original trials on therapy, prognosis, diagnosis, and harm, as well as guides to interpretation of systematic reviews, clinical practice guidelines, and many other peer-reviewed publications on health care. For example, for trials of a therapeutic intervention, in order to appraise the paper's validity and root out various sources of bias, one might ask the following questions (Guyatt, Sackett, and Cook, 1993):

• Was the assignment of patients to treatment groups randomized?
• Were all patients who entered the trial properly accounted for and attributed at its conclusion?
• Were patients, health workers, and study personnel blind to treatments?
• Were the groups similar at the start of the trial?
• Aside from the experimental intervention, were the groups treated equally?

In assessing a therapy article's results, we ask the following (Guyatt, Sackett, and Cook, 1994):

• How large was the treatment effect?
• How precise was the estimate of the treatment effect?

The terminology of clinical epidemiology can put results in terms that providers and patients can grapple with more appropriately than with a “statistically significant difference” measured by a “p-value.” For dichotomous outcomes, the concepts of relative risk, relative risk reduction, absolute risk reduction, and number needed to treat are the relevant markers of effect. The importance of each of these terms is demonstrated in Exhibit 13.1, in which two hypothetical treatments have identical relative effects but very different absolute mortality rates.

Finally, we ask about the applicability of the study's results (Guyatt, Sackett, and Cook, 1994):

• Can the results be applied to my patient's care?
• Were all clinically important outcomes considered?
• Are the likely treatment benefits worth the potential harm and costs?

After following this process, one should have a reasonably sound notion of whether the study in question is valid, what the results are, and whether they are truly applicable to one's patient.

The Users' Guides series has been updated and condensed into a textbook, finally compiling into a single tome the “liturgy” of EBM. While the basic methodology of EBM may be suitable for a textbook (although this itself is debatable, a point that the authors readily concede), the evidence needed to practice EBM cannot be so easily packaged, and that is where the Internet plays a vital role.

The Internet as a Source for Evidence

The original 1992 article that introduced EBM describes a clinical scenario wherein a curious resident conducts a computerized literature search in the library, at a cost of one hour of time and $2.68, including the photocopying charge (Evidence-Based Medicine Working Group, 1992). Ten years later, the same search would likely be conducted in the hospital ward or clinic, quite possibly at the patient's bedside, at no charge. The article chosen might be printed out on the nursing station's laser printer or simply read on-line or saved in an Adobe portable document format (PDF) file for later printing or e-mail distribution. A version of the article could be available for downloading directly into the resident's portable digital assistant (PDA), never using paper. And the entire process of searching and retrieval might take only a few minutes at most, rather than an hour.

EXHIBIT 13.1. THE IMPORTANCE OF DICHOTOMOUS MEASURES OF EFFECT

Indeed, it is astounding how far and fast the biomedical information retrieval technology of the Internet has spread in less than a decade. Virtually every bibliographic database in the biomedical sciences is available on-line, and those managed by the U.S. National Center for Biotechnology Information (NCBI) and National Library of Medicine (NLM) are accessible free of charge. The premier such database, MEDLINE, now available at PubMed (http://www.pubmed.gov/), contains over 11 million citations from 4,600 journals from seventy countries. The “Clinical Queries” function provides direct evidence from clinical trials and is particularly useful for clinicians. It incorporates a series of built-in filters that are specific to the type of query. There is a separate button for articles about therapy, prognosis, diagnosis, and etiology, which allows rapid focusing of searches on only the most relevant citations (Haynes and others, 1994). In addition to citations and abstracts, direct links to the full text of the on-line paper are often available, although either an institutional or an individual subscription to the journal in question is usually necessary to actually view the full text. Over 3,500 journals enable this “link out” feature from PubMed.

A separate but related issue, the concept that for-profit private companies (publishers) should control the record of scientific accomplishments, is being called into question. Numerous initiatives are trying to break down the centuries-old concept that scientists, who are usually publicly funded, should turn over copyright of their written analysis to publishers, who then require the authors and their peers to buy back this written record. Although this may be a reasonable model for commercial writers, who seek profit from sales of their work as the publishers do, with the advent of the Inter-net and the minimal cost of distribution of electronic “papers,” scientific authors often seek as wide a distribution as possible of their work with as few barriers as possible. In some cases, the financial firewalls are beginning to crumble. Scientists are joining boycotts of publishers that do not enable free access to their manuscripts on the Internet, at least after a limited period of time. The Public Library of Science (http://www.publiclibraryofscience.org), one organization calling for open access, has received signatures of support from over 30,000 scientists from 180 countries. Some journals now provide access to their archives three to twelve months after initial publication. Other journals have been free on the Web for several years, and new journals with altered business models (author pays instead of reader pays) are being developed. This is a critical issue for the future of EBM and the Internet, for without access to the full text of clinical research on-line, there is insufficient evidence with which to practice evidence-based medicine.

Other companies provide access to MEDLINE with proprietary interfaces, the most popular of which is OVID (http://www.ovid.com). In addition to point-and-click access to sophisticated searching functions, OVID has contracted with numerous publishers to provide their journals' full text from within the application, which is now most commonly accessed over the Internet with a browser. With many institutions paying for OVID access, clinicians and researchers can easily access thousands of publications' full texts either at no cost to the user or for a nominal yearly fee. OVID also provides full access to a number of other resources that are valuable for the evidence-based clinician. MedScape (http://www.medscape.com) also provides a search interface to MEDLINE and has arrangements with a number of journals to display selected full-text articles. Silverplatter (http://www.silverplatter.com), another proprietary service affiliated with OVID, offers access to MEDLINE and other databases.

Not all medical journals are cited in MEDLINE. The Institute of Scientific Information maintains several large scientific bibliographic databases, including the Current Contents series, which is now available by institutional subscription as the Web of Science (http://isiknowledge.com/wos/). CINAHL is a nursing and allied health bibliographic database, and CANCERLIT provides references related to cancer studies. EMBASE, maintained by the publisher Elsevier Science, includes some journals not referenced in MEDLINE. In addition, it provides subsets for more directed searching, such as EMBASE Pediatrics and EMBASE Anesthesiology.

Either with the aid of built-in filters—for example, at PubMed or in OVID—or by learning to manually add screening terms to searches, the evidence-hungry clinician can rapidly focus on relevant clinical trial reports, then continue the process of critical appraisal and application. Tools for learning EBM techniques are widespread on the Internet and will be discussed in the next section. But often there are numerous studies that are relevant to the patient question at hand. In such cases, it is far more appropriate and efficient to look for and appraise systematic reviews or evidence-based guidelines, should they exist. Is there a rapid way to access these secondary evidence sources on-line? In PubMed, a search can be limited to systematic reviews and to meta-analyses in OVID. The Cochrane Collaboration is an international group dedicated to creating, disseminating, and maintaining systematic reviews of virtually every aspect of health care. The Cochrane Library of Systematic Reviews is on-line (http://www.cochranelibrary.com/) and available by individual or institutional subscription. These reviews are rigorously prepared with structured software and are peer-reviewed and regularly updated. The United Kingdom's National Health Service Centre for Reviews and Dissemination at the University of York maintains the Database of Abstracts of Reviews of Effects (DARE) on-line (http://nhscrd.york.ac.uk/darehp.htm). Since DARE represents a regularly updated source of summaries of systematic reviews obtained from a variety of sources, it should be a clinician's first stop in a search for such resources.

Using systematic reviews, original clinical research, and consensus opinion under rigorous and structured rules, many organizations (for example, the American Academy of Pediatrics and the Society of Critical Care Medicine) have developed guidelines that summarize the current best evidence in the diagnosis and management of a variety of disorders. Unlike previous generations of consensus statements, which were often simply the opinion of experts, evidence-based guidelines explicitly present the evidence behind the recommendations and grade the recommendations accordingly (Oxford Centre for Evidence-Based Medicine, 2002). A unique source for such guidelines is the National Guideline Clearinghouse (NGC) (http://www.guideline.gov/), sponsored by the U.S. Agency for Healthcare Research and Quality. Only clinical practice guidelines from professional societies or organizations that have a documented evidence-based approach are included on the NGC site, which makes available nearly a thousand guidelines in either full-text or summary form.

When confronted with a clinical problem, it is usually not immediately clear whether original research, systematic reviews, guidelines, or other focused sources will be most useful in the specific case. What is needed is a metasearch engine that scours numerous other search sites and presents a coherent list of results. SUMSearch at the University of Texas Health Science Center is just such a resource (http://sumsearch.uthscsa.edu/). SUMSearch performs contingency searching—that is, it alters its search based on the number of results obtained, broadening or narrowing the focus as necessary—of PubMed, DARE, NGC, and other quality sites. It provides pull-down menus for filtering, based on the type of clinical question (therapy, prognosis, and so on), age group, and language. Search results are presented in an extremely user-friendly format, based on evidence type and source.

There are additional secondary sources of evidence. Structured critical appraisals and other succinct summaries of original clinical research studies (and of systematic reviews and guidelines) are available as independent journals and on a number of established Web sites. The American College of Physicians and American Society of Internal Medicine (ACP-ASIM) publishes the ACP Journal Club (http://www.acpjc.org), and the journal Evidence-Based Medicine is from the BMJ Publishing Group (http://ebm.bmjjournals.com/). A long-standing resource, Patient Oriented Evidence that Matters (POEMs) (http://www.infopoems.com/), offers such succinct and structured summaries and is now available by subscription. This resource is integrated with a search engine and a version for PocketPC PDAs. The University of Michigan Department of Pediatrics maintains a collection of Critically Appraised Topics (CATs) (http://www.med.umich.edu/pediatrics/ebm/), concise evidence-based summaries of research that addresses a particular patient issue. The PedsCCM Evidence-Based Journal Club posts peer-reviewed, structured critical appraisals of clinical trials and systematic reviews pertaining to intensive care medicine (http://PedsCCM.wustl.edu/EBJournal_club.html).

The Turning Research into Practice (TRIP) database (http://www.tripdatabase.com/) is a comprehensive resource for evidence sources on the Internet, cataloguing sources from seventy-five sites, including the abstracting and appraisal sites. Search results are grouped according to source—for example, peer-reviewed journals, guidelines, journal clubs. In addition, the TRIP site includes links to relevant chapters in Internet-based textbooks and digital image sources, recognizing that some users also require the type of background information that a textbook can still provide effectively.

For the busy clinician, tracking down and appraising the necessary information still sounds like a lot of work! If there were resources that synthesized validated evidence resources into a user-friendly interface, more physicians might be amenable to practicing evidence-based medicine. Clinical Evidence (http://www.clinicalevidence/com/) is a quarterly print and Web publication of the BMJ Publishing Group. Organized by clinical topics (for example, stroke management, myocardial infarction prevention), Clinical Evidence discusses focused questions and presents very succinct conclusions. Evidence appraisals are hyperlinked from the conclusions, enabling rapid drill-downs to the primary source material. Exhibit 13.2 shows a sample screen that illustrates the utilitarian user interface of this valuable resource.

Cliniguide, available via OVID, presents disease management information in a stripped-down textbook interface, also taking full advantage of hyperlinking to make more detailed material with source evidence appraisals readily available. EBM Solutions (http://www.ebmsolutions.com/) is a care management application that is available to physicians and patients on the Web; it was developed by six academic medical centers. The site offers succinct evidence-based guidelines on various aspects of management of numerous common conditions. Unique to EBM Solutions, symptom constellations (for example, abdominal pain in children and chronic cough) are also listed, improving the usefulness of this resource.

Although evidence-based medicine decries the pejorative label of “cookbook medicine,” in the face of overwhelming evidence from nearly every health care field that consistency of approach is a critical factor in explaining improved outcomes, decision support tools are being developed to steer practitioners to the “proper” diagnostic or management strategy for a patient with a given symptom complex or disease. Interactive calculators on the Web (and downloadable versions for PDAs) can determine everything from free water deficits to a patient's risk of breast cancer. The National Center for Emergency Medicine Informatics (http://www.ncemi.org/) provides a wide range of such calculators and acuity scores. Some could readily be labeled clinical decision support tools. For example, the simple Ottawa rule for ankle fracture, with the answers to three simple questions, can reliably predict the need for a radiograph of the ankle (Stiell and others, 1993). The more complex Goldman acute myocardial infarction probability calculator can also be used as a decision support tool (Goldman and others, 1988). Unique to pediatrics is Isabel, an on-line interactive tool that aids in differential diagnosis and also chronicles unique cases and treatment algorithms (http://www.isabel.org.uk/). A remarkable on-line tool, Evidence-Based On-Call (http://www.eboncall.co.uk/), offers nearly comprehensive evidence for the diagnosis and management of a limited number of common conditions, free of charge.

Resources for Learning Evidence-Based Medicine on the Internet

As many sites exist to learn about or provide tools for EBM as exist to provide evidence with which to practice EBM. With the hyperlinked Internet, it is hard not to find (or stumble on) this vast network of EBM resources. Certainly, a reasonable starting point for someone who wishes to learn about EBM would be one of the “mother ships” for EBM, either McMaster University's Health Information Research Unit in Canada (http://hiru.mcmaster.ca/) or the Oxford Centre for Evidence-Based Medicine in the United Kingdom (http://cebm.jr2.ox.ac.uk/). These two sites offer both internal educational resources and links to the vast array of resources elsewhere on the net. For learning how to practice Evidence Based Medicine, the Clinical Epidemiology for Effective Clinical Practice Online Tutorial at IntensiveCare.com is also an excellent starting point (http://intensivecare.com/Tutorial.html). The Introduction to Evidence-Based Medline tool, a joint effort of Duke University Medical Center Library and the Health Sciences Library at the University of North Carolina–Chapel Hill is another self-paced tutorial (http://www.hsl.unc.edu/lm/ebm/index.htm).

EXHIBIT 13.2. CHAPTER INTRODUCTION TO CLINICAL EVIDENCE

Many consider the Users' Guides to the Medical Literature series (mentioned earlier) to be the defining scripture of EBM, and the full text of the entire series is now freely accessible on-line at the Canadian Centres for Health Evidence (http://www.cche.net/usersguides/main.asp). A version of the guides with more interactive features is also available on-line by individual or institutional subscription (http://www.usersguides.org). The Centre for Evidence-Based Medicine at Mount Sinai Hospital in Toronto, Canada, is another resource-rich site (http://www.cebm.utoronto.ca/). This is the home of the popular EBM handbook noted earlier (Sackett and others, 2000); it offers supplementary materials and updates to the text. Also available at this site is a free EBM calculator for PalmOS and PocketPC PDAs, which enables rapid calculations of risks, odds ratios, and more. The site also maintains an excellent annotated set of links to on-line EBM resources around the world.

The basic science of evidence-based medicine is clinical epidemiology, and the Web has vast resources in this field; many are listed on the Epidemiology and Biostatistics Resources page (http://www.ped.med.utah.edu/genpedscrr/Epibio.htm) at the University of Utah. Another well-organized metasite with a list of EBM links is the University of Hertfordshire's Evidence Based Medicine Resource List (http://www.herts.ac.uk/lis/subjects/health/ebm.htm). It is unlikely that an important EBM Web site exists that is not listed on Andrew Booth's Netting the Evidence page (http://www.nettingtheevidence.org.uk/). A particularly useful feature of this site—which is organized into subsections for library, searching, appraising, implementing, software, journals, databases, and organizations—is the comprehensive library (bibliography), with direct links to the full text of many articles in various journals pertaining to EBM, systematic reviews, clinical trial design, and more.

The Future of EBM and the Internet

It is clear that evidence-based medicine and the Internet are thoroughly intertwined to the extent that it is virtually impossible to practice effective and up-to-date EBM without the use of the Internet. Because new generations of health care providers have grown up with computers and the Internet, teaching them how to use the Web to practice EBM is superfluous. However, they do need training on how to practice based on the best available evidence and where to go to find that evidence. Proper application of the tools of clinical epidemiology also requires a set of skills that must be learned. Above all, young trainees must be shown how to integrate EBM into their clinical routines; in many programs, it remains a separate academic exercise. EBM could never have evolved to its current state without the Internet, but the ongoing development of the on-line world does not guarantee the success of EBM. That will still require thoughtful, caring, and rational health care providers.

Case Questions

1. What is the significance of EBM for e-health care?
2. Why is the Internet an effective means or infrastructure for promoting EBM? What other platforms or infrastructures could be used?
3. Why is access to full-text clinical research critical to practicing EBM? How does one ensure the authoritative nature of the source from which evidence is to be gathered?
4. How can patients assist their caregivers in finding an appropriate evidence-based treatment for their symptoms? Imagine that you are asked to design a Web site to provide EBM support to patients. What features, design elements, or interface elements are needed to ensure that the site can provide this support in a convenient way?
5. How can EBM be integrated into clinical routines? How can the training of new residents help the integration of EBM? What is the impact of EBM on the future of e-health practices?

References

Evidence-Based Medicine Working Group. (1992). A new approach to teaching the practice of medicine: Evidence-Based Medicine Working Group. Journal of the American Medical Association, 268(17), 2420–2425.

Goldman, L., Cook, E., Brand, D., Lee, T., Rouan, G., Weisberg, M., Acampora, D., Stasiulewicz, C., Walshon, J., Terranova, G., et al. (1988). A computer protocol to predict myocardial infarction in emergency department patients with chest pain. New England Journal of Medicine, 318(13), 797–803.

Guyatt, G., Sackett, D., & Cook, D. (1993). Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. Journal of the American Medical Association, 270(21), 2598–2601.

Guyatt, G., Sackett, D., & Cook, D. (1994). Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. Journal of the American Medical Association, 271(1), 59–63.

Haynes, B., & Haines, A. (1998). Barriers and bridges to evidence based clinical practice. British Medical Journal, 317(7153), 273–276.

Haynes, R., Wilczynski, N., McKibbon, K., Walker, C., & Sinclair, J. (1994). Developing optimal search strategies for detecting clinically sound studies in MEDLINE. title to come, 1(6), 447–458.

Kuhn, T. The structure of scientific revolutions. Chicago: University of Chicago Press, 1970.

Oxford Centre for Evidence-Based Medicine. (2002). Levels of evidence and grades of recommendation. Retrieved on December 29, 2002, from http://www.indigojazz.co.uk/cebm/levels_of_evidence.asp

Rich, M. (2002). From clinical trials to clinical practice: Bridging the gap. Journal of the American Medical Association, 287(10), 1321–1323.

Sackett, D., Straus, S., Richardson, W., Rosenberg, W., & Haynes, R. (2000). Evidence-based medicine: How to practice and teach EBM. Edinburgh, Scotland: Churchill Livingstone.

Stiell, I., Greenberg, G., McKnight, R., and others. (1993). Decision rules for the use of radiography in acute ankle injuries: Refinement and prospective validation. Journal of the American Medical Association, 269(9), 1127–1132.

Vannevar, B. (1945, July). As we may think. Atlantic Monthly, 101–108.

Three individuals need to be acknowledged for their invaluable support and cooperation. Jim Flett was one of Canada's most senior hospital administrators. As executive director of the Association of Canadian Academic Health Organisations (ACAHO) at the time of this study, he was instrumental in gaining the support of teaching hospital CEOs across the country for this project. Murray Martin, CEO of the Hamilton Health Sciences Corporation, orchestrated our first audience with the board of ACAHO, which led to their enthusiastic support. Shan Satoglu, program head for health technology management at British Columbia Institute of Technology, provided assistance and objective analysis during the literature content analysis, which was the critical phase of the project.