10.2.1 Obtaining video data

The videos used in this test were collected over a period of 44 days. We chose these parameters for the duration of our study because this 6 week or month and a half period constitutes a time frame that is a long enough period for extracting useful conclusions. Thus we began our study on the 28th of February 2013 and ended on April 12th of that year. For each day, 19 queries focusing on different aspects of diabetes were issued to the YouTube web site. Each day, the top-500 ranked videos from each query were examined in order to identify videos coming from credible sources.

Through our study of health videos on YouTube we identified credible YouTube channels, such as hospitals and health organizations, and organized them into white-lists⁸⁶ of channels. The Health Care Social Media List started by Ed Bennett⁸⁷ was used as an initial white-list for credible channels, which we expanded with more channels that were identified during our studies (Karlsen et al. 2013).

Since users also seek information from their peers, we also identified videos coming from users that we classified as active in publishing diabetes-related videos. The generous availability of peer-to-peer healthcare video material demonstrates the fact that not only do patients and their caregivers have knowledge and experiences that they want to share but that patients themselves eagerly seek such information from their peers (Fox 2011a). To recognize this information need and to, likewise, investigate the use of medical terms in user-provided videos, we identified, in a third white-list, channels of active users that predominantly produced diabetes videos.

Our white-lists contained a total of 699 channels, where 651 were hospitals, 30 were organizations, and 18 were active users. The 19 queries used in the study all included the term “diabetes” and focused on different aspects concerning the disease. We used queries such as “diabetes a1c,” “diabetes glucose,” “diabetes hyperglycemia” and “diabetes lada.”

To execute the first stage of the project, which was the information-collection stage, we implemented a system that for each day of the study automatically issued the 19 queries (with an English anonymous profile). For each query, the system extracted information about the top 500 YouTube results, and identified new videos from white-listed channels that were included in our set of test videos. After the 44-days of the study, we had a total of 1380 unique diabetes-related videos from hospitals, health organizations, and active users. The title and description of each video were extracted and compared against medical terms in order to detect videos where its metadata contained medical terminology.

10.2.2 Detecting medical terms in video title and/or description

The experiments were carried out using two different sets of medical terms: one from the ICD-10 database,⁸⁸ and a second set from the MeSH database.⁸⁹ The medical vocabulary used was the result of searching the online databases of ICD-10 and MeSH with the phrase “diabetes mellitus.” For ICD-10 we retrieved 167 medical terms related to diabetes, while MeSH returned 282 diabetes related terms. A term is either a single word or a phrase consisting of two or more words.

The set of terms were selected from the MeSH vocabulary (in the following denoted as MeSH terms set) and from the ICD-10 vocabulary (in the following denoted as ICD-10 terms set). We further distinguish between exact terms and partial terms. An exact term is a complete medical term as given in the ICD-10 or MeSH vocabulary. A partial term is a single word and a subset of an exact term (which may constitute a phrase). For example, “neuropathy” is a partial term of the exact term “diabetic autonomic neuropathy.”

To generate lists of partial terms from ICD-10 and MeSH vocabulary, we first removed stop words, such as “of” “the” “in” “with” “and” (Baeza-Yates and Ribeiro-Neto 2011). In addition, since diabetes is the general topic of the selected videos and vocabulary terms, we choose to discard the most common terms (“diabetes” “diabetic”) from our list of partial terms. This was done to focus the attention on more specific medical terms related to diabetes. We also did a manual revision of the partial terms, to discard terms that are not specifically related to the diabetes disease. This included terms such as (diet, coma, complications, obese, latent, chemical, onset), which were used in phrases such as “diabetic coma” and “diabetic diet.” After processing the medical phrases, we had two lists of partial medical terms, one for the ICD-10 terms set and the other for the MeSH terms set.

We used two analytic methods to identify medical terms in video title and description, where we focused on exact term match and partial term match to ICD-10 and MeSH terms set, respectively. Criteria for determining medical term prevalence in video title and description were as follows:

1. Exact match: The system detects a positive match when the video title/ description contains an exact term from a vocabulary terms set.
2. Partial match: The system detects a positive match when the video title/ description contains a partial term from a vocabulary terms set.

Fig. 10.1: Architecture for medical term detection.

Figure 10.1 shows how medical terms are detected in video title and description. Title and description of a video is, together with a vocabulary term set, given as input to a process that detects exact term matches and partial term matches for the video. If medical terms are detected, information about the video and the detected terms are stored. The system then continues with the next video, and searches for medical terms in the subsequent video.

We handle medical terms from ICD-10 and MeSH separately, meaning that the system only includes exact and partial terms from one of the vocabularies at any given time. Medical terms detection is thus executed twice for the videos: once for detecting ICD-10 terms; and subsequently, in the next execution of the system, for detecting MeSH terms. This was done in order to identify the impact different vocabularies had on medical terms detection.

10.2.3 Medical vocabularies

The International Classification of Diseases is maintained by WHO (World Health Organization) and is now in its tenth revision (ICD-10) since its first version in 1893. The purpose of ICD-10 is to provide a common foundation for definition of diseases and health conditions that allows the world to compare and share health information using a common language (WHO 2013). The classification is used to report and identify global health trends. The classification is used primarily by health workers. We have used the 10th Revision (ICD-10) to define exact and partial match with metadata description of the videos.

Medical Subject Headings (MeSH) is the National Library of Medicine’s controlled vocabulary thesaurus used for indexing articles for PubMed. MeSH is continuously updated with terms appearing in medical publications (NLM 2013). Terms like “Diabetes Mellitus” and even “Twitter Messaging” are defined in a hierarchical structure that permits searching at various specificity levels.

Together these vocabularies can be used to describe the relevant health condition covered by creators of some pieces of information while placing the piece(s) into a hierarchy of information that allows identification of the piece by healthcare information seekers.

10.3.1 ICD-10 results

When comparing ICD-10 terms to video title and description, we found that 54 of the 1380 videos (4%) had a title/description with an exact term match to at least one ICD-10 term. For partial ICD-10 terms, we detected a positive match for 816 videos (59%), while 510 videos (37%) did not have any ICD-10 terms in its title and/or description altogether (see Fig. 10.2a).

Fig. 10.2: Results of ICD-10 term matching, with total result over all videos (2a) and positive match distribution over hospital, health organization and active users videos (2b).

When analyzing the distribution of positive result videos from hospitals, health organizations and active users, we found that the majority (66%) of videos with medical terms came from health organizations (see Fig. 10.2b), while videos from hospitals and active users had an equally low prevalence of ICD-10 terms (17%).

Figure 10.3 displays the total number of videos within each group (i.e., hospitals, health organizations, and active users) that had a title/description with an exact match (3a) or partial match (3b) to ICD-10 terms. As shown in Fig. 10.3, the videos produced by organizations contained the highest number of both exact match terms and partial match terms (54% and 67%, respectively) when compared to hospital and user-generated videos. For hospitals however, which came in second place, we noticed a significantly higher occurrence of exact match terms (39%), (more than twice as many of the partial match terms (15%) found in such videos), a mirror opposite of organizations whose partial match terms exceeded their exact match terms.

Fig. 10.3: Exact and partial match distribution over hospital, health organization and active users’videos.

The relative proportion of videos from each group, with respect to exact and partial match, is displayed in Fig. 10.4. There we see that hospital videos had the largest relative proportion of exact match terms. Among the 270 hospital videos, 21 (7.8%) had an exact match with an ICD-10 term. The corresponding numbers for organizations and active users were 3.4% (29 of 854 videos) and 1.6%, (4 of 256 videos), respectively. The proportion of videos with a partial match to ICD-10 terms is much higher, with videos from health organizations coming out on top with 63.9%, followed successively by active users videos (56.3%) and hospital videos (46.7%).

During the analysis we detected a total number of 27 distinct ICD-10 terms. Figure 10.5 presents the most frequently used ICD-10 terms, including all terms that were used in more than five videos. Terms with an occurrence of five or less are represented as “other terms,” and include 15 terms with a total of 39 occurrences. Exact terms in Fig. 10.5 are “diabetic retinopathy,” “diabetic ketoacidosis” and “insulin-dependent.” The rest of the terms are partial ICD-10 terms. For a complete list of both exact and partial terms, see Tabs. 10.2 and 10.3, respectively.

In Fig. 10.5, we include the total number of ICD-10 term occurrences. This means that if a title/description includes the terms “type 1” and “glucose,” both terms are counted and represented in the numbers given in Fig. 10.5. We observe that four terms, “type 1,” “type 2,” “insulin” and “glucose,” have a much higher number of occurrences than the other terms. These four terms represent 81% of the total amount of ICD-10 term occurrences.

Fig. 10.4: Relative proportion of videos with medical terms.

Fig. 10.5: The most frequently used ICD-10 terms in video title/descriptions.

10.3.2 MeSH Results

When comparing video title/description to MeSH terms, we found that 92 videos (7%) had an exact term match while 726 videos (52%) had a partial term match. Finally, there were 562 videos (41%) that did not have any MeSH terms in its title/ description (see Fig. 10.6a).

We used the same analysis method as for ICD-10 terms, and present in Fig. 10.6b the positive match distribution over hospitals’, organizations’ and active users’ videos, while in Fig. 10.7 we distinguish between exact match distribution and partial match distribution. For both ICD-10 and MeSH terms, we observe that hospital videos have a higher proportion of exact term matches compared with the proportion of partial term matches found in hospital videos.

The relative proportion of videos with MeSH terms (see Fig. 10.8), follows the same pattern as ICD-10 term matches (see Fig. 10.4), where hospitals have the highest relative proportion of exact term matches and organization have the highest relative proportion of partial term matches. A difference we noticed when looking for terms found in the MeSH and ICD-10 vocabularies is that the proportion of exact terms are for all groups higher when using the MeSH thesaurus as opposed to the ICD-10 thesaurus. In contrast, the proportion of partial terms for all groups were lower in the MeSH vocabulary when compared to ICD-10 vocabulary. This is why it is so important when doing this kind of research to distinguish between these two medical vocabularies as well as the kind of term that is found, indicating whether it is an exact or partial term.

Fig. 10.6: Results of MeSH term matching, with total result over all videos and positive match distribution.

Fig. 10.7: MeSH exact and partial match distribution over hospital, health organization and active users videos.

We detected a total number of 45 distinct MeSH terms. Figure 10.9 presents the most frequently used MeSH terms, including all terms that are used in more than five videos. Terms with an occurrence of 5 or less are represented as “other terms,” and include 25 terms with a total of 66 occurrences. Exact match terms in Fig. 10.9 are “diabetic retinopathy,” “gestational diabetes,” “diabetic ketoacidosis” and “prediabetes.” The rest of the terms consist of partial MeSH terms. The complete list of both exact and partial MeSH terms used in the test videos are seen in Tabs. 10.2 and 10.3, respectively.

As was the case for ICD-10 terms, the four terms (type 1, type 2, insulin, and glucose) had a very high number of occurrences while the majority of terms had relatively few occurrences. Using MeSH, we see that four most common terms represent 74% of the total amount of occurrences, while 41 terms are found among the remaining 26% of occurrences.

Fig. 10.8: Relative proportion of videos with MeSH terms.

Fig. 10.9: The most frequently used MeSH terms in video title/descriptions.

From the above analysis, we see that the MeSH results follow much of the same pattern as ICD-10 term detection. A main difference between ICD-10 and MeSH, is that a higher number of distinct terms were detected when using the MeSH vocabulary. We also found a higher number of exact match occurrences using MeSH.

10.3.3 Terms used in video titles and descriptions

The number of medical terms detected in video title and descriptions are few compared to the total number of diabetes related terms available from the ICD-10 and MeSH vocabularies. Among the 167 ICD-10 terms, only four exact terms were detected (2.4%), while 12 (out of 282) exact MeSH terms were detected (4.3%). Concerning partial terms, 23 were detected based on ICD-10 and 33 based on MeSH terms.

Below we present two tables listing the terms recognized in the exact term match (Tab. 10.2) and partial term match analysis (Tab. 10.3). Among the exact vocabulary terms from ICD-10 and MeSH, only 13 distinct terms were detected in video title and/or descriptions, while 36 distinct partial terms were found. Tables 10.2 and 10.3 list these terms, indicating to which vocabulary they belong, and provides the number of videos using each of these terms.

In addition to the partial terms listed in Tab. 10.3, we also detected 10 terms that appeared in title and/or description of one video only. These terms were: acidosis, intolerance, nephropathy, glycosylation, fetal, pituitary, resistant, products, central, and sudden.

We observe that a higher number of terms were detected when using the MeSH vocabulary. This may come as a consequence of the higher number of terms in the MeSH term set. Also, since the original use of MeSH is for indexing articles for PubMed, it might be natural (if one were to use medical terms) to choose a MeSH term also for describing videos.

Tab. 10.2: All exact terms detected in video title/descriptions. Including vocabulary affiliation and number of videos where the terms were used.

Tab. 10.3: Partial terms detected in video title/descriptions. Including vocabulary affiliation and number of videos where the terms were used. Not including terms with only one occurrence.

From Tabs. 10.2 and 10.3 we see that the two vocabularies complement each other, since some of the detected terms are only found in ICD-10 while others only in MeSH. To check the overlap between the ICD-10 and MeSH term sets, we merged the two selected term sets. We found that by removing duplicates and plural forms of terms, the total amount of non repetitive terms were 344. Among these, the number of terms included in both ICD-10 and MeSH were 105. When we set aside the overlapped terms between these two thesauri, we see that both ICD-10 and MeSH vocabularies contribute terms that are not identified in the other vocabulary respectively.

10.3.4 Occurrences of terms – when discarding the most common terms

As previously pointed out, the four partial terms (type 1, type 2, insulin, and glucose) had a very large number of occurrences when compared to the rest of the partial terms. To see how these four terms affect the total result, we made a new calculation of MeSH term matches, but now without recognizing the terms (type 1, type 2, insulin, glucose). We compared the new result (Fig. 10.10b) to the original result (Fig. 10.10a), and found that the number of “no match” videos dramatically increase (from 41% to 81%) when disregarding the (type 1, type 2, insulin, glucose) terms.

Figure 10.11 compares the relative proportion of partial MeSH term matches when all partial terms are included with those cases in which the four terms (type 1, type 2, insulin, glucose) were disregarded. When excluding the four most common terms, the proportion of videos including partial medical terms drops dramatically for videos from health organizations and active users, with a decrease of 83% and 87%, respectively. For hospital videos the proportion of partial MeSH term matches drops, but not to the same extent as for the two other groups. Here the decrease is 48%.

When disregarding the four most common terms, the distribution follows the same pattern as for exact term matches, where hospitals had the largest relative proportion of videos including medical terms. This indicates that hospitals more often, compared to health organizations and active users, describe videos using the more specific medical terms.

Fig. 10.10: The number of videos that have partial match for MeSH terms in title and/or description. In 10a all terms are included, while in 10b the four most common terms; (type 1, type 2, insulin, and glucose) were disregarded.

Fig. 10.11: Relative proportion of videos with MeSH partial terms. Analysis including all partial terms compared to analysis excluding the four most common terms (type 1, type 2, insulin, glucose).

10.4.1 Findings

This study presents an analysis of medical terminology used in title and description of YouTube diabetes health videos uploaded by hospitals, health organizations and active users. The results show a low prevalence of medical terminology and that few distinct terms are actually used for describing videos.

We found that only 4% of the video title/descriptions included an exact ICD-10 term while 7% included an exact MeSH term. A larger amount of the total videos had a partial match with ICD-10 and/or MeSH terms. However, many of the positive results match one or more of the terms (type 1, type 2, insulin, glucose). When disregarding these very common terms, the amount of videos that include a partial MeSH terms was only 12%. The same applies to the findings of ICD-10 terms, where the amount of videos including a partial ICD-10 term drops to only 19% when disregarding the four common terms.

When distinguishing between videos provided by hospitals, organizations, and active users, we found that relative to the number of videos from each group, hospital videos had the highest prevalence of medical terms, followed by health organizations’ videos and lastly by active users’ videos. This was the case for exact term match to both ICD-10 and MeSH, and also for partial term match when disregarding the most common terms (type 1, type 2, insulin, glucose).

Even though hospital videos had the highest relative prevalence of medical terms (both exact and partial) the numbers were still low. Among hospital videos, only 11.1% included an exact MeSH term and 7.8% an exact ICD-10 term. The relative prevalence of partial MeSH terms (except the four common terms listed above) found in hospital videos was 22.2%.

In addition to low prevalence of medical terms found in video title and description, we also found a very low variety of diabetes-related medical terms in the title/description of such videos. Among the 167 ICD-10 terms, only four exact terms were detected (2.4%), while 12 (out of 282) exact MeSH terms were detected (4.3%). Concerning partial terms, 23 such terms were detected based on ICD-10 and 33 such terms were detected based on MeSH.

10.4.2 How ICD-10 and MeSH terms can be useful

Some hospitals and health institutions have for a long time provided patients with information about their disease and/or the medical procedure performed to help their ailment, especially on their discharge from treatment. This information normally contains very specific medical information that the patient may not only use to show to health personnel, explaining why they were at a hospital, but also to help them learn more about their health condition and how to prevent serious relapses.

In primary care, for example, there is an increasing trend towards providing patients with information about their condition. The medical terms present in the documentation given by primary care providers, especially the diagnostic or procedure codes, provide potential links to specific information that is relevant to the patient. For medical practitioners on the other hand such the diagnostic codes and MeSH terms can be used to identify literature that would be directly relevant for the patient through MEDLINE.⁹⁰ The patient should also have the same opportunity to find relevant health information, including the wealth of medical information contained in patient health-related videos.

Providing both codes and the corresponding medical terms in video title and/ or description, will help the patient to find information targeted at their medical condition. Codes and corresponding medical terms will also reduce the uncertainty that a patient will experience confronted with an often unknown and very complex medical reality. Being certain that you have identified information that is relevant for the patients’ medical condition can therefore be of great value for the patient.

Using medical terms and codes in the title and description of videos will also help producers of information resources, like YouTube videos, to identify gaps in information resources that patients may need. As the situation is now, neither the patient nor the producers of the videos are able to explore the collection in the same systematic manner as the medical professional explores the medical domain using MEDLINE.

10.4.3 Discriminating power of terms

The specificity of a term is a measure of its ability to distinguish between documents in a collection (Spärck Jones 1972; Salton & McGill 1983) or in our case, between health videos. It is common to measure a term’s specificity using its Inverse Document Frequency (IDF) (Spärck Jones 1972; Baeza-Yates & Ribeiro-Neto 2011). The IDF measure is based on counting the number of documents, in the collection being searched, that contain the term in question. The intuition is that a term that occurs in many documents is not a good discriminator, and should be given less weight than one that occurs in few documents.

The results of our experiments show that the four partial terms (type 1, type 2, insulin, and glucose) have very low specificity among the diabetes related videos selected for this study. This reflects the common use of these terms. By describing a video using one or more of the most common terms, the video will be one among a large number of videos that are described in the same manner. To stand out, and be easy to identify and retrieve, a video needs a description that is (i) accurate with respect to video content and (ii) described through terms that have high specificity.

In this chapter we recommend providing textual descriptions to videos that also include medical terms and codes that as accurately as possible describe the content of the video. Following the consideration of specificity and discriminating power of terms, we also recommend avoiding the most common terms and rather describe video content through more specific medical terms.

10.4.4 The uniqueness of our study when compared to other work

A considerable amount of literature has been published on YouTube data analysis, such as studying the relations between video ratings and their comments (Yee et al. 2009) or focusing on the social networking aspect of YouTube and their social features (Cheng, Dale & Liu 2008; Chelaru, Orellana-Rodriguez & Sengor Altingovde 2012). Up till now, studies of YouTube performance have mainly focused on YouTube in general rather than on specific domains, such as health. However, some studies have evaluated YouTube health video content with respect to their quality of information for patient education and professional training (Gabarron et al. 2013; Topps, Helmer & Ellaway 2013). Such studies, focusing on different areas of medicine, include the work of Butler et al. (2012), Briones et al. (2012), Schreiber et al. (2013), Singh, Singh & Singh (2012), Steinberg et al. (2010), Murugiah et al. (2011), Fat et al. (2011), and Azer et al. (2013). In these studies reviewers evaluate the quality of selected videos, and assess their usefulness as an information source within their respective area.

Here is an illustration of how our study differs from those of our colleagues. Konstantinidis et al. (2013) identified the use of SNOMED terms among tags attached to YouTube health videos. The videos, providing information about surgery, were collected from a preselected list of hospital channels. This study examined tags from 4307 YouTube videos and found that 22.5% of these tags was a SNOMED term. Our study, however, stands in contrast to the work done by Konstantinidis et al. in a number of ways:

First, we detect the use of terminology from both the ICD-10 and MeSH vocabularies in video title and descriptions, which we found to be interesting medical vocabularies because of their purpose and application area. The MeSH terms were originally used for indexing articles for PubMed. However, such terms might also be relevant when health video creators describe the content of their videos. ICD, which is the standard diagnostic tool for epidemiology, health management, and clinical purposes, is used to classify diseases and other health problems.⁹¹ As such, the ICD-10 vocabulary should also be relevant for describing health related content in videos. Second, in contrast to the work presented by Konstantinidis et al., we expanded the types of YouTube channels to also include health organizations and users active in publishing videos about the disease, as opposed to just looking at hospital-produced YouTube videos. Third, we focused on a narrowly-defined medical condition, examining a broad range of health videos containing information specifically about diabetes, as opposed to the work of Kostantinidis et al. who examined health videos on the more general topic of surgery. For all these reasons, both individually and collectively, we feel that we can provide a richer and fuller understanding of how educational health videos measure up, when looking at how title and description conform to the medical terms that pertain to the health condition for which the user is seeking crucial web-based video material.