Introduction
With the advent of revolutionary information technology, most general medical information can be accessed by the community at large. However, the factual nature of information, its understandability, and actionability of diseases like Hemophilia are unknown to the general population. Hence the present study has been envisaged to assess the understandability and actionability of available video information on YouTube about Hemophilia.
Methods
A cross-sectional study was performed using the Patient Education Materials Assessment Tool for Audiovisual materials (PEMAT-AV) to assess the understandability and actionability of 50 videos shown by order of relevance utilizing three independent assessors. An online google survey was prepared using the PEMAT questionnaire as a basis and results were recorded and saved as a Microsoft Excel sheet for analysis. Data was analyzed using either Microsoft Excel or an online calculator as the case may be.
Results
A total of 50 short videos on Hemophilia were assessed by three independent assessors using PEMAT. The data so obtained was rechecked by an independent reviewer before data analysis. Three videos were excluded due to non-English language while only two videos out of 50 showed 100% average understandability and actionability. Average understandability and actionability scores range between 34 to 100 percent and 11.1 to 100 percent, respectively. Most videos have higher average understandability than actionability (P value=0.003).
Conclusion
Our study shows there are only a few high-quality short videos available as audio-visual patient education materials on YouTube about Hemophilia. There is a great need to develop content that is beneficial to patients as patient educational material.
Introduction
The advent of the computer and mobile revolution brought with it the high-speed internet at the present at cheap affordable prices for all. This has increased health education information access to the public through websites, self-help groups, discussion groups, patient groups, chat groups, and app groups [1,2]. However, the information presented available through the internet is only as good as the knowledge of the presenter who can be doctors, paramedical staff, and sometimes patients and other public for generation of awareness. This may cause the presentation of nonactionable, false, or less understandable information to the patient leading to possible increased mortality and morbidity. With the rise of social media, the public is more and more into learning and searching for diseases on free video sites like YouTube.
YouTube and search engines have become the source of information for patients replacing the medical personnel in this time of the 5th Generation advanced cellular technologies and information age. Before a patient approaches a doctor for treatment, many a time they might have approached search engines and are more knowledgeable than we might perceive in the aspect of disease pathogenesis, and management. Hemophilia is an age-old, mostly congenital disease with a lot of information available on the internet and there is a huge chunk of information pertaining to the disease incidence, clinical features, pathogenesis, and management. However, it is difficult to go through all the existing data. To assess how useful the information is for patients, we performed this study to assess the understandability and actionability of videos available on YouTube regarding Hemophilia in terms of understandability and actionability using the Patient Education Materials Assessment tool-Audio/Visual (PEMAT-A/V) [3].
Materials & Methods
The YouTube search engine was searched with the keyword “Hemophilia” sorted by relevance with filters with a duration of under 4 minutes on 5th January 2022 and a playlist was created. We assumed the playlist generated by relevance will be the most seen videos in view of its default nature and given the short duration as most people find themselves reluctant to spend more time to watch lengthy videos when trying to learn something.
A sample size of 30 was assumed sufficient to maintain the integrity of the study and enough to warrant assertions against the present study findings. Assuming exclusion of some videos due to exclusion criteria (non-English videos), additional 20 videos were included taking the total number of videos to 50. The playlist was created using the first 50 videos assuming that the patients will not have the patience to see a long list of videos. The videos in the English language were included in the study out of the 50 videos, thus eliminating three videos. All 47 videos were checked and scored by three independent assessors who have been trained to score in a questionnaire based on the Patient Education Materials Assessment Tool-Audio Visual (PEMAT-AV) using google forms. All three assessors were from the medical background and can understand the intricacies involved in Hemophilia and were to assess the videos with an empathic view towards patients with not much knowledge of Hemophilia. The videos were assessed over a period of three months, values reviewed, videos rechecked for the correctness of data entered, data reviewed, analyzed over a period of six months in the year 2022 as per fixed timelines assessed during the study design. The present study has not been designed to assess the difference between the length of videos and understandability, actionability of videos but to assess the actionability and understandability of short videos alone.
The understandability was to be assessed for content, word choice and style, organization, layout and design, use of visual aids by a 13-point scale. Actionability was directly assessed by the 4-point system. The PEMAT(A/V) system was used in view of its specificity and individual scoring system. A training was provided to three assessors and one reviewer individually. This was followed by the scoring of the 47 videos individually using a google survey sheet for the collection of responses.
A total of 141 responses have been recorded with 47 responses from each of the three assessors. The assessors were made to score the responses independently at different locations to eliminate bias. The recorded data has been coded in such a way as to hide the identity of the assessor by giving a code (A, B, and C) and to not trace the response to a particular assessor except for the principal investigator. The final data has been reviewed by an independent reviewer to check for completeness of responses and to recheck the correctness of scores. The assessor’s and reviewer’s names have not been disclosed to either of them. After ascertaining the completion of the sheet, the data was analyzed using Microsoft® Excel for computation of actionability and understandability scores of individual assessors and average scores. The average understandability and actionability scores were to be compared using Wilcoxon signed rank test to identify if there is a difference in the mean between the two groups using an online calculator [4].
Results
The title of the video, creator of content, and the year of upload have been recorded by the assessors (Table 1). The assessor’s responses for each of the videos pertaining to the PEMAT score (Table 2) were incorporated and collected into google survey sheets and the same was analyzed. The cumulative responses of the assessors for each of the 13 understandability and four actionability scoring items have been analyzed (Figure 1).
The individual PEMAT scores from three assessors for all videos have been analyzed and combinedly ranged from 7.69% to 100% with a median of 77.78% and an inter-quartile range of 48.07% to 100%. The actionability scores similarly ranged between 0% to 100% with a median of 66.67% and an inter-quartile range of 33.33% to 100% (Figure 2). The PEMAT score is by itself subjective, however, it has been tried to mitigate the subjective nature using individual PEMAT scores given by three independent assessors to calculate the average PEMAT scores.
Average understandability and actionability scores ranged between 34 to 100 percent and 11.1 to 100 percent respectively as depicted in Figure 3. The average understandability and actionability of all 47 videos have been analyzed (Figure 4). A total of two videos had 100% understandability and actionability from all three independent assessors.
A Wilcoxon signed rank test was performed to identify the significance of the difference of means between average understandability and actionability using an online calculator and a statistically significant difference was found with a two-sided P value of 0.003 at 0.05 significance levels with a Z value of -2.9743 and W value of 254.
Discussion
YouTube is the second most visited website after Google and the second most visited social media platform and the world’s first website with audio-visual content [5,6].
Hemophilia is a disease characterized by a congenital or rarely acquired deficiency of coagulation factors, Factors VIII, IX, and/or XI and this leads to spontaneous bleeding or uncontrolled traumatic bleeding due to a defect in the coagulation.
The present study has provided sufficient data to suggest that the existing video material on Hemophilia on YouTube is understandable and actionable. Based on the present study, which required 50 videos to be assessed with a final result of only two 100% understandable and actionable videos, it is presumed that at least 25 short videos must be seen to get at least one 100% understandable and 100% actionable video by a patient. In the present study, there are few videos with zero actionability scores which indicates that the video content developers are more into providing information to the patients than providing information as to what to be done next.
In the present study, out of 47 videos screened, seven and 13 videos had average understandability and actionability scores of less than 50% respectively. The mean understandability score (70.9%) has just crossed the threshold of the PEMAT score for patient education materials of 70% [3], while the mean actionability score is below a threshold level (61.8%).
No literature could be found that searched the video material on hemophilia to assess understandability and actionability to date. There are certain studies which have assessed understandability and actionability using PEMAT scores but on different diseases like diabetes [7], cardiovascular disease risk [8], asthma [9], macular degeneration [10], sinusitis [11], vocal cord paralysis [12], cardiac electrophysiological procedures [13], shoulder arthroscopy [14], over the counter rapid antigen COVID-19 testing [15], fall prevention [16], etc. Most of these studies have higher understandability scores and poor actionability studies as has been identified in our study also. One finding that can be observed is the earlier videos in studies that were more emphasizing understandability while the recent videos and recent studies have increased actionability terms incorporated into the videos. This can improve further as the content developers get to know how to create content useful for patients.
The present study is limited by the sample size and study design to assess the understandability and actionability of short videos on Hemophilia only. The sample size could not be calculated in view of the paucity of data to use in sample size calculation. However, the present data can be utilized in designing better studies with a larger sample size in the future with more varied objectives.
Conclusions
Our study shows there are only a few high-quality short videos available as audio-visual patient education materials on YouTube about Hemophilia. Most of the short videos are understandable but poorly actionable. There is a great need to develop content that is beneficial to patients as patient educational material.
