Readability Assessment of Online Men's Health Education Materials from the Sexual Medicine Society of North America, Urology Care Foundation, and European Association of Urology.

doi:10.21203/rs.3.rs-1735926/v1

Download PDF

Article

Readability Assessment of Online Men's Health Education Materials from the Sexual Medicine Society of North America, Urology Care Foundation, and European Association of Urology.

https://doi.org/10.21203/rs.3.rs-1735926/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Introduction: The use of online health information by patients has gained considerable attention during the last decade. The National Institute of Health (NIH) and American Medical Association (AMA) advocate the dissemination of accessible and readable resources. We aimed to evaluate the readability of online patient resources from the Sexual Medicine Society of North America (SMSNA) website.

Methods: Content published under the web-section “SMSNA: For Patients - conditions and topics'' portal (n=16) were reviewed. Identical topics published online via Urology Care Foundation (ie AUA; n=8) and the European Association of Urology (ie EAU; n=8) were also reviewed. Readability and estimated educational level required for understanding was assessed using validated readability and English language assessment tools including the Flesh Reading Ease score, Flesch Kincade, Gunning Fog, Simple Measure of Gobbledygook (SMOG), Coleman-Liau and Automated Readability index.

Results: SMSNA online patient education materials (PEM), on average, are written at a college sophomore reading level (14^th grade). This is 6.5 and 8 grade levels higher than the average U.S. adult and recommended reading levels for PEM, respectively. Comparable AUA and EAU resources were an average of 10th (p<0.05) and 12th (p=0.854) grade reading levels, respectively.

Conclusion: Men’s sexual health PEM’s published online are written at an advanced reading level, most notably the SMSNA compared to equivalent professional sources. This could pose a barrier to patient understanding and impact patients’ engagement and health decision-making. One proposed mitigatory strategy is for PEM to provide improved readability, and appropriate user-friendly language to facilitate easier and inclusive understanding, outreach and educational support.

Readability

sexual health

men’s health

online

The accessibility and prevalence of online health information available to patients have grown exponentially. While multifactorial, the meteoric rise in remote medical consultation and health care delivery has further accentuated patient pursuit for on-demand health-related materials. With an estimated 30% of adults in the United States (US) having an average education and literacy comparable to the seventh-eighth grade, professional associations including the American Medical Association (AMA), National Institute of Health (NIH), and Joint Commission (JCAHO) have advocated for patient education materials (PEM) to be written between a fifth and seventh-grade level. 1, 2. Existing literature has shown that a significant majority of PEMs’ available online often exceeds the literacy and comprehension level of the people reading it 3, 4. This can have a limit patients’ understanding, hinder shared decision making and even lead to poor patient satisfaction and outcomes.

Accurate, on-demand, and understandable PEM is especially important with sensitive health topics. Given that upwards of 70% of American internet users acknowledge using online sources for health information, understanding what resources are available can be very influential on patients' ultimate decision-making 5. Furthermore, ensuring that the available content is comprehensible to the greatest number of readers could also have far-reaching implications. According to a Council of Scientific Affairs report, patients with poor health literacy face increased risks of hospitalization, poor overall health status, and poor treatment compliance. They can face health care expenses nearly 4-fold that of a knowledgeable and health literate adult peer 6.

Patients affected by sexual health issues may be wary of discussing their symptoms with healthcare professionals, relying instead upon online sources for anonymous and free insight. However, resources must reach a broad audience with an accurate, consistent, and understandable message to have a constructive impact. This can ultimately be achieved by publishing high fidelity and easily readable content. While online material may vary in language or content, the inability to widely comprehend or understand through advanced readability could pose a significant barrier to patient care and understanding. Existing literature has evaluated PEMs within specific fields as well as specific conditions; however, not within the realm of men’s sexual health. In this manuscript, we aimed to describe online content specific to men's sexual health topics utilizing published content from the Sexual Medicine Society of North America (SMSNA), Urology Care Foundation (AUA), and the European Association of Urology (EAU).

We included and evaluated Sexual health-specific PEMs published online within the SMSNA (https://www.smsna.org/patients/conditions), Urology Care Foundation (https://www.urologyhealth.org/urology-a-z/all-conditions), and EAU (https://patients.uroweb.org/conditions/) patient information portals. SMSNA content was the principal source for topics, and when appropriate, analogous topics from both the Urology Care Foundation as well as EAU were included for comparison.

Text from each topic was copied into a blank Microsoft Word document (Microsoft Corp) where non-medical content was removed (author information, copyright information, website navigational tools and text, citations, advertisement text, figures, illustrations, etc.), leaving a clean text sample for analysis.

Language and reading assessments were then performed by entering each text through 6 validated instruments, including the Flesch Reading Ease Score (FRES), Flesch Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), and Automated Readability Index (ARI). These validated reading and language assessment calculators can be accessed through free online sources. FRES is an instrument that evaluates word and sentence length producing a score between 0-100, with higher scores (i.e., 90-100) representing easy-to-read text. FKGL, GFI, CLI, SMOG, and ARI are all validated indices that evaluate various components of a text and sentence structure (character count, syllables, word count, sentence length) to predict the approximate U.S. grade level required for ideal comprehension. Estimated grade levels from each index were rounded to the nearest whole grade number. The word count for each document was also captured.

When identical PEMs were identified within the AUA and EAU sites, this content was evaluated similarly and compared head-to-head with SMSNA content. All calculations and univariate analysis were conducted using Microsoft Excel (Microsoft Corp). This study was evaluated and exempt from IRB review (STUDY20220026).

Thirty-one PEMs were evaluated, 16 via SMSNA, 8 via AUA, and 7 via EAU patient resources (Table 1). The mean FRES was 35.8, 56.5, and 46.5 for SMSNA, AUA, and EAU content, respectively, all three of which would be classified as difficult-to-read content (Table 2).

Readability grade level scores from each of the societies were considerably greater than both the average and recommended reading level for patient-related materials amongst adults within the U.S.. The average estimated grade level for understanding all available SMSNA content across all readability assessments (FKRL, GFI, CLI, SMOG, ARI) was the highest of all sources at 14th grade (equivalent to second-year university); ranging between 11^th grade and 18^th grade (postgraduate education) (Table 1). The average grade level readability of AUA men's health content was equivalent to 10^th grade and ranging between 9^th and 14^th grade (second-year university) (Table 1). Lastly, the average grade level readability of EAU men’s health content was equivalent to 12^th grade and ranged between 10^th grade and 15^th grade (third-year university) (Table 1). Reading levels for all PEM’s are illustrated in Figure 1. SMSNA PEM’s were an average of 4 and 2 grade levels higher than AUA and EAU content respectively. Variance analysis results suggested a significant difference (p<0.05) in grade-level readability between SMSNA and AUA content. Comparison illustrations of similar PEM’s between both SMSNA and AUA as well as SMSNA and EAU are illustrated in Figure 2 and 3 respectively.

On average, the 31 articles analyzed were found to have a FRES of 47.9, corresponding to a difficult-to-understand reading level (Table 2). SMSNA featured the lowest FRES score (which conversely indicates the most difficult content) of 35.75 (Table 2). The 8 relevant PEMs analyzed from the AUA demonstrated an average FRES of 56.5 (difficult to read); the highest average FRES of the three sources analyzed (Table 2). As for the 7 relevant PEMs from EUA, the average calculated FRES was 46.5 (difficult to read) (Table 2). Overall, the documents analyzed far exceed widely accepted reading and comprehension levels for PEMs.

The average word count for SMSNA, AUA, and EAU material was 1892, 2829, and 3868, respectively.

Online content specific to men’s sexual health topics appears to be written at reading levels far above national recommendations. This gap in published material and understanding could become problematic, especially within the burgeoning realm of men’s health. Across a broad spectrum of medical (Rooney) journals, and within several surgical subspecialties (ophthalmology, otolaryngology, orthopedic surgery, and neurosurgery), evidence of PEMs authored at too high of a reading level for the general patient population have been reported (Table 2)(6-9). Urology is no stranger to this, as demonstrated most recently by Pook et al. evaluating online urologic content published through the British National Health Service; content which was reported to be at advanced reading levels to the general population (4). This is a trend that has been demonstrated within the urology literature for nearly a decade. Colaco et al. reviewed online urologic PEMs almost a decade ago from within the AUA and, more specifically, within academic urology departments, noting the excessive reading level required for understanding (10). Despite past and present reviews over nearly a decade, we continue to find a significant gap in appropriately published urology PEMs.

As online, self-directed health information engagement continues to grow, it is imperative for authors to understand the impact that content readability may have on patient outcomes. Evidence of this increased interest in men’s healthcare and therapeutics continues to be illustrated within the literature. For example, increasing the use of testosterone replacement therapy has been demonstrated time and time again 11-13. W hile these trends may likely be multifactorial, they are most certainly spurred on by growing patient motivation and access to free information. Ensuring that online patient-facing content is accurate and understandable is essential for all patients, regardless of access or resources.

Patients with inadequate financial, social or medical support may become marginalized through poor health literacy. This, compounded by difficulty understanding or often discussing complex and intricate disease processes, can profoundly impact morbidity. These patients face increased risks of hospitalization, poor overall health status, and poor treatment compliance. They can face health care expenses nearly 4-fold that of a knowledgeable and health literate adult peer (5). Understanding that male sexual dysfunction can potentially be a harbinger for more insidious and systemic conditions reinforces how important providing accurate and understandable content online can be for patients (14, 15).

Lastly, while all three sources queried are peer-reviewed and respectable bodies for urologic information, there appeared to be a discrepancy in readability across the board. PEMs fluctuated across 6 grade levels suggesting that content can be variable. Furthermore, when compared to each other, PEMs from the SMSNA and AUA were an average of 4 grade levels apart, enough to be statistically significant (p<0.5). EAU materials were only an average of 2 grade levels discordant from SMSNA (p>0.05). This wide discordance in PEMs online is likely to sow confusion and inconsistency regarding patients' proper understanding of their condition and ultimate therapeutic treatment options.

No singular mitigatory step can be performed to improve all PEM’s. Efforts to condense content for brevity and directness can be extensive requiring revision, piloting and input through multidisciplinary professionals (16). Examples of basic language principles used to create documents with easier readability include but are not limited to the avoidance of jargon, acronyms and abbreviations, organization of content presenting the most important information first and even the avoidance of statistics in favor of descriptive words such as ‘half’, ‘most’ or ‘many’ (17). Additionally, and not evaluated within the scope of this project, is the importance and ease of appearance and navigation of content to the reader especially with regards to the presentation and layout of the information with or without illustrations (both above sources).

In this study, we aimed to assess the readability of PEMs related to men’s sexual health published by premier urological associations. We included all pertinent material within each document and used validated assessment tools published within existing literature to perform this analysis. However, there are several limitations worth mentioning. Bias in locating these resources online could be present as medical professionals have often navigated such sites previously as opposed to patients who may use simpler searching or word-of-mouth navigation to reach. We were unable to assess how often the patients may view these articles and therefore unable to speculate or describe how impactful these PEMs are amongst the target population. Additionally, this study's readability and grade scores do not directly correlate to comprehension. There are several other factors, such as document layout, font size, the inclusion of graphics, and additional grammatical syntax, which can add to or detract from the ability of a layperson to understand a PEM (18).

Men’s sexual health PEMs published online are written at an advanced reading level, most notably the SMSNA compared to equivalent professional sources. This could pose a barrier to patient understanding and impact patients’ engagement and health decision-making. One proposed mitigatory strategy is for PEM publications to provide improved readability and appropriate user-friendly language to facilitate more accessible and inclusive understanding, outreach, and educational support.

US - United States

AMA - American Medical Association

NIH - National Institute of Health

JCAHO - The Joint Commission

PEM - patient education material

SMSNA - Sexual Medicine Society of North America

AUA - American Urologic Association

EAU - European Association of Urology

FRES - Flesch Reading Ease Score

FKGL - Flesch Kincaid Grade Level

GFI - Gunning Fog Index

CLI - Coleman-Liau Index

SMOG - Simple Measure of Gobbledygook

ARI - Automated Readability Index

Acknowledgements

None

Conflicts of Interest

No contributing author has any special interests, associations, or conflicts of interest to disclose with regards to this project.

Kutner M, Greenberg E, Jin Y, Paulsen C. The health literacy of America’s adults. National Center for Education Statistics. Published September 6, 2006. Available at:https://nces.ed.gov/pubs2006/2006483.pdf.
Stossel LM, et al. Readability of Patient Education Materials Available at the Point of Care. J Gen Intern Med. 2012;27(9):1165–70.
Colaco M, et al. Readability Assessment of Online Urology Patient Education Materials. The Journal of Urology. 2013;189(3):1048–52.
Pook CJ, Thomas K, Bultitude MF. Assessing the quality and readability of NHS urology webpages in England. Journal of Clinical Urology. May 2021. doi:10.1177/20514158211011904
Fox S, Rainie L. The online health care revolution: how the web helps Americans take better care of themselves. http://www.pewinternet .org/2000/11/26/the-online-health-care-revolution/. Accessed April 23, 2013.
Huang G, Fang CH, Agarwal N, Bhagat N, Eloy JA, Langer PD. Assessment of online patient education materials from major ophthalmologic associations. JAMA Ophthalmol. 2015 Apr;133(4):449 – 54. doi: 10.1001/jamaophthalmol.2014.6104. PMID: 25654639.
Eloy JA, Li S, Kasabwala K, et al. Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngol Head Neck Surg. 2012;147(5):848–854.
Sabharwal S, Badarudeen S, Unes Kunju S. Readability of online patient education materials from the AAOS website. Clin Orthop Relat Res. 2008;466(5):1245–1250.
Agarwal N, Chaudhari A, Hansberry DR, Tomei KL, Prestigiacomo CJ. A comparative analysis of neurosurgical online education materials to assess patient comprehension. J Clin Neurosci. 2013;20 (10):1357–1361.
Colaco M, et al. Readability Assessment of Online Urology Patient Education Materials. The Journal of Urology. 2013;189(3):1048–52.
Carter, I.V., Callegari, M.J., Jella, T.K. et al. Trends in testosterone prescription amongst medical specialties: a 5-year CMS data analysis. Int J Impot Res (2022). https://doi.org/10.1038/s41443-021-00497-6
Baillargeon J, Urban RJ, Ottenbacher KJ, Pierson KS, Goodwin JS. Trends in androgen prescribing in the United States, 2001 to 2011. JAMA Intern Med. 2013;173:1465–6.
Zhou CK, Advani S, Chaloux M, Gibson JT, Yu M, Bradley M, et al. Trends and patterns of testosterone therapy among U.S. male medicare beneficiaries, 1999 to 2014. J Urol. 2020;203:1184–90. https://doi.org/10.1097/JU.0000000000000744.
Jackson G. Erectile dysfunction and cardiovascular disease. Arab J Urol. 2013;11(3):212–216. doi:10.1016/j.aju.2013.03.003
Raheem OA, Su JJ, Wilson JR, Hsieh T-C. The Association of Erectile Dysfunction and Cardiovascular Disease: A Systematic Critical Review. American Journal of Men’s Health. May 2017:552–563. doi:10.1177/1557988316630305
Internet Citation: Assess, Select, and Create Easy-to-Understand Materials: Tool #11. Content last reviewed September 2020. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/health-literacy/improve/precautions/tool11.html
“Creating Patient Education Materials.” McGill University Health Centre Libraries, https://www.muhclibraries.ca/training-and-guides/creating-patient-education-materials/.
Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006;33:352–373.

Tables 1 and 2 are available in the Supplementary Files section.

There is NO conflict of interest to disclose.

Download PDF

Version 1

posted

You are reading this latest preprint version

Readability Assessment of Online Men's Health Education Materials from the Sexual Medicine Society of North America, Urology Care Foundation, and European Association of Urology.

Status:

Version 1

Abstract

Figures

Introduction

Methods

Results

Discussion

Conclusions

Abbreviations

Declarations

References

tables

Additional Declarations

Supplementary Files

Status:

Version 1