An assessment of the quality and readability level of online content on urinary tract infection treatment in Spanish and English

Introduction

Background

The increasing recurrence and prevalence of urinary tract infections (UTIs) have led to a recent rise in patients seeking online health guidance (1). This trend aligns with society’s broader adoption of the internet as a primary health source since the coronavirus disease 2019 (COVID-19) pandemic (2,3). While online engagement increases each year, previous studies have emphasized the suboptimal quality of medical content shared on the internet (4-6).

A major limiting factor of online health information is the readability levels of published content. Many studies have found that most online health information does not follow the protocol of creating health materials at a 6th-grade reading level and below, as established by the American Medical Association (AMA) and National Institutes of Health (NIH) (7,8). Advanced reading levels have also been found to compose the majority of online patient education materials in the field of urology, including those focused on oncology, benign prostatic hyperplasia, kidney stones, and other related-specialty topics (9-13). However, no prior analysis has identified the readability levels of online content relating to the urological condition of UTIs.

Rationale and knowledge gap

Despite this technological surge, the benefit of online information for certain medical conditions like UTIs remains in question due to the minimal studies published. To our knowledge, this study is the first to examine the readability levels of UTI-related online content and the second study to explore the quality of this information. Additionally, this is the first study to evaluate these topics for Spanish UTI information available on the internet, providing a dual language comparison. With over 10% of the population in the U.S. expected to suffer from at least one UTI infection in their lifetime and an annual estimated treatment cost of roughly three billion dollars, there are several impactful reasons to examine UTI-related information via online articles and videos in multiple languages (14,15).

Objective

In this study, the quality and readability levels of popular English and Spanish UTI online content will be analyzed to provide a perspective on the value of UTI medical advice offered online.

Methods

Assessing UTI public interest via Google Trends

Google Trends (http://google.com/trends) measures the popularity and trends of terms explored within Google search engines and the search history of subsidiaries, such as YouTube. The search engine tool was used to assess online-user UTI interest on Google and YouTube between November 4, 2018, and November 3, 2023, a 5-year span. The metric used by Google to gauge the online popularity of searched terms is the search volume index (SVI). SVI is a grading scale ranging from 0 to 100, with 0 equating to no search history and 100 representing the highest popularity a term can reach relative to others. UTI search history data within the U.S. and its states was gathered in both English and Spanish in this study. Four reviewers were involved with performing the systematic search and review for the Google Trends analysis.

BuzzSumo

BuzzSumo (https://buzzsumo.com) is a social media content marketing tool. It was used to identify the most popular and shared UTI-related online articles and videos between November 4, 2018, and November 3, 2023. Data extraction for the reported engagement levels occurred on November 3, 2023. BuzzSumo compiles the number of views, likes, comments, shares, reactions, and mentions into an algorithm to determine which content received the most engagement on platforms such as Facebook, Twitter/X, Reddit, and YouTube. The 30 articles and videos with the most engagement in both languages were identified by querying identical keywords into BuzzSumo’s content analyzer. The search terms employed for the English materials were “urinary tract infections” and “UTI”, while the terms for the Spanish materials were “infeccion urinaria” and “ITU”. The inclusion and exclusion criteria were applied to the materials with the most engagement to filter articles and videos that were not relevant to this study. Materials were manually removed by reviewers from BuzzSumo analysis if their web addresses were found to be irrelevant, expired, duplicated, created in a different language, contained drug advertisements, or related to clinical trial recruitment posts. The inclusion criteria required a minimum of one view, like, comment, share, reaction, and mention for each material. After screening, 15 English articles qualified for analysis, while 14 English YouTube videos, Spanish articles, and Spanish YouTube videos were eligible for evaluation (Figure 1).

Figure 1 Flow diagram of the BuzzSumo selection process for the UTI materials with the most engagement. URL, uniform resource locator; UTI, urinary tract infection.

DISCERN

The DISCERN instrument is a quality assessment tool designed for consumers to assess the credibility of publications addressing treatment options for different health problems (16). DISCERN is a questionnaire consisting of 16 questions, with each question evaluated on a 5-point scale to judge the validity of medical articles and videos. The average score calculated from the 16 questions provides an overall score that can classify published works as low-, moderate-, or high-quality. A low average score is 2 or below (serious or extensive shortcomings), and a score of 3 reveals moderate quality content (potentially important but not serious shortcomings). Average scores of 4 and above are considered works of high-quality content (with minimal shortcomings). Three reviewers applied the DISCERN tool to grade the online UTI content in English, while two reviewers were designated to analyze the Spanish UTI content. Inter-rater agreement analysis was performed using the intraclass correlation coefficient (ICC) for both sets of materials in both languages.

Readability formulas

The readability levels of the articles and videos were assessed with a particular set of readability calculators, depending on the language. The Flesch-Kincaid Grade Level (FKGL) and Flesch-Kincaid Reading Ease (FKRE) metrics were utilized to measure the English materials. Both the FKGL and FKRE algorithms follow similar core metrics with heavily considering the average sentence length (ASL) and average word length (ASW) of a given material (Table 1). For the FKRE metric, a scale of 0–100 is utilized to determine reading difficulty, with lower scores stipulating greater comprehension complexity and with certain numerical ranges aligned with grade reading levels. The calculated values from the FKGL formula correlate to the specific grade reading level for each readability material (17-20) (Table 2).

For the Spanish materials, the Fernández-Huerta Index (FHI) and Índice de Legibilidad de Flesch-Szigriszt (INFLESZ) were the readability grading scales utilized with their appropriate formulas (Table 1). A grading scale of 0–100 is used for both readability formulas with specific score ranges corresponding to the appropriate grade reading levels. Comparably, a lower calculated score for both Spanish readability measures discloses increased reading difficulty (21-28) (Table 2).

Statistical analysis

The Mann-Whitney U test was the designated statistical test for assessing the mean SVI scores for the 5-year Google Trends data for the selected keywords, and readability analysis for all materials between the two readability metrics for each language, due to having non-parametric data sets. The ICC was employed to calculate the inter-agreement agreement score for the DISCERN ratings of the English and Spanish UTI materials.

Results

Public interest in UTIs

Analysis of the English term “UTI” on Google Trends and YouTube search engines revealed mean SVI values of 79 and 65 [95% confidence interval (CI): 12.00–15.03, P<0.01]. Mean SVI values for the Spanish term “infeccion urinaria” calculated to be 81 and 46 on the same platforms (95% CI: 38.00–44.00, P<0.01). Public interest was highest for “UTI” during the week of June 25, 2023, to July 2, 2023, with an SVI value of 100, which is representative of peak popularity. Similarly, “infeccion urinaria” scored a maximum SVI of 100 during the timeframe of May 21, 2023, to May 28, 2023. Over the 5-year search history, there was an upward trend in popularity for both terms on Google Trends, indicating a growing interest in this domain (Figures 2,3).

Figure 2 Google Trends interest of “UTI” data between 11/4/2018 and 11/3/2023. UTI, urinary tract infection.

Figure 3 Google Trends interest of “infeccion urinaria” between 11/4/2018 and 11/3/2023.

Within the U.S., West Virginia was the state with the highest score (SVI =100) for the term “UTI” while Florida led (SVI =100) for the term “infeccion urinaria” (Figures 4,5). Geographically, for YouTube search popularity, Hawaii had the highest SVI for “UTI” while New Jersey had the highest SVI for “infeccion urinaria”, both sharing a maximum SVI of 100 (Figures 6,7).

Figure 4 Google Trends online web interest for U.S. States between November 4th, 2018, and November 23rd, 2023, for “UTI”. UTI, urinary tract infection.

Figure 5 Google Trends online web interest for U.S. States between November 4th, 2018, and November 23rd, 2023, for “infeccion urinaria”.

Figure 6 Google Trends YouTube search interest for U.S. States between November 4th, 2018, and November 23rd, 2023, for “UTI”. UTI, urinary tract infection.

Figure 7 Google Trends YouTube search interest for U.S. States between November 4th, 2018, and November 23rd, 2023, for “infeccion urinaria”.

Online content data

BuzzSumo search history for the 5-year timeframe found that the most popular UTI English and Spanish articles accumulated 19,884 and 56,225 search results, respectively. Facebook and X (formerly named Twitter) were the primary platforms for content engagement amongst online users.

The English articles that garnered the most engagement on BuzzSumo were found in blogs and magazines (53%, n=8). The Spanish articles had similar results with almost half being published in blogs and magazines as well (43%, n=6). Both the English and Spanish articles had a low number of materials written by medical professionals (27%, n=4; 14%, n=2) (Tables 3,4).

In terms of the UTI YouTube videos with the most engagement, the English subset amounted to 187,401 views and 1,310 likes, while the Spanish subset had 1.1 million views and 2,560 likes. The videos exhibited more authorship from medical professionals than the articles in both languages. A total of 43% (n=6) of the English and 64% (n=9) of the Spanish videos were produced by health experts. Blogs and magazines (86%, n=12) were the primary producers of the English videos, while half of the Spanish videos were posted from health centers and physician-associated channels (50%, n=7) (Tables 5,6).

Content quality assessment

The mean scores for Question 16 from the DISCERN tool, which relates to the overall quality of a given publication calculated to be of lower quality across all material subsets. The mean quality scores for the UTI English and Spanish articles were computed to be 3.27 and 2.64, with the corresponding ICC values of 0.50 (95% CI: 0.40–0.92, P<0.01) and 0.94 (95% CI: 0.90–0.99, P<0.01) (Tables 3,4). The mean quality scores for the UTI English and Spanish YouTube videos were computed to be 3.10 and 2.82, with ICC values of 0.42 (95% CI: 0.27–0.90, P<0.01) and 0.48 (95% CI: 0.06–0.90, P<0.02) (Tables 5,6). The ICC values computed for the English articles, English videos, and Spanish videos delineate moderate agreement amongst reviewers. Only the ICC value for the Spanish articles delineates excellent agreement amongst reviewers.

The DISCERN question that, on average, scored the lowest across all material subsets was Question 11, which emphasizes to what extent the risks of each treatment were explained. The average scores for Question 11 for the UTI English and Spanish articles were calculated to be 1.51 and 1.96, with the corresponding ICC values of 0.40 (95% CI: 0.05–0.64, P<0.01) and 0.59 (95% CI: 0.02–0.82, P<0.01). The average scores for this question for the UTI English and Spanish YouTube videos were calculated to be 1.62 and 2.18, with the corresponding ICC values of 0.68 (95% CI: 0.09–0.93, P<0.01) and 0.61 (95% CI: 0.13–0.82, P<0.01). The ICC values for the English and Spanish articles are representative of fair agreement. Meanwhile, the English and Spanish videos display moderate to good agreement amongst reviewers.

Readability level

The mean FKRE and FKGL scores for the UTI English articles were calculated to be 49.46 and 10.79. These mean FKRE and FKGL values for the English articles are deemed to be at a college reading level and a 10th-grade reading level. The UTI English YouTube video transcripts averaged FKRE and FKGL scores of 57.08 and 9.29. According to the FKRE and FKGL metrics, the English video transcripts are between 10th to 12th grades and a 9th-grade reading level. Only two of the English materials, which were both video transcripts, calculated mean readability scores that met the AMA and NIH recommendation of creating health education materials at a 6th-grade reading level or below when considering the FKGL metric (Tables 2,3,5).

When applying the FHI and INFLESZ metrics to the UTI Spanish articles, the mean scores were measured to be 57.69 and 53.11. Inserting the Spanish articles into the two Spanish readability metrics showed that the material subset is representative of a reading level between the 10th to 12th grades and between the 9th to 10th grades. The Spanish YouTube video transcripts recorded mean FHI and INFLESZ scores of 58.25 and 53.86. The FHI and INFLESZ rating scales view the Spanish video transcripts to be gauged at a reading level between the 10th to 12th grades and the 7th to 8th grades. No Spanish articles and only two video transcripts met the expected readability standard when evaluated on both the FHI and INFLESZ scales (Tables 2,4,6).

Discussion

Key findings

Overall, the results from Google Trends show increasing popularity for UTI searches in English and Spanish, indicating a growing interest in UTI online information. Since 2018, searches for “UTI” and “infeccion urinaria” have steadily increased, with the greatest search volume being recorded for both languages in the last two years of the study. The last year of the study in 2023, was the year in which popularity for the term “UTI” attained the highest search popularity with an SVI of almost 90. In contrast, the Spanish search term “infección urinaria” attained an SVI of 90 in 2022. On a state level, West Virginia and Mississippi both surpassed SVI values over 90 for UTI Google search inquiries in English, which also occurred for Florida with Spanish inquiries. The states of Alabama, Tennessee, and Arkansas also recorded high SVI scores with values of 93, 89, and 88, respectively, for the search term “UTI”, illustrating significant national interest in learning more about UTIs in Google search engines.

For quality measures, when averaging the DISCERN quality score of all the articles and videos in both languages, the mean value falls slightly under the “potentially important but not serious shortcomings” range. The mean DISCERN quality ratings were higher for the UTI English materials than those for the UTI Spanish materials with ratings of 3.19 and 2.73. The lack of high-quality ratings for the UTI online articles and videos with the most engagement, as reported by BuzzSumo, reveals a major gap in urological care and patient education.

The readability analysis of the most popular UTI articles and videos gathered from the BuzzSumo algorithm shows significant misalignment with the AMA and NIH recommendations. No subset of materials in either language was calculated at an average reading level at or below the 6th-grade with any of the four readability metrics. This finding highlights the disparity in accessibility of UTI online content for patients who may not read above a 6th-grade level.

Strengths and limitations

Our study has several strengths. The originality of our research objectives explored new perspectives in urology, providing a meaningful contribution to the field. Our research methods were comprehensively described for the BuzzSumo selection process, which should make it feasible for future studies to reproduce results from the popularity algorithm. For quality assessments, the DISCERN rating scale was applied in its original form, ensuring a validated assessment tool was used. As to pursuing a bilingual approach, over half of the 70 million Americans who speak English as a second language are native Spanish speakers. By also investigating the quality and readability of Spanish online UTI content, we hope this study can successfully raise awareness about the importance of making health education materials effective for all patient populations in culturally rich nations like the U.S. (29-31).

There are several limitations to this study. As no reliable tool exists to assess the comprehension level of video-form content, the auto-generated YouTube English and Spanish video transcripts were inserted into the same readability calculators for analysis. Moreover, one article and three YouTube videos were not included for readability assessment due to this evaluation occurring at a later date than when data extraction was performed on November 3, 2023, leading to some web addresses no longer being valid.

For the BuzzSumo selection process, there is a possibility that the results would drastically differ if the same approach were conducted now due to the high amount of new online content being posted each day. This sheer volume of the internet makes it difficult to perform an all-encompassing quality analysis of UTI online content. While this study’s sample size could have been increased greatly, we believe a larger sample size would have not provided more insight due to high variation with the engagement indicators including the number of views, likes, comments, shares, reactions, and mentions for the articles and videos that fall outside the top 30 in engagement from the BuzzSumo algorithm. With the inclusion criteria requiring at least one of each engagement indicator, the most engaging materials that follow the top 30 for each category show a strong imbalance in the quantity of each indicator. Our team wanted the articles and videos assessed to have relatively high numbers across all metrics of engagement. Therefore, we postulate that applying the inclusion and exclusion criteria to the top 30 English and Spanish articles and videos is efficacious, considering that it provides an understanding of the quality of UTI content that patients are most likely to consume. With that said, there can be potential bias of the BuzzSumo content analyzer valuing each of the engagement indicators differently, explaining the variability amongst the most popular materials.

Comparison with similar research

Only two other studies have analyzed the quality of UTI online content. A study conducted by Tam et al. was the first to investigate UTI content on TikTok. The results of Tam et al. are limited as they adopted a simplified DISCERN scale consisting of a three-question rating system. This can likely explain why a discrepancy in quality ratings exists between our studies. Another key difference between our studies is that no readability analysis was conducted on their behalf. Their study was also limited by examining content solely on the social media platforms of TikTok and YouTube (32). The second study performed by Thaker et al. expanded the types of online UTI materials studied by including online articles and videos across various social media platforms. While our study follows a similar methodology to that of Thaker et al., our team evaluated the quality and readability levels of nearly 15 articles and videos in both English and Spanish, compared to their decision to assess only the quality of 10 English articles and videos. Our study, along with these two others, conveys that the quality of UTI online content can undergo significant improvement (32,33).

From our literature review, this is the first paper to evaluate the quality and readability levels of UTI online content. Nevertheless, two studies have also examined the quality and readability levels of YouTube videos for other urological conditions. Research conducted by Şahin et al. and Herbert et al. analyzed the quality of YouTube videos along with the readability levels of the corresponding transcripts. These papers’ results agree with this study’s low-quality and high readability scores. Together, there is unanimous agreement that the field of urology should prioritize improving upon these aspects for patients. All studies are consistent with emphasizing the importance of delivering reliable medical information on social media and the internet (34,35).

The publication of Şahin and colleagues applied the FKRE and FKGL metrics for their readability assessment, which prompted this study to follow the same process for judging UTI English content. We believe that sharing this direction would provide the most alignment and progress with advancing this relatively novel topic for UTIs. Furthermore, our procedure of inputting transcripts into the assigned readability calculators mimics that of Herbert et al. (34,35). We also appointed one team member to edit the auto-generated transcripts of our YouTube videos for punctuation before inserting them into readability calculators. This traditional approach was preferred over artificial intelligence (AI)-assisted software due to these programs being relatively new in academic research. While manual transcription of the YouTube videos might have offered greater accuracy for our readability results, the auto-generated transcripts were favored in this case to preserve uniformity with the closed captioning issued in each video. Certain patient populations depend on closed captioning to understand video content. For instance, deaf patients are forced to read closed captioning to learn or reinforce prior knowledge (36-38). Hence, our research team was convinced that this was a more equitable approach since some groups are often required to interact with closed captioning.

Studies evaluating the quality and readability levels of urology-specific online content are very limited. However, several studies have been published within urology applying the DISCERN tool to evaluate the quality of YouTube videos for conditions, such as bladder pain syndrome, vesicoureteral reflux, nocturnal enuresis, testicular cancer, percutaneous nephrolithotomy, and kidney transplantation. Almost all of these studies disclosed poor quality with the YouTube videos reviewed for each educational topic (39-44). For consistency purposes, our research team integrated the DISCERN scale to assess our materials to align with these prior studies. The literature shows that DISCERN is a prevalent tool for examining the quality of urology content on YouTube, but other validated quality assessment tools like the Patient Education Materials Assessment Tool, Journal of the American Medical Association (JAMA) Benchmark Score, and Global Assessment Score have been used as well. Regardless of quality assessment tools, the quality of YouTube videos associated with patient education in urology is relatively low-grade (45,46).

As mentioned earlier, evaluations of online readability levels have been performed for other urological conditions on the internet, but have yet to be completed for UTIs. Our paper’s readability results of UTI online content align with similar research inquiring into the readability of online materials for other urological conditions (9-13).

This study is the first to conduct a bilingual analysis of readability levels for UTI online content in English and Spanish. Several bilingual readability investigations have been published for other medical conditions, but are limited to the specialty of urology. These studies display the versatility of the many readability tools that can be used and interchanged to gauge appropriate Spanish reading levels. Our review process conveyed that a frequent approach involved administering the FHI and INFLESZ metrics for Spanish readability measures (47-49). This finding is what led our team to select the FHI and INFLESZ metrics to assess our Spanish UTI articles and videos for increased reliability purposes. We hope this endeavor adds to the status of global urology with increasing linguistic diversity by analyzing readability levels in multiple languages. This trend has recently been improving in urology, with readability assessments having been performed for associated conditions in other foreign languages (50,51). Nonetheless, there still lies a great need for readability evaluations to be carried out in more languages when examining UTI materials.

Explanations of findings

With UTI search history popularity reaching its maximum at the tail end in this study (Figures 2,3), it leads us to expect that internet usage for health educational purposes will increase in future years as the internet becomes more widely used. Currently, younger patient populations utilize the internet to learn and help resolve their medical conditions at a higher rate than older patient populations. According to previous research, approximately 94% of patients younger than 65 years old use the internet for health concerns compared to 66% of patients older than 65 years (52,53). As briefly emphasized by Hernandez et al., the current patient population older than 65 years old will be replaced by the younger patient population who almost ubiquitously use the internet, eventually leading to all-aged patients becoming dependent on online content for medical purposes (54). This expected reality of increased online engagement will make medical information more widespread on the internet, including content pertaining to UTIs.

It is plausible that online search popularity for UTIs can be increasing for reasons other than the internet becoming more mainstream for all ages. The rise in UTI incidence has been linked primarily to increasing worldwide multidrug resistance. Close to 80% of UTIs stem from bacterial sources with most cases relating to gram-negative bacteria (55,56). The significant number of medical prescriptions given per year to treat UTIs is the assumed cause for this microbial resistance (57,58). Laboratory studies have shown that gram-negative bacteria are acquiring gain-of-function mutations for their resistance mechanisms, such as encoding for extended-spectrum β-lactamases. The increase in β-lactamase production requires the use of more broad-spectrum antibiotics as cephalosporins are becoming less effective (59,60). This adaptive mechanism is very concerning from an infectious disease standpoint, as these gain-of-function mutations are occurring on genes that have already developed resistance to other antibiotic classes. UTI-inducing bacteria are now resistant to other antibiotic classes, including aminoglycosides, sulfonamides, and quinolones to some extent (61,62). While this study did not investigate antibiotic resistance in UTIs directly, it is critical to reflect upon the possibilities for upward trends in UTI online searches (Figures 2,3). Simply contributing this shift to increasing digital integration, as touched upon previously, can overlook the underlying issues concerning UTIs. Future research that aims to uncover the fundamental reasons why patients are becoming more reliant on the internet for UTIs can provide a rationale for the outcomes in this study.

In terms of Google Trends data of popularity by U.S. states, the fact that West Virginia, Mississippi, Texas, and Florida recorded peak SVI values during our 5-year study can be suggestive of a broader U.S. healthcare issue with access to care varying state-by-state (63). Recent U.S. healthcare data has ranked Texas, West Virginia, and Mississippi as three of the four worst states for health system performance, and Florida ranks in the bottom third of states (64). Alabama, Arkansas, and Tennessee were the U.S. states to record the next highest SVI values for the term “UTI” and from the same U.S. rankings list these three states were placed in the bottom 10 of states. While this study did not examine potential reasons for state-to-state online interest differences, future studies can examine this socio-cultural difference to determine why residents of certain states are seeking online health information more than others for UTIs and other health conditions.

Considering the popularity of the internet for health information, the DISCERN tool was implemented in this study to evaluate the effectiveness of the selected UTI online content. The DISCERN tool is a validated instrument designed to interpret the quality of health information and sources (16,65). Low-quality health information along with high readability levels hinders shared decision making (SDM) between physicians and patients, which is fundamental for upholding the medical ethical principle of autonomy. SDM can also be affected when patients are poorly informed about the risks of certain treatments as this study uncovered to be the case when assessing the materials via Question 11 on the DISCERN grading scale. Between the four subsets of materials, the mean score for Question 11 was 1.70, raising concerns for significant shortcomings. Garnering patient trust is key to establishing the patient-physician dynamic and without properly communicating the upsides and downsides of treatments it can negatively affect SDM (66,67).

With the average patient in the U.S. reading at an 8th-grade reading level, there is a clear disparity with the medical content available online to patients with urological conditions (68). Both the AMA and NIH recommend creating health education materials at 6th-grade reading level and below (7,8). With most of the health materials studied far exceeding this readability recommendation, health literacy is brought into question when assessing if online users benefit from the information being presented (69,70). A direct link exists between high readability scores and lower health literacy levels, which have been cited to impact patients by causing lower medical adherence and increased healthcare costs (71-73). For future health online information, it is advised that online content creators employ readability calculators to determine the reading grade level of materials and utilize AI tools, such as ChatGPT and Google Bard to help simplify the information being presented to meet the target readability level (74).

Implications and actions needed

The subpar quality and readability levels that repeatedly comprise online health information are likely the consequence of these materials not undergoing a rigorous approval process via external review or quality certification by professional committees. The absence of an online standardized verification process has raised concerns for propagating misinformation, which, in turn, disempowers patients with their overall well-being (75,76). Academic institutions and health systems are aware of this fact and mutually agree that no systematic approaches have been implemented on the web for regulating health content published online (77). In response to the absence of a validated online tool, acclaimed health organizations have designed free step-by-step guides informing the public on how to evaluate health information on the internet (78,79). Despite these resource evaluation guides, patients across all demographic groups are still having trouble identifying dependable online resources (80-82). Until an adequate framework is in place for monitoring the accuracy and dependability of online information, the medical community should continue incorporating and spreading awareness of data-driven resources that are valuable to both online creators and users with the aim of improving virtual health education. It is advised that individuals reference the current guidelines for UTI management set by the European Association of Urology for updated information (83).

Mobile health applications (apps) available to patients also lack regulatory measures, bringing forth more hurdles in evaluating the quality of digital health information (84). Prior investigations have conveyed that only a small percentage of health apps are regarded as higher quality, and of this fractional subset of apps, the majority are not the most downloaded or shared amongst app users (85,86). Several app evaluation tools have been developed, such as the Mobile App Rating Scale, Enlight, THESIS, and others, for quality assessment purposes. Even with these tools exemplifying proficiency with assessing the quality of health information, they are not broadly adopted due to a lack in other fundamental areas required for proper evaluation, such as practicality and clinical translation (84,87-89). Without accreditation mandates for mobile devices, distinguishing the quality of more than 350,000 health apps on device marketplaces is an infeasible task and unfair expectation for patients to overcome (90). It is recommended for patients to ask physicians directly for their advice on the best health apps to use for their specific medical conditions, considering the current state of health apps dispensing unverified information (91).

In essence, this study’s objective was to call attention to the low-quality content of popular UTI online health education materials being shared with the increasing number of patients using the internet for health-related reasons. Simultaneously, this study acknowledges the significant need for the medical community to improve upon developing patient educational materials for diverse patient populations. The readability levels of such materials directly influence the quality of the content being presented. Low-quality or complex presentations of health information can be expected to disproportionately affect patients facing challenges with health literacy or English comprehension (92,93). These findings highlight the importance of conducting future research aimed at following research-based recommendations when creating health education materials.

Conclusions

English and Spanish-speaking urology patients resort to online health information to resolve their UTI concerns. However, UTI online content for both languages exhibits low to medium quality ratings and readability levels that exceed evidence-based protocols. As the internet continues to become universally accessible to patients, it will be critical to improve the effectiveness of online health education materials.

Acknowledgments

Aspects of this manuscript were presented as a poster at the New England Student Urology Symposium on March 28, 2024, at the Warren Alpert Medical School of Brown University in Providence, Rhode Island. The title of the poster was initially named “An Assessment of Online UTI Treatment Content in Spanish and English”. An analysis of readability was performed and included in this manuscript after the conference.

Footnote

Peer Review File: Available at https://tau.amegroups.com/article/view/10.21037/tau-2025-221/prf

Funding: None.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tau.amegroups.com/article/view/10.21037/tau-2025-221/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Gonzalez G, Vaculik K, Khalil C, et al. Using Large-scale Social Media Analytics to Understand Patient Perspectives About Urinary Tract Infections: Thematic Analysis. J Med Internet Res 2022;24:e26781. [Crossref] [PubMed]
2. Venegas-Vera AV, Colbert GB, Lerma EV. Positive and negative impact of social media in the COVID-19 era. Rev Cardiovasc Med 2020;21:561-4. [Crossref] [PubMed]
3. Wang X, Cohen RA. Health information technology use among adults: United States, July-December 2022. doi: 10.15620/cdc:133700.
4. Weiner J. A personal reflection on social media in medicine: I stand, no wiser than before. Int Rev Psychiatry 2015;27:155-60. [Crossref] [PubMed]
5. Tasnim S, Hossain MM, Mazumder H. Impact of Rumors and Misinformation on COVID-19 in Social Media. J Prev Med Public Health 2020;53:171-4. [Crossref] [PubMed]
6. Johnson-Arbor K. I Read Health Information Online-Can I Trust It? JAMA Intern Med 2024;184:1138. [Crossref] [PubMed]
7. Weiss BD. American Medical Association. Health literacy. 2003;253:358. Available online: http://lib.ncfh.org/pdfs/6617.pdf
8. Rooney MK, Santiago G, Perni S, et al. Readability of Patient Education Materials From High-Impact Medical Journals: A 20-Year Analysis. J Patient Exp 2021;8:2374373521998847. [Crossref] [PubMed]
9. Colaco M, Svider PF, Agarwal N, et al. Readability assessment of online urology patient education materials. J Urol 2013;189:1048-52. [Crossref] [PubMed]
10. Pruthi A, Nielsen ME, Raynor MC, et al. Readability of American online patient education materials in urologic oncology: a need for simple communication. Urology 2015;85:351-6. [Crossref] [PubMed]
11. Bergersen AM, Khan I, Wong AC, et al. Online Kidney Stone Educational Materials Do Not Meet Recommended Readability Standards. Urol Pract 2021;8:246-52. [Crossref] [PubMed]
12. Pyun S, Malik RD. Assessing the Quality of Online Patient Decision Aids for Urological Conditions. Urol Pract 2021;8:692-8. [Crossref] [PubMed]
13. Giakas JA, Zaliznyak M, Kohut-Jackson A, et al. Quality and Readability of Online Health Information on Common Urologic Cancers: Assessing Barriers to Health Literacy in Urologic Oncology. Urol Pract 2024;11:670-6. [Crossref] [PubMed]
14. Simmering JE, Tang F, Cavanaugh JE, et al. The Increase in Hospitalizations for Urinary Tract Infections and the Associated Costs in the United States, 1998-2011. Open Forum Infect Dis 2017;4:ofw281. [Crossref] [PubMed]
15. Chu CM, Lowder JL. Diagnosis and treatment of urinary tract infections across age groups. Am J Obstet Gynecol 2018;219:40-51. [Crossref] [PubMed]
16. Charnock D, Shepperd S, Needham G, et al. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53:105-11. [Crossref] [PubMed]
17. FLESCH R.. A new readability yardstick. J Appl Psychol 1948;32:221-33. [Crossref] [PubMed]
18. Kincaid JP, Fishburne Jr RP, Rogers RL, et al. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. 1975; Institute for Stimulation and Training: 56. Available online: https://apps.dtic.mil/sti/html/tr/ADA006655/
19. Fitzsimmons PR, Michael BD, Hulley JL, et al. A readability assessment of online Parkinson’s disease information. J R Coll Physicians Edinb 2010;40:292-6. [Crossref] [PubMed]
20. Edmunds MR, Denniston AK, Boelaert K, et al. Patient information in Graves' disease and thyroid-associated ophthalmopathy: readability assessment of online resources. Thyroid 2014;24:67-72. [Crossref] [PubMed]
21. Fernández Huerta J. Simple measures of readability. Consigna 1959;214:29-32.
22. Garais EG. Web applications readability. Romanian Economic Business Review. 2011;5:117-21.
23. San Norberto EM, Taylor J, Salvador R, et al. The quality of information available on the internet about aortic aneurysm and its endovascular treatment. Rev Esp Cardiol 2011;64:869-75. [Crossref] [PubMed]
24. Bea-Muñoz M, Medina-Sánchez M, Flórez-García MT. Quality of websites with patient information about spinal cord injury in Spanish. Spinal Cord 2016;54:540-5. [Crossref] [PubMed]
25. Coco L, Colina S, Atcherson SR, et al. Readability Level of Spanish-Language Patient-Reported Outcome Measures in Audiology and Otolaryngology. Am J Audiol 2017;26:309-17. [Crossref] [PubMed]
26. Fuente-Cortez BE, García-Vielma C. Readability analysis of informed consent forms for genetic tests in Mexico. Gac Med Mex 2021;157:52-7. [Crossref] [PubMed]
27. Robert Sabaté L, Diego L. Are we offering patients the right medicines information? A retrospective evaluation of readability and quality in online patient drug information. Eur J Hosp Pharm 2021;28:144-8.
28. Gorrepati PL, Smith GP. Contrasting readability and availability of Spanish language with English language patient education materials. Pediatr Dermatol 2021;38:142-3. [Crossref] [PubMed]
29. Williams DR, Collins C. Racial residential segregation: a fundamental cause of racial disparities in health. Public Health Rep 2001;116:404-16. [Crossref] [PubMed]
30. Dietrich S, Hernandez E. Nearly 68 million people spoke a language other than English at home in 2019. United States Census Bureau. 2022. Available online: https://www.census.gov/library/stories/2022/12/languages-we-speak-in-united-states.html.
31. Lopez Vera A, Thomas K, Trinh C, et al. A Case Study of the Impact of Language Concordance on Patient Care, Satisfaction, and Comfort with Sharing Sensitive Information During Medical Care. J Immigr Minor Health 2023;25:1261-9. [Crossref] [PubMed]
32. Tam J, Porter EK, Lee UJ. Examination of Information and Misinformation about Urinary Tract Infections on TikTok and YouTube. Urology 2022;168:35-40. [Crossref] [PubMed]
33. Thaker S, Chan JYH, Thaker KN, et al. Public Interest in Online Information on Recurrent Urinary Tract Infections Is Greatest for Information with the Poorest Publication Quality. Pathogens 2024;13:1125. [Crossref] [PubMed]
34. Herbert AS, Nemirovsky A, Hess DS, et al. An Evaluation of the Readability and Content-Quality of Pelvic Organ Prolapse YouTube Transcripts. Urology 2021;154:120-6. [Crossref] [PubMed]
35. Şahin MF, Topkaç EC, Doğan Ç, et al. AI-assisted transcription of YouTube videos on penile enlargement: analysis of their text quality and readability. Sex Med 2025;13:qfaf023. [Crossref] [PubMed]
36. Gernsbacher MA. Video Captions Benefit Everyone. Policy Insights Behav Brain Sci 2015;2:195-202. [Crossref] [PubMed]
37. Revuelta P, Ortiz T, Lucía MJ, et al. Limitations of Standard Accessible Captioning of Sounds and Music for Deaf and Hard of Hearing People: An EEG Study. Front Integr Neurosci 2020;14:1. [Crossref] [PubMed]
38. Crandell HA, Silcox JW, Ferguson SH, et al. The Effects of Captioning Errors, Background Noise, and Hearing Loss on Memory for Text-Captioned Speech. J Speech Lang Hear Res 2022;65:2364-90. [Crossref] [PubMed]
39. Toprak T, Tokat E. A quality analysis of nocturnal enuresis videos on YouTube. J Pediatr Urol 2021;17:449.e1-6. [Crossref] [PubMed]
40. Duran MB, Kizilkan Y. Quality analysis of testicular cancer videos on YouTube. Andrologia 2021;53:e14118. [Crossref] [PubMed]
41. Toksoz A, Duran MB. Analysis of videos about vesicoureteral reflux on YouTube. J Pediatr Urol 2021;17:858.e1-6. [Crossref] [PubMed]
42. Morra S, Collà Ruvolo C, Napolitano L, et al. YouTube(TM) as a source of information on bladder pain syndrome: A contemporary analysis. Neurourol Urodyn 2022;41:237-45. [Crossref] [PubMed]
43. Yilmaz M, Sahin Y, Hacibey I, et al. Quality and utility of YouTube videos about mPCNL. Urolithiasis 2022;51:9. [Crossref] [PubMed]
44. Haberal HB, Piana A, Pecoraro A, et al. Decoding YouTube: An In-depth Analysis of Living Donor Kidney Transplantation Videos and Their Implications for Patient Education. Eur Urol Open Sci 2024;70:64-9. [Crossref] [PubMed]
45. Ji L, Sebesta EM, Stumbar SE, et al. Evaluating the Quality of Overactive Bladder Patient Education Material on YouTube: A Pilot Study Using the Patient Education Materials Assessment Tool. Urology 2020;145:90-3. [Crossref] [PubMed]
46. Tonyali S. YouTube: A good source for retrograde intrarenal surgery? Investig Clin Urol 2021;62:180-5. [Crossref] [PubMed]
47. Miguéns-Vila R, Ledesma-Ludi Y, Rodríguez-Lozano F, et al. Disparities between English and Spanish in readability of online endodontic information for laypeople. J Am Dent Assoc 2018;149:960-6. [Crossref] [PubMed]
48. Ramirez JL, Wang K, Sung E, et al. Readability of Spanish-Language Online Patient Educational Materials for Peripheral Artery Disease Do Not Meet Recommended Standards and Represent a Literacy Barrier to Care. Ann Vasc Surg 2024;101:157-63. [Crossref] [PubMed]
49. Ghahremani JS, Chapek M, Xie V, et al. Both English- and Spanish-Language Anterior Cruciate Ligament Reconstruction Online Patient Education Materials Are Written at Higher-Than-Recommended Reading Levels. Arthrosc Sports Med Rehabil 2024;6:100982. [Crossref] [PubMed]
50. Betschart P, Zumstein V, Hasan Ali O, et al. Readability Assessment of Patient Education Material Published by German-Speaking Associations of Urology. Urol Int 2018;100:79-84. [Crossref] [PubMed]
51. Lyatoshinsky P, Pratsinis M, Abt D, et al. Readability Assessment of Commonly Used German Urological Questionnaires. Curr Urol 2019;13:87-93. [Crossref] [PubMed]
52. Chen X, Siu LL. Impact of the media and the internet on oncology: survey of cancer patients and oncologists in Canada. J Clin Oncol 2001;19:4291-7. [Crossref] [PubMed]
53. Myers EM, Rustowicz L, Wells D, et al. Internet Use Among Urogynecology Patients in North Carolina. Female Pelvic Med Reconstr Surg 2015;21:269-72. [Crossref] [PubMed]
54. Hernandez CJ, Lim J, Takele R, et al. Identifying and Evaluating Online Kidney Stone Pain Resources: A Cross-Sectional Study. Cureus 2024;16:e73140. [Crossref] [PubMed]
55. Christy VR, Athinarayanan G, Mariselvam R, et al. Epidemiology of urinary tract infection in south India. Biomed Res Clin Prac 2019;4:1-5.
56. Ahmadi Z, Shamsi M, Roozbahani N, et al. The effect of educational intervention program on promoting preventive behaviors of urinary tract infection in girls: a randomized controlled trial. BMC Pediatr 2020;20:79. [Crossref] [PubMed]
57. Li X, Fan H, Zi H, et al. Global and Regional Burden of Bacterial Antimicrobial Resistance in Urinary Tract Infections in 2019. J Clin Med 2022;11:2817. [Crossref] [PubMed]
58. Biondo C. Bacterial Antibiotic Resistance: The Most Critical Pathogens. Pathogens 2023;12:116. [Crossref] [PubMed]
59. Bedenić B, Meštrović T. Mechanisms of Resistance in Gram-Negative Urinary Pathogens: From Country-Specific Molecular Insights to Global Clinical Relevance. Diagnostics (Basel) 2021;11:800. [Crossref] [PubMed]
60. Mancuso G, Midiri A, Zummo S, et al. Extended-spectrum beta-lactamase & carbapenemase-producing fermentative Gram-negative bacilli in clinical isolates from a University Hospital in Southern Italy. New Microbiol 2021;44:227-33.
61. Shaikh S, Fatima J, Shakil S, et al. Antibiotic resistance and extended spectrum beta-lactamases: Types, epidemiology and treatment. Saudi J Biol Sci 2015;22:90-101. [Crossref] [PubMed]
62. Mancuso G, Midiri A, Gerace E, et al. Bacterial Antibiotic Resistance: The Most Critical Pathogens. Pathogens 2021;10:1310. [Crossref] [PubMed]
63. Johnson EK, Wojtesta MA, Crosby SW, et al. Varied Health Spending Growth Across US States Was Associated With Incomes, Price Levels, And Medicaid Expansion, 2000-19. Health Aff (Millwood) 2022;41:1088-97. [Crossref] [PubMed]
64. Radley DC, Baumgartner JC, Collins SR, et al. scorecard on state health system performance. The Commonwealth Fund. 2020;2020:2020-09.
65. Rees CE, Ford JE, Sheard CE. Evaluating the reliability of DISCERN: a tool for assessing the quality of written patient information on treatment choices. Patient Educ Couns 2002;47:273-5. [Crossref] [PubMed]
66. Muscat DM, Shepherd HL, Nutbeam D, et al. Health Literacy and Shared Decision-making: Exploring the Relationship to Enable Meaningful Patient Engagement in Healthcare. J Gen Intern Med 2021;36:521-4. [Crossref] [PubMed]
67. Barlet MH, Barks MC, Ubel PA, et al. Characterizing the Language Used to Discuss Death in Family Meetings for Critically Ill Infants. JAMA Netw Open 2022;5:e2233722. [Crossref] [PubMed]
68. Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Community Health 2005;30:213-9. [Crossref] [PubMed]
69. Shedlosky-Shoemaker R, Sturm AC, Saleem M, et al. Tools for assessing readability and quality of health-related Web sites. J Genet Couns 2009;18:49-59. [Crossref] [PubMed]
70. Hirsch M, Aggarwal S, Barker C, et al. Googling endometriosis: a systematic review of information available on the Internet. Am J Obstet Gynecol 2017;216:451-458.e1. [Crossref] [PubMed]
71. Howard DH, Gazmararian J, Parker RM. The impact of low health literacy on the medical costs of Medicare managed care enrollees. Am J Med 2005;118:371-7. [Crossref] [PubMed]
72. Herndon JB, Chaney M, Carden D. Health literacy and emergency department outcomes: a systematic review. Ann Emerg Med 2011;57:334-45. [Crossref] [PubMed]
73. Rasu RS, Bawa WA, Suminski R, et al. Health Literacy Impact on National Healthcare Utilization and Expenditure. Int J Health Policy Manag 2015;4:747-55. [Crossref] [PubMed]
74. Moons P, Van Bulck L. Using ChatGPT and Google Bard to improve the readability of written patient information: a proof of concept. Eur J Cardiovasc Nurs 2024;23:122-6. [Crossref] [PubMed]
75. Arsenault M, Blouin MJ, Guitton MJ. Information quality and dynamics of patients' interactions on tonsillectomy web resources. Internet Interv 2016;4:99-104. [Crossref] [PubMed]
76. Beaunoyer E, Arsenault M, Lomanowska AM, et al. Understanding online health information: Evaluation, tools, and strategies. Patient Educ Couns 2017;100:183-9. [Crossref] [PubMed]
77. Daraz L, Morrow AS, Ponce OJ, et al. Can Patients Trust Online Health Information? A Meta-narrative Systematic Review Addressing the Quality of Health Information on the Internet. J Gen Intern Med 2019;34:1884-91. [Crossref] [PubMed]
78. US National Library of Medicine (NLM). Evaluating internet health information: a tutorial from the National Library of Medicine. 2010. Available online: https://medlineplus.gov/webeval/webeval.html
79. National Institutes of Health. How to evaluate health information on the internet: Questions and answers. 2011. Available online: https://ods.od.nih.gov/HealthInformation/How_To_Evaluate_Health_Information_on_the_Internet_Questions_and_Answers.aspx
80. Battineni G, Baldoni S, Chintalapudi N, et al. Factors affecting the quality and reliability of online health information. Digit Health 2020;6:2055207620948996. [Crossref] [PubMed]
81. Jin R, Simmons C, Lim HJ, et al. An Evaluation of the Quality of COVID-19 Internet Resources for Cancer Patients. J Cancer Educ 2023;38:763-71. [Crossref] [PubMed]
82. Daraz L, Dogu C, Houde V, et al. Assessing Credibility: Quality Criteria for Patients, Caregivers, and the Public in Online Health Information-A Qualitative Study. J Patient Exp 2024;11:23743735241259440. [Crossref] [PubMed]
83. Kranz J, Bartoletti R, Bruyère F, et al. European Association of Urology Guidelines on Urological Infections: Summary of the 2024 Guidelines. Eur Urol 2024;86:27-41. [Crossref] [PubMed]
84. Sedhom R, McShea MJ, Cohen AB, et al. Mobile app validation: a digital health scorecard approach. NPJ Digit Med 2021;4:111. [Crossref] [PubMed]
85. Singh K, Drouin K, Newmark LP, et al. Many Mobile Health Apps Target High-Need, High-Cost Populations, But Gaps Remain. Health Aff (Millwood) 2016;35:2310-8. [Crossref] [PubMed]
86. Byambasuren O, Sanders S, Beller E, et al. Prescribable mHealth apps identified from an overview of systematic reviews. NPJ Digit Med 2018;1:12. [Crossref] [PubMed]
87. Stoyanov SR, Hides L, Kavanagh DJ, et al. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015;3:e27. [Crossref] [PubMed]
88. Baumel A, Faber K, Mathur N, et al. Enlight: A Comprehensive Quality and Therapeutic Potential Evaluation Tool for Mobile and Web-Based eHealth Interventions. J Med Internet Res 2017;19:e82. [Crossref] [PubMed]
89. Levine DM, Co Z, Newmark LP, et al. Design and testing of a mobile health application rating tool. NPJ Digit Med 2020;3:74. [Crossref] [PubMed]
90. Wells C, Spry C. An Overview of Smartphone Apps. Canadian Journal of Health Technologies 2022; [Crossref]
91. Mescher T, Hacker RL, Martinez LA, et al. Mobile health apps: guidance for evaluation and implementation by healthcare workers. J Technol Behav Sci 2025;10:224-35.
92. Richwine J. Immigrant literacy: Self-assessment vs. reality. Washington DC: Center for Immigration Studies; 2017.
93. Escobedo LE, Cervantes L, Havranek E. Barriers in Healthcare for Latinx Patients with Limited English Proficiency-a Narrative Review. J Gen Intern Med 2023;38:1264-71. [Crossref] [PubMed]