Risk Factors for Bladder Cancer: Challenges of Conducting a Literature Search Using PubMed

by Ashish Joshi, MD, MPH; and Elicia Preslan, MS

Abstract

The objective of this study was to assess the risk factors for bladder cancer using PubMed articles from January 2000 to December 2009. The study also aimed to describe the challenges encountered in the methodology of a literature search for bladder cancer risk factors using PubMed.

Twenty-six categories of risk factors for bladder cancer were identified using the National Cancer Institute Web site and the Medical Subject Headings (MeSH) Web site. A total of 1,338 PubMed searches were run using the term “urinary bladder cancer” and a risk factor term (e.g., “cigarette smoking”) and were screened to identify 260 articles for final analysis. The search strategy had an overall precision of 3.42 percent, relative recall of 12.64 percent, and an F-measure of 5.39 percent. Although search terms derived from MeSH had the highest overall precision and recall, the differences did not reach significance, which indicates that for generalized, free-text searches of the PubMed database, the searchers’ own terms are generally as effective as MeSH terms.

Key words: bladder cancer, risk factors, PubMed, information, retrieval

Introduction

Bladder cancer is a chronic condition and is estimated to be the ninth most common type of cancer worldwide (386,000 cases in 2008) and the thirteenth most frequent cause of cancer mortality (150,000 deaths).¹ Incidence rates are high in many southern and western European countries, in parts of North Africa and the Middle East, and in North America. The highest estimated mortality is in Egypt (age-standardized rate [ASR] 9.11 per 100,000), where rates are more than two times greater than the highest rates in Europe (Turkey, ASR 4.55; Macedonia, 4.27; Spain, 4.26) and four times greater than in the United States (ASR 2.11).²

Bladder cancer remains a significant health problem in the United States, with an estimated 70,500 new cases and 14,700 deaths from the disease expected in 2010.³ It affects approximately three to four times more men than women, across all races, and incidence increases with age.⁴ Data from 2009 indicated that more than half a million people in the United States are living with a diagnosis of bladder cancer.⁵ About 75 percent of the patients present with non-muscle-invasive disease in stage Ta (noninvasive papillary carcinoma), Tis (carcinoma in situ), or T1 (a tumor invading the bladder’s connective tissue but not the muscle layer), and although they typically have a long survival, these patients suffer from a high rate of tumor recurrence. Management of this population is costly because of the extended surveillance and repeated use of endoscopic and intravesical therapies. Patients with muscle-invasive and metastatic disease contribute greatly to the cost of bladder cancer care due to the expense of radical cystectomy and systemic chemotherapy. As a result, bladder cancer was ranked the fifth most expensive cancer and had the highest per patient treatment costs, which amounted to almost 3.7 billion U.S. dollars (2001 values) in direct costs.^6,7

It is difficult to interpret statistics on the incidence of bladder cancer “because of changing classification, variations in the counting of multiple cancers in the same individual, and most importantly, the variable inclusion of non-invasive cancers in different data sets.”⁸ Recent review articles have mentioned a broad spectrum of factors that could contribute to a person’s risk of developing bladder cancer.^9-12 However, these articles only addressed well-known risk factors such as cigarette smoking, food and beverage consumption, occupational exposures to aromatic amines (aniline, benzopyrene, liquefied natural gas, etc.), use of hair dyes, comorbid schistosomiasis infection, anticancer drugs, and contaminants in drinking water rather than surveying a broader range of possible risk factors like geographic location, drug interactions and side effects, or poor social conditions. Thus, the primary goal of this study was to assess the risk factors of bladder cancer using PubMed. In this article, we describe the challenges encountered while applying a generalized search strategy to locate the maximum number of relevant articles describing risk factors for bladder cancer.

Methods

In order to identify risk factors for bladder cancer, we started at the National Cancer Institute (NCI) Web site to search for relevant information (accessed October 2009). Some of the risk factors categorized included air and water pollution, alcohol, diet and nutrition, exogenous and endogenous hormones, familial aggregation, human papillomavirus, immunologic factors, inherited susceptibility, ionizing and solar radiation, occupation, pharmaceuticals other than hormones, tobacco, and viruses.¹³

We also consulted the NCI list of cancer causes and risk factors, which listed chemicals and environmental exposures (asbestos, agricultural products, benzene, formaldehyde, hair dyes), cancer clusters, food (acrylamine, artificial sweeteners, fluoridated water, heterocyclic amines), genetics, hormones (contraceptives, postmenopausal hormones), infectious agents (HIV, HPV, H. pylori), radiation (cell phones, magnetic fields, nuclear fallout, radon, medical imaging radiation), tobacco, physical activity (weight, obesity), and other factors (psychological stress, personal care products, other viruses).¹⁴

Finally, commonly mentioned risk factors in meta-analyses and review articles were included: smoking, occupational and chemical exposures, fluid intake, diet, urinary diseases, medications, family history of bladder cancer, genetic polymorphisms, radiation, physical exercise, and demographic factors (age, race, and gender).^15-18 Several environmental factors such as air, water, and soil pollution have been identified as increasing risk of bladder cancer. Other factors that are likely to increase risk of bladder cancer include radiation exposure, viral and bacterial infections, sexually transmitted diseases, decreased selenium intake, meat additives, and primary and secondhand smoking.

Using the Medical Subject Headings (MeSH) hierarchies from the National Center for Biotechnology Information (NCBI) as a template, we expanded and organized the topics from the NCI and scientific literature mentioned above into a set of risk factors within 26 categories and four broad domains (see Table 1).¹⁹ Each risk category contains a number of specific risk factors. For example, the nutrition risk category contains the dietary risk factors of food, beverages (specifically coffee and alcohol), and supplements (specifically vitamins, minerals, and food additives). A complete table of the risk factors used here can be found in Appendix 1.

Table 1: Potential Risk Factor Domains and Categories Identified for Urinary Bladder Cancer

Search Strategy

Once a set of risk factors was identified, we defined a list of search terms. Four sources were utilized to choose search terms: MeSH entry terms, MeSH subhierarchy terms, synonyms from Merriam-Webster’s Unabridged Collegiate Thesaurus online, and terms added by the searcher (E.P.).^20,21 The MeSH database is a “controlled vocabulary thesaurus” to index MEDLINE articles and help users build a search strategy.²² MeSH contains a hierarchy of biomedical terms with each term referencing other entry terms (also called reference mappings) as well as subhierarchy terms arranged below the given term on the MeSH tree. This search strategy included all the entry terms for a risk factor concept and the subhierarchy terms found one step lower on the MeSH tree. Thesaurus synonyms were included to supply nonbiomedical phrasings that might be found within the article’s text. Finally, supplemental terms were added by the searcher (E.P.) to provide other word choices and to supply terms not recorded by other sources. A sample list is shown in Figure 1.

Figure 1: Sample Search Terms for the Risk Factor of Health Behavior

To locate the maximum number of relevant articles, we decided to use free-text searches in order to include articles that were poorly defined by their title. To standardize the searches, we decided to use one fixed term for bladder cancer in conjunction with a second variable term to define the risk factor. For example, a search string might be “urinary bladder cancer health attitudes.” PubMed heuristics add the Boolean operator “AND” to free-text searches during the automatic concept mapping process, so for simplicity, search terms described in this paper were not combined using operators.²³

To define a single search term for the concept of bladder cancer, a series of searches was conducted on the 17 MeSH entry terms for bladder cancer (see Appendix 2). Word rearrangements (such as “cancer, bladder”) and terms that included prepositions (such as “cancer of the bladder”) had no effect on the number of articles found, so nine terms were excluded. Three terms were plural forms of other entry terms (as in “bladder cancers”) and were excluded since they located fewer articles than the singular forms. Bladder cancer and other tumors of the kidney, renal pelvis, ureters, and urethra all fall under the ICD-10 block of malignant neoplasms of the urinary tract (C64–C68), so to locate targeted results for neoplasms of the urinary bladder, terms lacking the word “urinary” were excluded. Of the three remaining terms (“urinary bladder neoplasm,” “urinary bladder tumor,” and “urinary bladder cancer”), “urinary bladder cancer” was selected to be the primary search term because it represented the broader concept of cancer rather than the just the physical tumor.

Searches were conducted in the same format each time, and the lists of articles retrieved were saved as text files and as saved searches within PubMed’s “My NCBI” feature. The number of articles retrieved by each search was also recorded. An overview of the search strategy is shown in Figure 2. Due to the large number of articles retrieved by certain searches, the search terms were screened using predefined exclusion criteria before the articles on a single risk factor were compiled.

Figure 2: Overview of the Search Strategy

Note: Risk factors were organized into four domains and 26 categories, and risk factor search terms were synonyms originating from four sources: MeSH entry terms, MeSH subhierarchy terms (terms one step lower on MeSH trees), thesaurus terms (from Merriam-Webster's Unabridged Collegiate Thesaurus online), and supplemental terms added by the searcher. Articles were identified by PubMed searches using the identified risk factor search terms with the term “urinary bladder cancer.” Duplicate articles were removed using the PubMed “Collections” feature of My NCBI. The remaining article titles were screened using exclusion criteria to identify articles of interest. Sources: U.S. National Library of Medicine. MeSH Subject Headings. (accessed April 2010). Merriam-Webster, Inc. Merriam-Webster's Unabridged Collegiate Thesaurus. (accessed November 2009).

Article lists were compiled using PubMed’s “Collections” feature in order to group all the articles for each risk factor and to eliminate duplicate articles. Each collection was manually screened by title to identify articles related to the given risk factor, which served as the reference standard. Titles were screened by both researchers a total of three times to achieve a fine precision, and each screening was saved as a separate collection within PubMed. Finally, the reference standard articles located by the search strategy were manually correlated back to the list of articles from each individual search term.

Exclusion Criteria and Search Limits

All searches were conducted with the following limits: date range 1/1/2000 to 12/3/2009 (environmental risk factor searches only) or 12/31/2009 (all other searches); clinical trial, controlled clinical trial, randomized controlled trial, review, meta-analysis, journal article, practice guideline, guideline, and government publications article types; human studies only; and English articles only. Meta-analysis and review articles were included in order to locate previous work for reference, but they were excluded from the articles selected for this systematic review.

Exclusion criteria for the search terms included duplicate terms, terms with multiple meanings (such as radiation), and terms retrieving zero articles. Searches retrieving an unusually large number of articles, as compared to other searches within a given risk factor, were assumed to be nonspecific and were excluded. Terms clearly representing the same article set were excluded for risk factors within the environmental exposure domain, but this criterion was not applied to other domains due to its subjectivity.

Articles were excluded if they did not refer to the risk factor of interest, included multiple cancer types, or were better studied under another risk factor. Articles involving the genetics of bladder cancer, treatments, or biomarkers were removed as were case reports, meta-analyses, and reviews.

Statistical Analysis

Search terms were analyzed according to their original source as well as by risk category. Percentages were calculated for the number of included and excluded terms and for the number of included terms locating reference standard articles. The search strategy was analyzed by precision, recall, and F-measure statistics using the method described by Hripcsak and Rothschild.²⁴ Precision is the proportion of articles retrieved by PubMed that were identified by manual screening (similar to positive predictive value). Relative recall, or the search sensitivity, was calculated as the number of articles found by manual screening divided by the expected number of articles. The F-measure is the harmonic mean of the precision and recall, which represents the proportion of reference standard articles that were agreed upon by PubMed and the searcher. Given that it is impossible to know how many articles of interest were not retrieved by our searches, we calculated relative recall using the total number of reference standard articles for each risk factor that were found by manual screening as the expected number of articles. The number of reference standard articles actually found for each search term was then used as the observed number of articles. The expected and observed values for each search term were then summed to give overall totals for each risk factor, risk category, and risk domain. A chi-square test statistic was calculated for each search term source and stratified by risk domain. Z-ratios were used to compare the significance of the difference between two proportions. ANOVA and two-sample t-tests were used to analyze the significance of the precision, relative recall, and F-measure statistics.

Results

A total of 1,338 searches were conducted. Overall, 490 search terms were excluded (36.7 percent) based, in general, on the number of articles each term retrieved. Among the 848 included searches, 517 searches (61 percent) failed to locate any reference standard articles, leaving only 331 search terms that found at least one reference standard article (39.0 percent). Figure 3 displays the outcome of the search terms from each source.

Figure 3: Percentage of Search Terms Excluded, Included But Not Locating Reference Standard Articles, or Included and Locating at Least One Reference Standard Article

Thesaurus search terms had the greatest number of terms excluded, frequently based on the criterion of finding zero articles (58.3 percent, n = 154. MeSH entry terms and terms added by the searcher had a similar distribution, but MeSH entry terms had a significantly greater proportion locating reference standard articles (160/445) when compared to searcher terms (70/254; z-ratio p = .023). Table 2 and Table 3 contain summary statistics of the number of articles identified, excluded, screened, and saved for future review.

Table 2: Number of Search Terms and Articles Identified, Screened, and Excluded Under the Search Strategy for Each Risk Factor Category

Table 3: Number of Search Terms Identified, Excluded, Included, and Returning Reference Standard Articles Under the Search Strategy for Each Source of Search Terms

The numbers of reference standard articles located by searches from each of the four term sources were all significantly different what was expected (chi-square p < .0001). However, there was no significant difference in the overall precision, relative recall, or F-measures of search terms from different sources. Table 4 shows that the overall precision ranged from 1.8 percent for thesaurus terms to 4.2 percent for MeSH entry terms (p = .9986). MeSH entry terms had the highest relative recall (23.7 percent), but the difference, when compared to the other search term sources, was nonsignificant (p = .1228). The overall F-measures were also nonsignificant (p = .8248).

Table 4: Total Number of Articles Located by Searches, Expected and Observed Numbers of Reference Standard Articles, and Precision, Relative Recall, and F-measures Stratified by Risk Factor Domain

In contrast, comparison of the overall precision, relative recall, and F-measures of search terms in each of the four risk domains did show significant differences (Table 5). The precision of searches for lifestyle risk factors was statistically different from the precision of each of the other domains (environmental exposures chi-square p = .00354, health status p = .03083, sociodemographics p = .05846). Lifestyle risk factors had also significantly different F-measures when compared to environmental exposures (p = .00827) and health status risks (p = .03496), but not when compared to sociodemographic risks. Relative recall for sociodemographic risk factors reached 31.1 percent, but there were no individually significant differences between specific domains.

Table 5: Overall Precision, Relative Recall, and F-measures for Searches in Each Risk Domain

Discussion

This study identified 260 articles related to 20 of the 26 risk factor categories for bladder cancer. In addition, this methodology demonstrated what a typical PubMed user might experience when using this database for generalized searching. MeSH terminology has been promoted as “a consistent way to retrieve information that may use different terminology for the same concepts.”²⁵ This study found that search phrases from MeSH entry terms were more likely to return at least one reference standard article than the search terms from other sources. However, there was no statistical difference in the precision or recall of the MeSH entry terms, and low precision and recall rates are one source of frustration for PubMed users.

This search strategy included choices that favored broad searches with the aim of increasing the number of relevant articles retrieved. An overall precision of 3.4 percent (Table 4) is logical if the search engine attempts to give back every possible result rather than miss relevant articles, yet, for a typical PubMed user, this indicates that the searcher must look through a list of search results in which 95 to 97 percent are not the desired articles. This study did not collect data on whether the reference standard articles were located in the first page of results, but other studies have reported that searchers are most likely to view abstracts from titles on the first page, with decreasing likelihood of viewing the remaining articles as the result size increases.^26,27 Few users have the patience to look through all the article titles that PubMed retrieves, and many do not make it past the first 10 or 20 results.

The low relative recall found by these searches is more troubling because, in general, precision and recall are inversely related. Given that there was no statistical difference in the precision and relative recall of search terms from different sources, this study appears to have internal validity. However, the methodological choices that lowered the precision should have increased the overall relative recall. One explanation might be that certain phrasings of an idea are much more commonly used in scientific literature, with the large number of less common phrases identified here decreasing the overall statistic. With 20 to 40 percent of included searches yielding no reference standard articles (Figure 3), the number of observed reference standard articles was greatly reduced, which also affected the relative recall.

The low recall rates for MeSH subhierarchy terms and thesaurus synonyms are more easily understood. Terms lower on MeSH trees tended to be examples of a concept rather than a synonym, so they may not have been as closely related to the topic and thus failed to find relevant articles. Thesaurus terms consisted of advanced as well as informal vocabulary rather than the scientific terminology used in biomedical literature. If these terms occur less frequently in PubMed, the searches will result in a lower recall. This could also explain the observed exclusion rate of thesaurus search terms of nearly 60 percent (Figure 3), which was mostly due to searches locating zero articles.

Other studies have been able to achieve higher precision and recall using targeted search algorithms, professional searchers, filters, or a narrower topic.^28-31 Another report found that professional librarian search experts had better precision but equivalent recall to other searchers with prior experience.³² However, the present study specifically chose not to use these strategies in order to attempt a more comprehensive review of the literature. As PubMed is widely used by searchers of all fields, sophisticated search strategies should not be necessary in order to retrieve the desired articles.

This study identified a number of gaps in the current knowledge about urinary bladder cancer. No articles could be found related to mental health, leisure activities, sexual behavior, education level, or social conditions as risk factors for bladder cancer. Differences in the precision and recall across the different risk domains reflect the level of knowledge about certain risk factors. For example, lifestyle and sociodemographic risk factors are more easily studied than environmental risks, and there may be more agreement on the terminology used to describe and search for these risk factors.

Some possible limitations of this study are the subjective nature of a manual screening process and the amount of research available on certain risk factors. It is likely that some risk factors were not included in this study. In addition, this study included potential risk factors that have not been well characterized, such as health behaviors, geographic location, marital status, and social conditions. There may have been slight variations in the results for environmental risk factors as the search methodology was being refined during these searches, and these searches were conducted a month earlier than those for the other domains. However, the other three risk domain searches were conducted in the same manner and time as each other, and the environmental domain results were consistent with the other domains.

Conclusions

This research has implications for the design of the PubMed database. As more diverse journals are added to the PubMed collections, new consumers are utilizing this resource, with a wide range of search demands. The database must be accessible to unsophisticated users as well as professional searchers. In addition, serious consequences can ensue if consumers fail to find all the literature relevant to their topic: at least one clinical trial has resulted in the death of a research participant because the researchers did not access key articles in the literature.³³ In summary, the literature search methodology described in this article was not as effective as was expected. The low precision and recall described here may account for the difficulty that users experience when conducting PubMed database searches. It is critical that literature databases be accessible to users of all backgrounds and take into account both generalized and targeted searches.

Ashish Joshi, MD, MPH, is an assistant professor in the College of Public Health at the University of Nebraska Medical Center in Omaha, NE.

Elicia Preslan, MS, is a graduate student in the Department of Epidemiology and Public Health at the University of Maryland in Baltimore, MD.

Support

No financial disclosure.

Acknowledgements

Thank you to Dr. Patrick McArdle for statistical advice.

Notes

¹ Ferlay, J., et al. “GLOBOCAN 2008: Cancer Incidence and Mortality Worldwide.” IARC CancerBase No. 10. Lyon, France: International Agency for Research on Cancer, 2010. Available at http://globocan.iarc.fr/ (accessed November 2010).

² Ibid.

³ Altekruse, S. F., et al. (Editors). SEER Cancer Statistics Review, 1975–2007. Bethesda, MD: National Cancer Institute, based on November 2009 Surveillance Epidemiology and End Results (SEER) data submission, posted to the SEER Web site 2010. Available at http://seer.cancer.gov/csr/1975_2007/index.html (accessed November 2010).

⁴ Ibid.

⁵ National Cancer Institute, Surveillance Research Program. “Fast Stats—Bladder Cancer Prevalence.” Based on SEER November 2009 submission. Available at http://seer.cancer.gov/faststats/ (accessed November 2010).

⁶ Sievert, Karl D., et al. “Economic Aspects of Bladder Cancer: What Are the Benefits and Costs?” World Journal of Urology 27 (2009): 295–300.

⁷ Botteman, Marc F., et al. “The Health Economics of Bladder Cancer: A Comprehensive Review of the Published Literature.” Pharmacoeconomics 21, no. 18 (2003): 1315–30.

⁸ Parkin, Donald M. “The Global Burden of Urinary Bladder Cancer.” Scandinavian Journal of Urology and Nephrology Supplement, no. 218 (2008): 12.

⁹ Hirao, Yoshihiko, Wun-Jae Kim, and Kiyohide Fujimoto. “Environmental Factors Promoting Bladder Cancer.” Current Opinions in Urology 19, no. 5 (2009): 494–99.

¹⁰ Jankovic, Slavenka, and Vladan Radosavljevic. “Risk Factors for Bladder Cancer.” Tumori 93 (2007): 4–12.

¹¹ Murta-Nascimento, Cristiane, et al. “Epidemiology of Urinary Bladder Cancer: From Tumor Development to Patient’s Death.” World Journal of Urology 25, no. 3 (2007): 285–95.

¹² Olfert, Sandra M., Sarah A. Felknor, and George L. Delclos. “An Updated Review of the Literature: Risk Factors for Bladder Cancer with Focus on Occupational Exposures.” Southern Medical Journal 99, no. 11 (2006): 1256–63.

¹³ National Cancer Institute. Current Research Topics. Available at http://dceg.cancer.gov/research/risk-factors (accessed November 2009).

¹⁴ National Cancer Institute. Cancer Causes and Risk Factors. Available at http://www.cancer.gov/cancertopics/causes (accessed November 2009).

¹⁵ Hirao, Yoshihiko, Wun-Jae Kim, and Kiyohide Fujimoto. “Environmental Factors Promoting Bladder Cancer.”

¹⁶ Jankovic, Slavenka, and Vladan Radosavljevic. “Risk Factors for Bladder Cancer.”

¹⁷ Murta-Nascimento, Cristiane, et al. “Epidemiology of Urinary Bladder Cancer: From Tumor Development to Patient’s Death.”

¹⁸ Olfert, Sandra M., Sarah A. Felknor, and George L. Delclos. “An Updated Review of the Literature: Risk Factors for Bladder Cancer with Focus on Occupational Exposures.”

¹⁹ U.S. National Library of Medicine. MeSH Subject Headings. Available at http://www.ncbi.nlm.nih.gov/mesh (accessed April 2010).

²⁰ Ibid.

²¹ Merriam-Webster, Inc. Merriam-Webster’s Unabridged Collegiate Thesaurus. Available at http://unabridged.merriam-webster.com/ (accessed November 2009).

²² U.S. National Library of Medicine. MeSH Subject Headings.

²³ National Center for Biotechnology Information, U.S. National Library of Medicine. “PubMed Help.” PubMed.gov. Available at http://www.ncbi.nlm.nih.gov/books/NBK3830/ (accessed May 2010).

²⁴ Hripcsak, George, and Adam S. Rothschild. “Agreement, the F-measure, and Reliability in Information Retrieval.” Journal of the American Medical Informatics Association 12, no. 3 (2005): 296–98.

²⁵ U.S. National Library of Medicine. MeSH Subject Headings.

²⁶ Islamaj Dogan, Rezarta, et al. “Understanding PubMed User Search Behavior Through Log Analysis.” Database (2009): bap018.

²⁷ Hoogendam, Arjen, et al. “Evaluation of PubMed Filters Used for Evidence-based Searching: Validation Using Relative Recall.” Journal of the Medical Library Association 97, no. 3 (2009): 186–93.

²⁸ Jiao, Dazhi, and David J. Wild. “Extraction of CYP Chemical Interactions from Biomedical Literature using Natural Language Processing Methods.” Journal of Chemical Information Modeling 49, no. 2 (2009): 263–69.

²⁹ Gehanno, Jean-François, et al. “Precision and Recall of Search Strategies for Identifying Studies on Return-to-Work in Medline.” Journal of Occupational Rehabilitation 19, no. 3 (2009): 223–30.

³⁰ Bachmann, Lucas M., et al. “Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read.” Journal of the American Medical Informatics Association 9, no. 6 (2002): 653–58.

³¹ Hoogendam, Arjen, et al. “Analysis of Queries Sent to PubMed at the Point of Care: Observation of Search Behavior in a Medical Teaching Hospital.” BMC Medical Informatics and Decision Making 8 (2008): 42–52.

³² McKibbon, Kathleen Ann, et al. “How Good Are Clinical MEDLINE Searches? A Comparative Study of Clinical End-User and Librarian Searches.” Computers and Biomedical Research 23, no. 6 (1990): 583–93.

³³ Glanz, James. “Clues of Asthma Study Risks May Have Been Overlooked.” New York Times (July 27, 2001). Available at http://www.nytimes.com/2001/07/27/us/clues-of-asthma-study-risks-may-have-been-overlooked.html (accessed April 2010).