Evaluation of Dengue-Related Health Information on the Internet

by Navya R. Rao, MPH; Manaswini Mohapatra, MPH; Swayamprabha Mishra, MPH; and Ashish Joshi, MD, MPH

Abstract

The objective of this study was to examine the quality of dengue-related health information on the Internet. Three raters used the keyword dengue to search the Google, Yahoo!, and Bing search engines during August 2011.

The first 20 websites from each search engine were examined for a total of 60 sites. Duplicate, nonfunctional, non-English, and nonoperational websites were excluded from the study, resulting in 36 sites for final analysis. The 16-item DISCERN tool was used to evaluate the quality of dengue-related health information on the Internet. Chi-square analysis and analysis of variance were performed to compare the DISCERN scores. Inter-rater reliability analysis showed significant differences in the level of agreement among the three raters. The 36 unique websites were categorized into .com, .edu, .gov, .org, and other sites. The .com sites had the lowest DISCERN scores. Educating consumers on how to find and recognize valid health information on the Internet may lead to better informed decision making.

Key words: dengue, health information, Internet, quality

Introduction

Dengue is an infectious disease that is transmitted by the bite of the Aedes mosquito infected with dengue viruses.^1,2 The symptoms range from a mild fever to an incapacitating high fever, with severe headache, pain behind the eyes, muscle and joint pain, and rash.³ Dengue, also called dengue fever, is a febrile illness that affects infants, young children, and adults and occurs in the tropical and subtropical areas of the world.⁴ A potentially lethal complication of dengue is hemorrhagic fever with more severe symptoms.⁵

Dengue has emerged as a rapidly growing public health problem with as many as 100 million people infected yearly.⁶ It is a significant cause of morbidity and mortality in many developing countries, including India.⁷ Important factors behind the increased incidence of dengue are increasing urbanization, crowding, and international travel. In recent times, the National Vector Borne Disease Control Programme in India has reported 28,292 cases of dengue with 110 deaths as of 2010.⁸ A systematic literature review on the economic burden of dengue infections has shown considerable economic losses incurred by developing countries such as India during each dengue epidemic.^9,10 According to an economic burden study conducted in 2006, the median cost of treatment per hospitalized dengue patient at a tertiary-level private hospital in north India is estimated to be $432.20 in U.S. dollars.¹¹ On an average, the total economic burden was estimated to be $27.4 million in U.S. dollars.¹²

As there is no specific anti-viral treatment for dengue, one method of controlling and preventing transmission of the dengue virus is to combat the vector mosquitoes by increasing awareness of the various preventive strategies.^13,14 These strategies include promoting awareness of environmental management practices and addressing basic services such as water supply, disposal of used water, solid waste management, and disposal of used tires.

Health education increases people’s knowledge about the disease and motivates them to participate actively in control measures. It informs people of available scientific knowledge related to the disease so that they can use this knowledge to bring about change in attitudes and practices for better health.¹⁵ Health education to prevent dengue transmission is essential to ensure that the knowledge, attitudes, and practices of community members are strengthened and thereby reduce disease severity and avoid fatalities.¹⁶ A prior study showed that dengue-related education increased the understanding of the problem and encouraged participation in measures aimed at preventing and controlling the disease.¹⁷

The Internet has empowered the human race in the pursuit of knowledge as no other media have before.¹⁸ It is continuously evolving as a global communications network, with several hundred million people worldwide using the Internet.¹⁹ More and more, patients go to the Internet to look for health information, which makes it necessary to ensure that they are referred to sources that do no harm. It is to the advantage of healthcare practitioners if patients want to educate themselves about their conditions. However, the quality of Internet-based medical information is extremely variable as there are no mandatory standards for peer review of websites. In a review of studies that examined the quality of health-related websites, 70 percent of the studies, using various scoring systems, concluded that the quality of information was inadequate.²⁰ Prior literature has shown that dengue affects all economic strata, which results in a diverse group of users seeking dengue-related information on the Internet.²¹ Providing patients with accurate information regarding dengue is an essential component of medical care. Traditional sources of healthcare information for patients include written material, pamphlets, drug information leaflets, oral communication with healthcare professionals, and formal educational programs.²² Patients also have the potential to acquire as much as information as their healthcare professional has by searching the Internet for the most current and applicable health information. However, the Internet also has the potential to provide wrong information to the care-seeking population looking for medical solutions online.²³ Therefore, it is necessary to assess the credibility of health information found on the Internet.²⁴ A previous study suggests three basic requirements for quality information: (1) the information is presented in a manner free from propaganda or disinformation (objectivity); (2) the information is a complete, not partial, picture of the subject (completeness); and (3) all aspects of the information are given, and the information is not restricted to present a particular viewpoint (pluralism).²⁵

The quality of health information on the Internet needs to be evaluated, and several tools exist for this purpose. The Information Quality Tool (IQT), a 21-item questionnaire, includes items relating to authorship, sponsorship, currency, accuracy, confidentiality, and navigability. Although each item requires a “yes” or “no” answer, items are weighted according to their importance. Items perceived to be most important are given a weight of 1, and the three items weighted 1 must be answered “yes” for the site to pass the test. Items perceived to be least important are given a weight of 0.036. The total score for the scale can range from 0 to 4. The internal consistency of the IQT, as measured by Cronbach’s a, was 0.634. The quality scale includes seven items relating to ownership, authorship, source, currency, interactivity, navigability, and balance. Each item is accompanied by a three-point Likert scale in which 0 indicates failure to satisfy the criteria for that item, 1 means that the criteria are partially satisfied, and 2 indicates that the criteria for that item are completely satisfied. The total score for the scale can range from 0 to 14. The internal consistency of the quality scale, as measured by Cronbach’s a, was 0.413. The DISCERN tool, a 16-item questionnaire, is also used to evaluate the quality of Internet information on treatment choices. The first section of this tool evaluates the reliability of the information, and the second section considers the quality of the information on treatment choices. Five-point Likert scales ranging from 1 (no) to 5 (yes) accompany these items. The final question assesses the overall rating of the publication on a five-point Likert scale ranging from 1 (low quality with serious or extensive shortcomings) to 5 (high quality with minimal shortcomings). Total scores can be calculated by summing the scores for items 1 to 15, which gives a score ranging from 15 to 75, with low scores indicating poor quality and high scores indicating good quality. The internal consistency of the DISCERN tool, as measured by Cronbach’s a, was 0.777. The DISCERN questionnaire is a reliable instrument for analyzing written consumer health information. It is the first standardized quality index of consumer health information that can be used as a critical appraisal tool to evaluate health information not only by health professionals but also by patients and the general population. This questionnaire was derived systematically with the input of an expert panel, health information providers, and patients from a self-help group.^26,27

Previous studies using the DISCERN tool have focused on the quality of websites for pain,²⁸ low back pain,^29,30 rheumatoid arthritis,^31,32 burn scar management,³³ and osteopororis.³⁴ In another prior study, a random sample of 25 websites was assessed by the three reviewers to determine the characteristics most commonly present on web pages.³⁵ It was decided that each subsequent website would be evaluated on potential for commercial gain, website seals of approval, language(s), and multimedia. The number of websites presenting these characteristics or lacking them was recorded. About 6.8 percent of the websites offered patients commercial products for their pain condition, 36.0 percent had a health-related seal of approval, 75.8 percent presented information in English only, and 40.4 percent offered an interactive multimedia experience.³⁶ Website seals of approval and potential for commercial gain were contributing factors to higher DISCERN scores, while seals of approval and interactive multimedia contributed to lower grade-level readability. The authors of a prior study evaluated 89 websites containing information about smoking cessation with the IQT, the quality scale, and the DISCERN tool.³⁷ Other studies assessed the quality of web-based information on multiple sclerosis³⁸ and Alzheimer’s disease.³⁹ One-quarter of the websites scored zero according to the IQT because they either did not disclose authorship or did not provide contact details for the author.

The DISCERN tool has shown to be relatively easy to use, and inter-rater reliability for assessments was good. The learning time is also acceptable, taking a user about one to two hours initially to become familiar with the rationale and layout of the document.⁴⁰ The objective of our study was to examine the quality of dengue-related health information on the Internet using the previously studied DISCERN tool.

Methods

Selection of Websites

We used the search engines Google, Yahoo!, and Bing in August 2011 to assess the quality of dengue-related health information. These search engines are often used by the consumers to obtain health information on the Internet.⁴¹ We recorded first 20 websites that appeared in the results of a keyword search for “dengue” on each of these search engines, thus generating 60 URLs. Duplicate sites were excluded from this search, allowing analysis of unique sites only. Additional exclusion criteria included nonfunctional websites and websites in languages other than English. Sites with inadequate information (sites with very limited information, such as information on only diagnosis or treatment of dengue), sites with nonoperational links, and nonfunctional sites (where the page was not found or was inactive) were also excluded from the study.

Criteria Development

Previous studies^42,43 have identified 9 criteria for website assessment, including the following:

1. Source (such as credentials, conflict of interest, and biases);
2. Disclosure (statement of purpose and profiling);
3. Accuracy;
4. Correctness of material;
5. Statement of original source;
6. Levels of evidence;
7. Disclaimers;
8. Link content (evaluated according to selection, architecture, content, and back linkages); and
9. Peer-review mechanisms (content reviewed by experts and colleagues in the related area).

Earlier, some studies indicated that the criteria to assess the quality of health information on the Internet must assess accuracy of the material, relevancy and clarity of the topic, limitations of the sites, and level of evidence in form of citations of peer-reviewed material.^44,45 In our study we used the DISCERN tool (www.discern.org.uk), which was designed to be used by health consumers and does not require previous knowledge of the subject.⁴⁶ It is a validated rating tool and can be used by health consumers or professionals alike.⁴⁷ The tool is a 16-item questionnaire (see the Appendix) that is used to assess the quality of health information on a site as well as to help health information providers and health consumers evaluate the site.⁴⁸

The 16 items of the DISCERN tool were placed into categories (Table 1). Each item in the questionnaire was scored on a scale of 1 to 5, in which 5 indicated that the site was a useful and appropriate source of information and 1 indicated a lack of information in relation to the described categories. Each website was assessed and given a score from 1 to 5 on each item. The content of a dengue-related website was considered accurate and complete only when the site provided information such as disease transmission, breeding sites, signs and symptoms, diagnosis, treatment approaches, and treatment effect. The sites that gave updated information on the incidence of dengue were also evaluated. Scores were assigned based on the number of evaluation items that were addressed within each category.

Table 1
Categories and Examples of the DISCERN Tool

Evaluation of Websites

The websites were cataloged as .com, .edu, .gov, .org, and other sites in accordance with their URL suffixes. Three raters, who were from diverse healthcare backgrounds and were frequent Internet users, independently assigned scores on a scale of 1 to 5 to each website. An average of the combined scores was used in the final assessment. A total score gives an overall rating of the information source and aims to provide high-quality criteria for consumer health information in various media.

Statistical Analysis

Descriptive analysis was performed using univariate statistics, and results of this analysis are reported as means and standard deviations. Inter-rater reliability analysis was performed to assess the level of agreement among the raters. Analysis of variance (ANOVA) was performed to compare the average scores assigned by the different raters for evaluating the objectives, relevance, information credibility, treatment choices, treatment effects, prevention, and overall quality of the publication. Stratified analysis was performed to compare the proportion of DISCERN scores across the categories of .com, .edu, .gov, .org, and other websites. Tukey’s test was also utilized to explore the multiple comparisons among the different raters. All results have been reported with 95 percent confidence intervals and p-values. We used SAS version 9.1 for the statistical analysis.

Inter-rater Agreement Statistics

A kappa statistic was applied to different website categories to compare the level of agreement among the three raters who evaluated the dengue-related health information on the Internet using the DISCERN tool. Results show differences in the level of agreement among DISCERN scores assigned by the three raters to the different website categories (see Table 2).

Table 2
Inter-rater Reliability Analysis

Results

The initial search resulted in 60 websites, of which 24 were excluded because they contained insufficient information, were not in English, had nonfunctional links, or were duplicates. The remaining 36 unique websites were included in the final evaluation (see Figure 1).

Figure 1
Description of the Internet Search for Dengue-Related Information

Results showed differences in the scores of the various website categories. The majority of the websites were found to be in the medium and high score categories. The three raters gave comparatively higher scores to sites in the .edu, .gov, and other website categories (see Table 3). Examples of websites that received low scores were http://en.wikipedia.org/wiki/2006_dengue_outbreak_in_India, which had limited information on the treatment choices, and http://www.1911encyclopedia.org/Dengue, in which the objectives were not clearly defined, information validity was poor, and no or limited information was provided on the treatment choices and dengue prevention.

Table 3
Total DISCERN Scores by Raters and Website Category

Significant differences among raters were found in the DISCERN scores for the various variables (see Table 4). Rater 1 was found to consistently give lower scores on all the variables in relation to evaluating the quality of the information. No significant difference in the scores was seen for the dengue information related to prevention.

Table 4
Comparison of the DISCERN Scores by the Three Raters on Each Variable

Results also showed that .com websites received the lowest average scores for website objectives, relevance, information credibility, treatment choices, treatment effects, and prevention. Websites in the .gov category received the highest scores for relevance, information credibility, and treatment choices, indicating a better quality of dengue-related health information. For prevention, sites in the .gov and other categories received the highest scores, followed by .edu sites (see Table 5).

Table 5
Comparison of the DISCERN Scores on Each Variable for the Various Website Categories

The websites were grouped into categories of excellent (scores of 63–75), good (51–62), fair (39–50), poor (27–38), and very poor (15–26) content.⁴⁹ Of all the websites rated, 46 percent were found to be excellent, 15 percent were good, 18 percent were fair, 9 percent were poor, and 11 percent were very poor (see Figure 2).

Figure 2
Frequency Distribution of DISCERN Scores by Percentage for Various Website Categories

Results showed variation in how the raters assigned scores to the different website categories. The lowest scores were received by .com websites. Results showed that the .edu sites were excellent in conveying dengue-related health information; however, they received lower scores for information credibility and relevance. This further suggests that information needs to be not only reliable but also personalized or tailored according to the needs of the user.

Discussion

The results of the study show that the number of high-quality websites was limited, but those sites had high information credibility and were more relevant. The other websites (not .gov, nor .org. nor .edu, nor .com that were given higher scores by the raters were predominantly World Health Organization sites that communicated evidence-based high-quality information to the users. The findings of our study are important as they help to identify the gaps in the dissemination of dengue-related health information. In the .com and .org website categories, the treatment choices and treatment effects received the lowest scores, resulting in the poor quality of dengue-related health information.

Significant variation has been found in previous studies with regard to information on the Internet about dengue.⁵⁰ Several studies^51-54 have suggested that the Internet can be a useful source of health information and can assist patients and providers with clinical decision making. Because the Internet reaches a large part of the population today and individuals increasingly turn to it for health information, it plays an important role in public health.^55,56

The ability to obtain online medical information accurately, quickly, and conveniently offers consumers an opportunity for better informed decision making.⁵⁷ Searching for useful and valid information on the Internet can be difficult because of the speed and lack of control with which the information is accumulating. Tools such as Internet directories, indexes, and search engines assist healthcare providers and consumers in their search for health information on the World Wide Web. In this study, three independent raters who were frequent Internet users were selected to search for information on the Internet using the most common search engines. There were differences in how the quality of the information was rated by the three independent raters. However, there could be various reasons for these differences, including prior knowledge and familiarity with the health information content. Another reason for this lower inter-rater reliability could be the subjective nature of some of the questions. It is important to consider the user’s perspective when presenting health information content over the Internet.⁵⁸

Previous study results have demonstrated a fair degree of disagreement between medical experts when they are asked to rate medically related postings from the Internet.⁵⁹ Numerous studies have shown that the information on the Internet is of poor quality and have suggested that future studies should employ more than one rater.⁶⁰ It is less of a concern if one expert fails to agree with the others than if several experts disagree with each other. It is possible that training or other resources might increase agreement between experts, and future research could consider this. The benefits of having many raters need to be weighed against the possibility of having unqualified or uninformed medical workers (or laypeople) judge web information incorrectly. For best results, a previous study has recommended having two observers simultaneously rate the quality of health information on the Internet and resolve any differences by consensus.⁶¹

Previous study results have suggested that voluntary organizations should regularly review information on their websites, specifically relating to the provision of up-to-date information on preventive aspects of dengue, and should also ensure the quality of information on the website by providing author credentials and affiliations.⁶² Variability in the quality of dengue-related health information websites with respect to core content points to the need for a grading system that would allow healthcare professionals to signal trustworthy, up-to-date websites so that consumers can receive high-quality information to assist them in making informed decisions regarding treatment and care. There are no clear universal guidelines governing healthcare information; therefore, several different approaches, such as trust marks that sites can display and principles that sites can use to govern their own behavior, should be advocated. Opportunities must also be created for public health experts and officials to work more effectively with local journalists to increase the impact of public health messages available on websites.

This study has several limitations. It provides only a snapshot in time of information represented in a rapidly changing medium. We expect that changes to the websites that were evaluated would already alter some of the findings from the date of the search. Earlier studies have also recommended several ways to improve web-based information:

1. Methods of indexing Web pages should be improved.
2. Website developers need to ensure that the information is accurate and may be useful.
3. Information on the web needs to be made more readable for users across different socioeconomic backgrounds.⁶³

The Internet has the potential to be a powerful resource for meeting some of the public’s health information needs. A shared responsibility between health information consumers and website developers would enable the design and development of websites that are targeted to address the needs of the individuals. Consumers are generally not aware of characteristics that indicate quality information on the Internet.⁶⁴ The results of our study help us recognize websites that might be useful to users for gathering dengue-related health information on the Internet. Educating consumers to find and recognize valid health information may lead to better informed decision making.

Conclusion

This study addresses the lack of high-quality websites that deliver dengue-related health information on the Internet. Further, the study emphasizes the need for websites to provide a clear statement of purpose and tailored health information in order to serve as a means of consumer empowerment.

Appendix

DISCERN Tool

Navya R. Rao, MPH, is a researcher at the Center for Public Health Informatics, Asian Institute of Public Health in Bhuabneswar, India.

Manaswini Mohapatra, MPH, is a public health student at the Asian Institute of Public Health in Bhuabneswar, India.

Swayamprabha Mishra, MPH, is a public health student at the Asian Institute of Public Health in Bhuabneswar, India.

Ashish Joshi, MD, MPH, is an assistant professor of the Center for Global Health and Development in the Nebraska Medical Center’s College of Public Health in Omaha, NE.

Notes

¹ World Health Organization. “Dengue.” 2011. Available at http://www.who.int/topics/dengue/en/ (accessed August 21, 2011).

² MedlinePlus. “Dengue Fever.” U.S. National Library of Medicine, 2011. Available at http://www.nlm.nih.gov/medlineplus/ency/article/001374.htm (accessed August 21, 2011).

³ World Health Organization. “Dengue.”

⁴ Centers for Disease Control and Prevention. “Dengue.” 2011. Available at http://www.cdc.gov/dengue/ (accessed August 21, 2011).

⁵ MedlinePlus. “Dengue Fever.”

⁶ Garg, P., J. Nagpal, P. Khairnar, and S. L. Seneviratne. “Economic Burden of Dengue Infections in India.” Transactions of the Royal Society of Tropical Medicine and Hygiene 102, no. 6 (2008): 570–57.

⁷ Centers for Disease Control and Prevention. “Dengue.”

⁸ National Vector Borne Disease Control Programme. “Dengue Cases and Deaths in the Country since 2007.” Government of India, Directorate General of Health Services, 2011. Available at http://nvbdcp.gov.in/den-cd.html (accessed August 21, 2011).

⁹ Garg, P., J. Nagpal, P. Khairnar, and S. L. Seneviratne. “Economic Burden of Dengue Infections in India.”

¹⁰ Beatty, M. E., P. Beutels, M. I. Meltzer, D. S. Shepard, J. Hombach, R. Hutubessy, D. Dessis, L. Coudeville, B. Dervaux, O. Wichmann, H. S. Margolis, and J. N. Kuritsky. “Health Economics of Dengue: A Systematic Literature Review and Expert Panel’s Assessment.” American Journal of Tropical Medicine and Hygiene 84, no. 3 (2011): 473–88.

¹¹ Garg, P., J. Nagpal, P. Khairnar, and S. L. Seneviratne. “Economic Burden of Dengue Infections in India.”

¹² Gubler DJ, Meltzer M, 1999 .Impact of Dengue/Dengue Hemorrhagic Fever on the Developing World. Advances in Virus Research 53:35 – 70.

¹³ World Health Organization. “Dengue and Dengue Haemorrhagic Fever.” 2009. http://www.who.int/mediacentre/factsheets/fs117/en/ (accessed August 21, 2011).

¹⁴ World Health Organization. Guidelines for Treatment of Dengue Fever/Dengue Haemorrhagic Fever in Small Hospitals. 1999. Available at http://www.searo.who.int/LinkFiles/Dengue_Guideline-dengue.pdf (accessed August 21, 2011).

¹⁵ World Health Organization. “Regional Guidelines on Dengue/DHF Prevention and Control.” Available at http://www.searo.who.int/en/Section10/Section332/Section554_2570.htm (accessed August 21, 2011).

¹⁶ World Health Organization. Guidelines for Treatment of Dengue Fever/Dengue Haemorrhagic Fever in Small Hospitals.

¹⁷ World Health Organization. “Regional Guidelines on Dengue/DHF Prevention and Control.”

¹⁸ Leaffer, T., and B. Gonda. “The Internet: An Underutilized Tool in Patient Education.” Computers, Informatics, Nursing 18, no. 1 (2000): 47–52.

¹⁹ Wyatt, J. C. “Measuring the Quality and Impact of the World Wide Web.” British Medical Journal 314, no. 7098 (1997): 1879–80.

²⁰ Eysenbach, G., J. Powell, O. Kuss, and E. R. Sa. “Empirical Studies Assessing the Quality of Health Information for Consumers on the World Wide Web: A Systematic Review.” Journal of the American Medical Association 287, no. 20 (2002): 2691–2700.

²¹ Bhavsar, A. T., D. S. Shepard, J. A. Suaya, M. Mafowosofo, C. L. Hurley, and M. W. Howard. “A Private Hospital-based Study Assessing Knowledge, Attitudes, Practices and Costs Associated with Dengue Illness in Surat, India.” Dengue Bulletin 34 (2010).

²² University of Oxford and the British Library. DISCERN Handbook: Quality Criteria for Consumer Health Information on Treatment Choices. 1998. Available at http://www.discern.org.uk/discern.pdf (accessed August 21, 2011).

²³ Eysenbach, G., J. Powell, O. Kuss, and E. R. Sa. “Empirical Studies Assessing the Quality of Health Information for Consumers on the World Wide Web: A Systematic Review.”

²⁴ Bennett, G. G., and R. E. Glasgow. “The Delivery of Public Health Interventions via the Internet: Actualizing Their Potential.” Annual Review of Public Health 30 (2009): 273–92.

²⁵ Floridi, L. “Brave.Net.World: The Internet as a Disinformation Superhighway?” Electron Library 14, no. 6 (1996): 509–14.

²⁶ Kaicker, J., V. B. Debono, W. Dang, N. Buckley, and L. Thabane. “Assessment of the Quality and Variability of Health Information on Chronic Pain Websites Using the DISCERN Instrument.” BMC Medicine 8 (2010): 59.

²⁷ Currie, C., P. D. Mambro, A. Joice, R. McGilp, M. O’Neill, G. Ralston, R. Davidson, and J. Scot. “Evaluating the Quality of Educational Materials about Schizophrenia.” Psychiatric Bulletin 26 (2002): 96–98.

²⁸ Lorence, D., and J. Abraham. “A Study of Undue Pain and Surfing: Using Hierarchical Criteria to Assess Website Quality.” Health Informatics Journal 14, no. 3 (2008): 155–73.

²⁹ Greene, D. L., A. J. Appel, S. E. Reinert, and M. A. Palumbo. “Lumbar Disc Herniation: Evaluation of Information on the Internet.” Spine 30, no. 7 (2005): 826–29.

³⁰ Li, L., E. Irvin, J. Guzman, and C. Bombardier. “Surfing for Back Pain Patients: The Nature and Quality of Back Pain Information on the Internet.” Spine 26, no. 5 (2001): 545–57.

³¹ Culver, M., and A. Chadwick. “Internet Information on Rheumatoid Arthritis: An Evaluation.” Musculoskeletal Care 3, no. 1 (2005): 33–43.

³² Newman, M. A., S. Ziebland, and K. L. Barker. “Patients’ Views of a Multimedia Resource Featuring Experiences of Rheumatoid Arthritis: Pilot Evaluation of www.healthtalkonline.org.” Health Informatics Journal 15, no. 2 (2009): 147–59.

³³ Bohacek, L., M. Gomez, and J. S. Fish. “An Evaluation of Internet Sites for Burn Scar Management.” Journal of Burn Care and Rehabilitation 24, no. 4 (2003): 246–25.

³⁴ Eysenbach, G., and T. L. Diepgen. “Towards Quality Management of Medical Information on the Internet: Evaluation, Labelling, and Filtering of Information.” British Medical Journal 317 (1998): 1496.

³⁵ Kaicker, J., V. B. Debono, W. Dang, N. Buckley, and L. Thabane. “Assessment of the Quality and Variability of Health Information on Chronic Pain Websites Using the DISCERN Instrument.”

³⁶ Khazaal Y, Chatton A, Cochand S, Coquard O, Fernandez S, Khan R, Billieux J, Zullino D:Brief DISCERN, Six Questions for the Evaluation of Evidence-based Content of Health-related Websites. Patient Education and Counsiling 2009, 1:33-37.

³⁷ Ademiluyi, G., C. E. Rees, and C. E. Sheard. “Evaluating the Reliability and Validity of Three Tools to Assess the Quality of Health Information on the Internet.” Patient Education and Counselling 50, no. 2 (2003): 151–55.

³⁸ Harland, J., and P. Bath. “Assessing the Quality of Websites Providing Information on Multiple Sclerosis: Evaluating Tools and Comparing Sites.” Health Informatics Journal 13, no. 3 (2007): 207–21.

³⁹ Bouchier, H., and P. A. Bath. “Evaluation of Websites That Provide Information on Alzheimer’s Disease.” Health Informatics Journal 9, no. 1 (2003): 17–31.

⁴⁰ Lorence, D., and J. Abraham. “A Study of Undue Pain and Surfing: Using Hierarchical Criteria to Assess Website Quality.”

⁴¹ Wyatt, J. C. “Measuring the Quality and Impact of the World Wide Web.”

⁴² Kaicker, J., V. B. Debono, W. Dang, N. Buckley, and L. Thabane. “Assessment of the Quality and Variability of Health Information on Chronic Pain Websites Using the DISCERN Instrument.”

⁴³ Currie, C., P. D. Mambro, A. Joice, R. McGilp, M. O’Neill, G. Ralston, R. Davidson, and J. Scot. “Evaluating the Quality of Educational Materials about Schizophrenia.”

⁴⁴ Joshi, A., R. S. Bhangoo, and K. Kumar. “Quality of Nutrition Related Information on the Internet for Osteoporosis Patients: A Critical Review.” Technology and Health Care 19, no. 6 (2011): 1–10.

⁴⁵ Agency for Healthcare Research and Quality (AHRQ). “Assessing the Quality of Internet Health Information.” 1999. Available at http://www.ahrq.gov/data/infoqual.htm (accessed August 21, 2011).

⁴⁶ Lorence, D., and J. Abraham. “A Study of Undue Pain and Surfing: Using Hierarchical Criteria to Assess Website Quality.”

⁴⁷ Rees, C.E.. Ford, J.E. Sheard, C.E. Evaluating the Reliability of DISCERN: a Tool for Assessing the Quality of Written Ppatient Informationon Treatment Choices.

⁴⁸ Ream, E., E. Blows, K. Scanlon, and A. Richardson. “An Investigation of the Quality of Breast Cancer Information Provided on the Internet by Voluntary Organisations in Great Britain.” Patient Education and Counselling 76, no. 1 (2009): 10–15.

⁴⁹ Bhavsar, A. T., D. S. Shepard, J. A. Suaya, M. Mafowosofo, C. L. Hurley, and M. W. Howard. “A Private Hospital-based Study Assessing Knowledge, Attitudes, Practices and Costs Associated with Dengue Illness in Surat, India.”

⁵⁰ World Health Organization. “Regional Guidelines on Dengue/DHF Prevention and Control.”

⁵¹ Winker, M. A., A. Flanagin, B. Chi-Lum, J. White, K. Andrews, R. L. Kennett, C. D. DeAngelis, and R. A. Musacchio. “Guidelines for Medical and Health Information Sites on the Internet: Principles Governing AMA Web Sites.” Journal of the American Medical Association 283, no. 12 (2000): 1600–1606.

⁵² Tram, T. T., N. T. H. Anh, N. T. Hung, N. T. Lan, L. T. Cam, N. P. Chuong, L. Tri, L. Fonsmark, L. Poulsen, and E. D. Heegaard. “The Impact of Health Education on Mother’s Knowledge, Attitude and Practice (KAP) of Dengue Haemorrhagic Fever.” Dengue Bulletin 27 (2003): 174–180.

⁵³ Eysenbach, G., and C. Kohler. “How Do Consumers Search for and Appraise Health Information on the World Wide Web? Qualitative Study Using Focus Groups, Usability Tests and In-depth Interviews.” British Medical Journal 324 (2002): 573–77.

⁵⁴ Maloney, S., D. Llic, and S. Green. “Accessibility, Nature and Quality of Health Information on the Internet: A Survey on Osteoarthritis.” Rheumatology 44, no. 3 (2004): 382–85.

⁵⁵ Purcell, G. P., P. Wilson, and T. Delamothe. “The Quality of Health Information on the Internet: As for Any Other Medium It Varies Widely; Regulation Is Not the Answer.” British Medical Journal 324, no. 7337 (2002): 557–58.

⁵⁶ Maloney, S., D. Llic, and S. Green. “Accessibility, Nature and Quality of Health Information on the Internet: A Survey on Osteoarthritis.”

⁵⁷ University of Oxford and the British Library. DISCERN Handbook: Quality Criteria for Consumer Health Information on Treatment Choices.

⁵⁸ Newman, M. A., S. Ziebland, and K. L. Barker. “Patients’ Views of a Multimedia Resource Featuring Experiences of Rheumatoid Arthritis: Pilot Evaluation of www.healthtalkonline.org.”

⁵⁹ Craigie, M., B. Loader, R. Burrows, and S. Muncer. “Reliability of Health Information on the Internet: An Examination of Experts’ Ratings.” Journal of Medical Internet Research 4, no. 1 (2002): e2.

⁶⁰ Cline RJ, Haynes KM. Consumer Health Information Seeking on the Internet: the State of the Art. Health Education Research. 2001 Dec;16(6):671-92.

⁶¹ Frémont, P., M. Labrecque, F. Légaré, L. Baillargeon, and L. Misson. “Evaluation of Medical Web Sites: Interobserver and Intraobserver Reliability of an Evaluation Tool.” Canadian Family Physician 47, no. 11 (2001): 2270–78.

⁶² Theodosiou, L., and J. Green. “Emerging Challenges in Using Health Information from the Internet.” Advances in Psychiatric Treatment 9 (2003): 387–96.

⁶³ University of Oxford and the British Library. DISCERN Handbook: Quality Criteria for Consumer Health Information on Treatment Choices.

⁶⁴ A. R. Jadad, and A. Gagliardi. “Rating Health Information on the Internet: Navigating to Knowledge or to Babel?” Journal of the American Medical Association 279, no. 8 (1998): 611–14.