Research brief: designing a student centered approach to developing competent health information literacy skills.
Abstract: Little is known about health topics that interest college students, or what online resources are used to obtain answers. The aim of this study was to identify student centered information to promote online health information literacy skills. The objectives were to explore the health issues of interest, document what web sites addressed their interest, and evaluate the quality of these websites. Methods included a health web site evaluation tool. Results indicated that interests were varied, with most health information coming from .gov websites. Study results identified limitations in student health information literacy skills. Teaching strategies that address CHES Responsibility VI A are presented.
Article Type: Clinical report
Subject: Universities and colleges (Health aspects)
Online health care information services (Health aspects)
Authors: Putnam, Janice
Kitts, Royce
Pulcher, Karen L.
Pub Date: 09/22/2010
Publication: Name: American Journal of Health Studies Publisher: American Journal of Health Studies Audience: Professional Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2010 American Journal of Health Studies ISSN: 1090-0500
Issue: Date: Fall, 2010 Source Volume: 25 Source Issue: 4
Topic: Computer Subject: Online health care service
Product: Product Code: 8220000 Colleges & Universities NAICS Code: 61131 Colleges, Universities, and Professional Schools SIC Code: 8221 Colleges and universities
Accession Number: 309069164

The National Assessment of Adult Literacy defines literacy as "the ability to use printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential" (White & McCloskey, 2003). In its narrowest sense, information literacy includes the practical skills involved in effective use of information technology using print or electronic information resources.

Information literacy is a set of abilities requiring individuals to "recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information (American Library Association, 2006)." According to the Association of College and Research Libraries (2000) an "information literate individual" is able to determine the extent of information needed; access the needed information effectively and efficiently; evaluate information and its sources critically; incorporate selected information into one's knowledge base; use information effectively to accomplish a specific purpose; understand the economic, legal, and social issues surrounding the use of information; and access and use information ethically and legally.

In focusing specifically on information literacy related to health, the Medical Library Association (2003) offered the following definition of health information literacy, health information literacy is the set of abilities needed to recognize a need for health information, identify likely information sources and use them to retrieve relevant information, assess the quality of the information and its applicability to a specific situation, and analyze, understand, and use the information to make good health decisions. The Institute of Medicine (2004) similarly defined health literacy as "the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions." A health communication objective set by Healthy People 2010 (2001) added that public health professionals should be assisting consumers to seek health information via the internet and at the same time reinforce standards for quality and evaluation. Furthermore, the Responsibilities and Competencies for Health Educators discuss the importance of being an information resource (Responsibilities and Competencies for Health Educators, n.d.). Specifically, Responsibility VI Acting as a Resource Person in Health Education, competency A states that health educators need to be able to utilize computerized health information retrieval systems effectively. Competency C states that they needs to interpret and respond to requests for health information. Finally, Competency D says the health educators should be able to select effective educational resource materials for dissemination.

The internet is being used at ever increasing frequency by college students to obtain health information. The Pew Internet and American Life Project Report: (2002) reported that 85% of college students have their own computer and are twice more likely to have used the internet than all other users. Specific to health information, Escoffery (2005) showed that 74% of college students have searched the internet to obtain health information online, and 40% report that they frequently do so.

Using the internet to search for health information is not without its problems. Common user end issues with health information websites include access, consumption, and interpretation (Action for Health, 2007). Pajarillo (2001) added that other problematic aspects of health information websites may include the ease and use of retrieval systems that may be counter intuitive for student use thereby not providing correct useful valid information." An ideal retrieval system is most appreciated and effective for (users) who have limited knowledge and skills in handling information search systems", (Pajarillo, 2001).

Health information literacy skills can be operationally defined to include items such as searching and screening, integrating, analyzing, applying, and presenting (Ku, Sheu, Kuo, 2007). Appropriate website selection and evaluation is the initial step in developing health information literacy skills and involve searching and screening skills. There are many tools and guidelines available for use in evaluating websites. Barker and Terry (2005) indicate that the following areas should be considered in the evaluation of online resources: organization, authority, objectivity/authority, scope, and currency. Goldschimdt (2003) recommend using the following criteria when obtaining medical information from medical websites : 1) site ownership; 2) sponsorship and interest statements; 3) organization and ease of navigation; 4) currency of information; 5) mission, purpose and scope; 6) intended audience; 7) author credentialing; 8) references used; 9) editorial procedures; 10) links to other health sources 11) disclaimers, privacy and advertising policies.

Health Information literacy skills in the college student are at the novice level with typical usage of the internet for basic information. According to Ivanitaskaya, O'Boyle, and Casey (2006), student ability to accurately report their search skills only weakly correlates to their actual skill level (Cronbach alpha =.78). By self report, few students were able to narrowly conduct their online search and only 50% were able to evaluate the trustworthy features of a site. Up to half of the students were unsure if a reference was needed or how to reference the site for paraphrased sentences.

There are many studies available about college students and information literacy, but not about college students and health information literacy. Therefore, our question was, how are college students applying health information literacy in an online environment? There is a paucity of the literature addressing health information literacy from a student perspective. From the literature review, this study appears to be original research that has not been conducted, cited in, or utilized by others.


The Prevent Google Whacking & Foster Cyber Wellness Nursing Education Model (Pulcher & Putnam, 2007) was used to guide the thinking of the investigators during the design, implementation and interpretation of the results of this study. This model views technologic interactions as complex phenomena that are congruent with a pluralistic world view. This world view is often used in nursing education and embraces both the positivistic and humanistic paradigms. The model is composed of an open technologic environmental system that encompasses social network, technical learning support, peer collaboration, and faculty mentoring. Each of these four components exists on a continuum of novice to expert skill acquisition converging as expert status in the epicenter. Stakeholder skill level has the most important implications for educational intervention and was our area of interest in this study. It is theoretically assumed that skill acquisition from novice to expert can be addressed through structured teaching and learning strategies and will be discussed later in this article.

Prior to initiation of data collection in this study, a pilot survey was conducted. Undergraduate students (N = 22) who where enrolled in an Honor's Colloquium class were surveyed to determine their interest in online health information. This pilot questionnaire provided validation of the study questions. It assured the appropriateness of research in this population for the purpose of gaining the information needed to answer the research question.


The aim of this study was to identify student centered information that educators may use to promote online health information literacy skills. Our goals were to: 1. to better understand health issues in which college students are interested, 2. to document and identify web site types that were accessed by students and 3. to document the quality of web sites visited by students.


This study was conducted in a small Midwest university. Following IRB approval, study participants included students enrolled in the university's Honor's College. Criteria for enrollment in the course were a cumulative GPA of at least 3.5 and ACT score of at least 25. The nonrandomized sample included 42 junior and senior college students from two consecutive semesters (spring semester n = 19. fall semester n = 23).


The Health Related Web Site Evaluation Form (Robbins, 1998) is a 36 question instrument. Documented reliability or validity data could not be identified for this tool. This new instrument contained the components identified through our literature review to provide a best practice approach to evaluating website quality engaging students in searching and screening skills. The instrument is divided into nine sections. These sections included Web site information, content, accuracy, author, currency, audience, navigation, external links and structure. Each question is scored with points: disagree (1), agree (2), or not applicable (0). The points scored and points possible are totaled. The total percentile score can then be calculated to determine the overall rating of the web site: >90% -Excellent. Indicating a score of 33 or greater suggests that this website is an excellent source of online health information. The information is easily accessed and understood. > 7589% Adequate: Indicating a score of 27-32. While this web site provides relevant information and can be navigated without much trouble, it might not be the best site available. If another source cannot be located, this site will provide good information. < 74% Poor: Score of 26.64 or less is not recommended for health information. Validity and reliability of the information can not be confirmed. Another site should be used to prevent false or partial information from being used or read.


Using a traditional lecture format, the e-Health Literacy Skill teaching learning strategy for this study was adapted from Elfrink and colleagues (2001). This included: 1) the need for e-health information was introduced; 2) a searchable question was identified; 3 information retrieval, synthesis and evaluation methods were reviewed; resulting in 4) understanding the information applicability and evaluation of the results.

After completing the lecture component of the e-health literacy unit, the participants' were given the Health Related Web Site Evaluation tool with instructions about how to use it. Students were sent to an open computer lab to develop a health information question of their choice and to browse online for an applicable website. Each student was responsible for choosing their own website and the evaluation. Students were monitored by the instructor while they completed the course assignment and the evaluations. All data were collected at the end of class. These data were secured in a locked cabinet by the instructor. There were no attempts to match the surveys with specific students.


Post review of the data, 2 evaluations were excluded due to missing/flawed evaluation data leaving an N = 41 sites. The overall web site evaluation scored was 88.9, an adequate evaluation according to the instrument. Twenty five of the sites evaluated produced an excellent score on the evaluation, twelve of the sites evaluated produced an adequate score on the evaluation, and four of the sites received a poor rating. Site evaluations took a total of 30 minutes of faculty and student time using direct faculty supervision. Results indicate that the Health Related Web Site Evaluation Form is a useful tool in developing student online health literacy from novice to expert. According to Nunnally and Bernstein (1994), it is encouraging to the researcher to find an instrument that produces interesting findings and is found to be utilitarian, even if it is new.


The web sites that were identified and accessed to address their interest were mostly .org or .gov. Our finding is consistent with Ivanitskaya and colleagues (2006) study who also found that the ease of access to what students perceive as reliable information may preclude them from narrowing a search or being able to differentiate credible from questionable websites. The notion of ease is further expounded by Crespo (2004) who found that users focused on the speed of retrieval rather than quality or depth of information. In addition, Eysenbach and Kohler (2002) demonstrated a lack of depth in their searches as indicated by only making use of the first few links obtained.

Overall, assessment of the majority of the sites selected was adequate to excellent. This finding lends support to the idea that .org and .gov sites are of adequate quality for health information searches. Topics of sites evaluated that were considered poor included those addressing the topics of genetic disorders, cloning, and multiple sclerosis.


Health and wellness awareness are content areas critical to 21st Century student success (Ullman, 2007). To address this content area, learning and thinking skills, information and communication technology skills and life skills have been identified. According to the Prevent Google Whacking & Foster Cyber Wellness Nursing Education Model (Pulcher & Putnam, 2007), it is theoretically assumed that skill acquisition from novice to expert can be addressed through structured teaching and learning strategies. This study provides preliminary information relevant to the initial skill acquisition from novice to expert can be addressed through structured teaching and learning strategies. This study provides preliminary information relevant to the initial health information literacy skill of searching and screening web sites. Since health information literacy also includes such skills as integrating, analyzing, applying and presenting information, further studies should be designed to address all levels of skill development.

Additionally, educators must consider alternative avenues through which online health literacy skills are acquired. Personal competency with navigating online health information is often obtained independently from academic environments such as Google,, and youtube. com. While entertaining, these sources are not valid reliable sources of information. However, they are intuitively easy to navigate with novice technological skills. Future research could include tracking and application of valid and reliable health information on these trendy, heavily trafficked popular Millennial age student sites. Academically, formative and summative assignments should be developed and studied to provide evidence based teaching learning strategies for health information literacy skill mastery. Further research endeavors should also include instrument validity and reliability development. Pre and post test online literacy skill assessment methods could be produced to define or outline competency development milestone achievement. Bibliographic curriculum review specific to education of online literacy competency strategies and meta-analyses can be done to further enhance and define institutional responsibilities.


Promoting health information literacy as a teaching learning strategy is a multifaceted problem solving approach to addressing health and wellness content. Collaborative projects may include the development of a hands-on problem solving assignment that combines known health topics of interest with health information literacy skill development and the application of bibliographic instruction. A framework for information literacy is strengthen when educator and bibliographers engage in joint ventures and will result in successful student knowledge engagement. Future projects can be realize by stretching student and faculty creativity using blogs, URL site development, pod casts and online environments such as Initial education strategies should focus on bibliographic instruction, teaching students on proper search techniques that can be generalized across a variety of online information platforms. Online learning management systems such as Blackboard, Web CT, and Sakai provide a unique interactive environment in which educators and librarians can customize bibliographic instruction to meet the needs of students (Jackson, 2007).

Limitations of this study include the size and scope of the project. This study explored the needs of a select group of traditional college students...those in an Honor's course. However, different needs of other populations should be addressed. The generalizability of this project is recognized well suited to a myriad of other disciplines seeking online literacy skill acquisition due to the ease of operation and application of this teaching learning strategy. The results of this study suggest that the students are good at picking good sites, but when it comes to finding the information they need, they may be drawn to ease of access. Therefore, with challenging topics needing more sophisticated searching skills additional levels of skill are required by both the student and the educator, We anticipate that this study will support a paradigm shift in education in order to meet the information needs of the Millennial students.


The authors would like to acknowledge R. Sue Lasiter PhD(c), RN for her assistance in editing this manuscript.


Action for Health (2007). Applied Communications Technology Information, Organizations, Networks Retrieved Sept 11, 2007 from

American Library Association and the Association for Educational Communications and Technology. (1998). Information literacy standards for student learning. In Information Power: Building Partnerships for Learning. Retrieved May 28, 2002 from

American library Association. (2006). Information Literacy Competency Standards for Higher Education. Retrieved February 14, 2007 from

American Library Association. (1989). Presidential Committee on Information Literacy: Final Report. Retrieved October 29, 2007 from

Association of College and Research Libraries. (2000). Information Literacy Competency Standards for Higher Education. Retrieved October 29, 2007 from: informationliteracycompetency.html#ildef

Barker, D. I., & Terry, C. D. (2005). Internet Research (2nd ed.). Boston: Course Technology.

Burnham, E., & Peterson, E.B. (2005). Health Information Literacy: a library case study. Library Trends, 53, 422-433.

Crespo, J. (2004). Training the health information seeker: Quality issues in health information websites. Library Trends, 53, 360-374.

Elfrink, V., Bakken, S., Coenen, A., McNeil, B., & Bickford, C. (2001). Standardizing nursing vocabularies: A foundation for quality care. Seminars in Oncology Nursing. 17(1), 18-23.

Escoffery, C., Miner, K. R., Adame, D., Butler, S., McCormick, L., Mendell, E. (2005). Internet use for health information among college students. Journal of American College Health. 53(4), 183-188.

Eysenbach, G., & Kohler, C. (2002). How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. British Medical Journal, 324, 573-577.

Goldschmidt, P. (2003). A Report on the Evaluation of Criteria Sets for Assessing Health Web Sites. A Report of the Health Information Institute and Consumer Health Reports. Retrieved October 19, 2007, from

Institute of Medicine. (2004). Health literacy: a prescription to end confusion. Washington, DC: National Academies Press.

Ivanitakaya, L., O'Boyle I., & Casey A. M. (2006). Health Information Literacy and Competencies of Information Age Students: Results from the Interactive Online Research Readiness Self-Assessment (RRSA). Journal of Medical Internet Research, 8(2). Retrieved October 29, 2007, from http://www.jmir. org/2006/2/e6/

Jackson, P. (2007). Integrating information literacy into Blackboard: Building campus partnerships for successful student learning. The Journal of Academic Librarianship, 33(4), 454-461.

Jones, S., & Madden, M. (2002). The internet goes to college: How students are living in the future with today's technology. The Pew Internet and American Life Report. Retrieved October 10, 2007 from The Pew Internet and American Life Project Web Site: asp

Ku, Y., Sheu, S., & Kuo, S. (2007) Efficacy of integrating information literacy education into a women's health course on information literacy for RN-BSN students. Journal of Nursing Research. Vol. 15(1).

Medical Library Association. (2003). Health Information Literacy. Retrieved July 23, 2003 from http://www.

Nunnally, J.. & Bernstein, I. (1994), Psychometric Theory(3rd ed.). McGraw Hill: New York.

Pajarillo, E. (2001). The use and evaluation of search databases by professional nurses: a case study. The Electronic Library, 19(5), 296-306.

Pulcher, K.. & Putnam, J. (2007). Prevent Google whacking & foster cyber wellness: An ethically based nursing education model. Unpublished manuscript.

Responsibilities and Competencies for Health Educators (n.d.). American Journal of Health Studies. Retrieved November 13, 2007 from:

Robbins, L. T. (1998). Health-Related Web Site Evaluation Form. Retrieved October 29, 2007 from http://

Thede, L. Q. (2003). Informatics and Nursing: Opportunities and Nursing.(2nd ed.). Philadelphia: Lippincott, Williams & Wilkins.

Ullman, E. (2007) Future Proofing Our Students: Preparing students to succeed in a global workforce. i.e. magazine, 3(3), 24-27.

United States Department of Health and Human Services. (2001).Healthy People 2010: Understanding and Improving Health.McLean, VA: International Medical Publishing, Inc.

Vacca, R.T. (2006). They can because they think they can. Educational Leadership, 63(5), 56-59. White, S., & McCloskey, M. Framework for the 2003. National Assessment of Literacy (NCES 2006-473). US Department of Education, Washington, DC

Janice Putnam PhD, RN

Royce Kitts

Karen L. Pulcher

Janice Putnam PhD, RN, is an Assistant Professor of Nursing, University Health Center Room 117, University of Central Missouri, phone: 1 660 543 8438, E-mail: Royce Kitts, is a Graduate Student, 1200 Commercial St., Emporia State University, phone: 1 785 539 1197, E-mail: rkitts@, Karen L. Pulcher, is an Assistant Professor of Nursing, North Kansas City, Kansas City, MO, University of Central Missouri. E-mail: phone: 816-802-2222
Table 1: e-Health Survey

Question                             Response--Yes   Percentage

Would you like to get your health    14               64%
information online?
Have you received health             14               64%
information online?
Have you searched for health         17               77%
information online?
Would you like to attend a health    8                36%
program online?

Table 2: Results of the health issues college students are
interested in defined by topic: (N of sites N = 41)

Interest area            Specific Topics

Illness:                 Cancer (6), heart disease and strokes
                         (4), diabetes (4), cystic fibrosis,
                         Ehlers Danlos, MS, lupus, Alzheimer's
Wellness or cosmetics:   Body building, side effects of steroids,
                         dental implants, pregnancy, allergies,
                         women's health, children's health,
                         feeding your immunity, colds, teen health,
                         psychology advice, drug use/abuse
Current Topics           Cloning, genetic disorders, avian flu, e
  in the News:           coli, stem cell research, e-health,
                         electronic medical records
Miscellaneous:           heel pain, pseudocholinesterase deficiency

Table 3: Results of the URL types of web sites were
identified and accessed to address their interest
included: (N of sites N = 41)

URL Type        Number of sites   Percentage

.org            23                53%
.com            10                23%
.gov or .us     8                 19%
.edu or .info   2                 5%
Gale Copyright: Copyright 2010 Gale, Cengage Learning. All rights reserved.