Media-savvy scientific literacy: developing critical evaluation skills by investigating scientific claims.
Abstract: Students must learn content knowledge and develop scientific literacy skills to evaluate and use scientific information in real-world situations. Recognizing the accessibility of scientific information to the average citizen, we developed an instructional approach to help students learn how to judge the quality of claims. We describe a project-based applied learning (PAL) approach that utilizes engaging questions about biological issues relevant to students. Working through these projects, students are challenged to evaluate sources of information and communicate their understanding of scientific claims. We discuss challenges that students encounter and offer suggestions for enacting this approach in a general-education college classroom.

Key Words: Investigating scientific claims; project-based learning; enhancing student learning; scientific literacy; argumentation; STEM education reform.
Article Type: Report
Subject: Sciences education (Technology application)
Learning (Research)
Biological research (Forecasts and trends)
Biological research (Technology application)
Biology, Experimental (Forecasts and trends)
Biology, Experimental (Technology application)
Technological literacy (Research)
Authors: Brickman, Peggy
Gormally, Cara
Francom, Greg
Jardeleza, Sarah E.
Schutte, Virginia G.W.
Jordan, Carly
Kanizay, Lisa
Pub Date: 08/01/2012
Publication: Name: The American Biology Teacher Publisher: National Association of Biology Teachers Audience: Academic; Professional Format: Magazine/Journal Subject: Biological sciences; Education Copyright: COPYRIGHT 2012 National Association of Biology Teachers ISSN: 0002-7685
Issue: Date: August, 2012 Source Volume: 74 Source Issue: 6
Topic: Event Code: 310 Science & research; 010 Forecasts, trends, outlooks Computer Subject: Technology application; Market trend/market analysis
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 298172414
Full Text: In the past few decades, there has been enormous effort to advocate for science curricula that prepare adults for navigating our scientifically and technologically complex society (AAAS, 1990, 2010; Bybee, 1993; Millar et al., 1998). From deciding on a course of action for medical therapies, to understanding the relative risks and benefits of vaccination, to discriminating among various dietary plans, adults are bombarded with contradictory claims. Many of our current science curricula fail to prepare adults to evaluate the quality of sources of scientific information and weigh evidence to make informed decisions about scientific claims (Bramble & Workman, 2007; Bray Speth et al., 2010; Osborne, 2010; Porter et al., 2010). Faculty would like to see a greater effort put toward teaching these skills (Coil et al., 2010; Brewer & Smith, 2011). With the Internet and other media increasingly serving as the major source of scientific information for scientists and the public (Fox & Jones, 2009), information literacy skills must comprise a critical component of instruction (Porter et al., 2010). However, most science courses do not include instruction on evaluating sources, and without this ability, students acquire erroneous conceptions and fail to improve their understanding of scientific issues (Wiley et al., 2009).

In response to the evident need for curricula specifically designed to focus on the scientific literacy involved in evaluating source quality and weighing evidence, we created an approach called project-based applied learning (PAL). This curriculum was specifically designed for students fulfilling their general education (Gen Ed) science requirement, and as such was designed to nurture habits of thinking and emphasize connections to everyday life (Meinwald & Hildebrand, 2010). Each project revolves around authentic life situations that provide opportunities for adult students to develop source-evaluation and argumentation skills while simultaneously engaging their interest. The guiding principle for each project is to allow students to focus on what the National Science Education Standards advocate as using "scientific information to make choices that arise every day," "engaging intellectually in public discourse and debate," and "sharing in the excitement and personal fulfillment that can come from understanding and learning about the natural world" (National Research Council, 1996). We discuss the challenges encountered and make suggestions for enacting PAL curricula in the often large-enrollment Gen Ed college classroom.

* Project-Based Applied Learning (PAL): An Approach to Develop Skills in Scientific Inquiry & the Nature of Science

We chose to adopt a project-based science instructional approach because of its demonstrated potential to build students' content knowledge as well their skills (Krajcik & Blumenfeld, 2006; Cook, 2009) while also providing an opportunity to integrate information resources (Land & Greene, 2000). In project-based instruction, students choose a project of interest and engage in collaborative investigation about a compelling problem. Classroom time is spent supporting students' ability to build arguments (Belland et al., 2008) as well as evaluating others' scientific arguments (Novak et al., 2009). The projects require students to seek sources of information, interpret quantitative data concerning the problem, draw conclusions, and communicate the results, in a way that mimics how science is conducted (Colley, 2008; Krajcik et al., 2008). In addition, students confront discrepancies in data interpretation, methods, and diversity of approaches to solving problems that provide a more complex view of the nature of science (Bybee, 2006). This is a constructivist learning approach: students reevaluate their previous knowledge during the process of addressing the problem, which helps them consolidate their existing knowledge and elaborate further (von Aufschnaiter et al., 2008). An outline of instructional steps is presented in Figure 1. Complete course materials for each project can also be downloaded for free at http://www.pal.uga.edu.

[FIGURE 1 OMITTED]

* Compelling Problems

Each PAL project begins by engaging students in an example problem that demonstrates how they can use biology content in their lives. For example, the PAL project "Worst Food in America" is first introduced to students by showing them a graphic from an online article highlighting a particularly unhealthy "Worst Supermarket Food." Students are asked to read the graphic presented and brainstorm about what they found convincing, and what information they would need to know to improve the argument and sources of evidence used in the graphic. Their assignment is to use the information studied in class to create their own scientifically sound graphic that convinces someone of the unhealthy nature of a food (Figure 2). The "Worst Snack Food" example that students were shown, like many media pieces, contains little supporting evidence, so students are reminded that they need to generate either a graph or figure in their project that includes a figure legend stating their claim as well as an interpretation of the evidence described (i.e., their reasoning through a valid scientific argument to explain whether the evidence supports the claim--a warrant statement). They must also provide references and descriptions of how they concluded whether the sources that they used were reliable. The students are encouraged to communicate creatively, and their graphics often include compelling applications of the content (Figure 3).

* Classroom Content & Skill Development

After introducing the problem, students embark on two or three classroom sessions involving lecture and student-active exercises. Concurrently, students work on their projects outside of class, using what they learn as the foundation for their investigation (Alozie et al., 2010). Short readings from a non-science-majors textbook, followed by pre-class online reading quizzes to guide students' mastery of the content, are assigned before each class. These quizzes test for definitional knowledge of terms and ability to classify terms from their reading. For example, for the "Worst Food in America" project, students classify molecules such as sucrose and fructose as simple sugars, starch and cellulose as complex carbohydrates, and fats and oils as triglycerides. This instructional format differs from most biology courses, in which instructors spend class time lecturing students on content, and has been shown to increase student learning compared to a course with more lecture during the class period (Moravec et al., 2010). In PAL, class time is spent on applied activities, such as creating an organization matrix of all the food molecules, including functions; identifying ingredients from a food label; or calculating relative contributions of calories from foods and determining how this relates to daily recommendations. Student responses to these activities are collected on worksheets or by clicker questions and are accompanied by small group discussion in a method proven to be effective for learning (Smith et al., 2009, 2011; Bennett et al., 2010).

* Out-of-Class Investigation of Scientific Claims, Construction of Products, & Evaluation

Students work individually outside of class to find sources that support their claim. As a group in class, they evaluate the sources, reach a consensus, and assemble the best evidence for their claim as well as a graphic that communicates their findings. The process of giving and receiving feedback on their product rough drafts is one of the most important opportunities for improving students' ability to both evaluate the quality of data and recognize flaws in scientific arguments. This process can be done online, but students often need personal consulting sessions that provide opportunities for the instructor to reinforce the elements of argumentation (claim, evidence, and warrants; Toulmin, 1969). Students often need redirection to the suggested starting points and search terms for improving their Internet searches to avoid their almost exclusive reliance on Google or Wikipedia.

[FIGURE 3 OMITTED]

Final products are reviewed by instructors and peers, utilizing an online course management tool for grading rubrics that indicate three levels with detailed explanations for each rubric category: meets expectations, progressing, and needs improvement. Students earn a small number of points for providing this peer critique. Final project grades take into account suggested peer scores for each project. For example, the instructor first grades the project and then views a summary of all peer scores. If the instructor and peer scores differ, the instructor considers altering the score to reflect a consensus, but always provides the final assessment. Peer scores help the instructor view student perspectives in the area of understanding and relevance as well as assess how well students are progressing in their own ability to evaluate the quality of sources and the explanations of figures. Peer review has been shown to benefit the assessor (Li et al., 2010). Also, providing a student audience reinforces the project's focus on communication to the general public rather than solely to the instructor. Students choose which of the five content areas they wish to investigate for these projects (Figure 4).

* Student Responses to PAL

During the second year of the PAL curriculum, a survey was administered to students after each project. Survey questions focused on students' self direction and engagement during the projects. Students judged the PAL projects as realistic and relevant because they felt that the projects connected to their life experiences. Students reported that the information-seeking and evaluation activities, leading to making claims, played an important role in the projects and were realistic to everyday life. One student discussed the important role that scientific information could play in daily life: "There is always some pop science story in the news about how a popular food is bad for you, I wanted to learn how to investigate for myself." Another student discussed the realistic nature of seeking scientific information: "Throughout our lives we are going to have to research different things and decide whether or not they are valid and what products could potentially put us or our loved ones at risk."

Students indicated that the information seeking and evaluation activities, as well as group management processes featured in the projects, required them to take charge of their own learning experience. However, some students indicated that they were not prepared to take such a central role in their learning process. As one student mentioned, "I feel like this class in general is mainly about learning on our own. This is really scary for me because science is not my strongest area and I respond much better to more direction and structure." However, this same student also discussed some positive aspects of this type of learning: "For these projects, we are responsible for the research and the organization of the results, which makes it vital for us to teach ourselves and exercise our critical thinking skills as adults." Another student reported feelings about taking charge of the learning experience: "The project was very self-directed. Yes, there was a rubric, but it was really your own ideas and claims. No one forced you into boring subjects for research." In general, students felt that the projects required them to take charge of their own learning processes, which was challenging for some students, but also rewarding.

Challenging students--expecting them to study, having high expectations, and introducing courses with substantial reading and writing, are all associated with growth in critical-thinking skills during their first 2 years of college (Roksa & Arum, 2011). This should be the goal of Gen Ed courses. Freedom of choice can help sustain students' interest in a topic (Falk et al., 2007), and relating science to their own lives is one of the major factors in choosing to study a particular science (e.g., biology vs. chemistry; Osborne et al., 2003).

The PAL approach provides a way to engage students in solving compelling problems that foster the development of scientific literacy skills (e.g., finding and evaluating the quality of information sources and communicating a scientific argument). Instructors should be aware, however, that students often resist extra work that is required of them (Loughran & Derry, 1997). Moreover, students, unused to learning in this way, may indicate a preference for assessment by examination (Watters & Watters, 2007). These projects are not primarily designed just to introduce content, but rather to challenge students to develop scientific literacy skills to utilize that content understanding. Students discovering gaps in ability, developing increased awareness of the difficulty in weighing pieces of evidence and multiple interpretations, and developing skepticism are all part of science and should hold an important place in the Gen Ed science classroom (Germann & Aram, 1996; Crawford et al., 2000; Seethaler, 2005; Coil et al., 2010). Thus, introducing these projects and their complexities provides a way to develop skills that are often overlooked in college classrooms.

[FIGURE 4 OMITTED]

DOI: 10.1525/abt.2012.74.6.4

References

AAAS. (1990). Science for All Americans. New York, NY: Oxford University Press.

AAAS. (2010). Vision and Change: A Call to Action. Washington, D.C.: American Association for the Advancement of Science.

Alozie, N., Eklund, J., Rogat, A. & Krajcik, J. (2010). Genetics in the 21st century: the benefits & challenges of incorporating a project-based genetics unit in biology classrooms. American Biology Teacher, 72, 225 -230.

Belland, B.R., Glazewski, K.D. & Richardson, J.C. (2008). A scaffolding framework to support the construction of evidence-based arguments among middle school students. Educational Technology Research and Development, 56, 401-422.

Bennett, J., Hogarth, S., Lubben, F., Campbell, B. & Robinson, A. (2010). Talking science: the research evidence on the use of small group discussions in science teaching. International Journal of Science Education, 32, 69-95.

Bramble, J. & Workman, M. (2007). Data-rich case studies improve students' abilities to interpret graphs in a large non-majors course. Teaching Issues and Experiments in Ecology, 5. [Online.] Available at http://tiee. esa.org/vol/v5/research/bramble/article.html.

Bray Speth, E., Momsen, J.L., Moyerbrailean, Ebert-May, D., Long, T.M., Wyse, S. & Linton, D. (2010). 1, 2, 3, 4: infusing quantitative literacy into introductory biology. CBE Life Sciences Education, 9, 323-332.

Brewer, C. & Smith, D.C., Eds. (2011). Vision and Change in Undergraduate Biology: A Call to Action. Washington, D.C.: American Association for the Advancement of Science.

Bybee, R.W. (1993). Reforming Science Education: Social Perspectives and Personal Reflections. New York, NY: Teachers College Press.

Bybee, R.W. (2006). Scientific inquiry and science teaching. In L.B. Flick & N.G. Lederman (Eds.), Scientific Inquiry and Nature of Science: Implications for Teachinf, Learning, and Teacher Education, pp. 1-14. Dordrecht, The Netherlands: Springer.

Coil, D., Wenderoth, M.P., Cunningham, M. & Dirks, C. (2010). Teaching the process of science: faculty perceptions and an effective methodology. CBE Life Sciences Education, 9, 524-535.

Colley, K. (2008). Project-based science instruction: a primer--an introduction and learning cycle for implementing project-based science. Science Teacher, 75(8), 23-28.

Cook, K. (2009). A suggested project-based evolution unit for high school: teaching content through application. American Biology Teacher, 71, 95-98.

Crawford, T., Kelly, G.J. & Brown, C. (2000). Ways of knowing beyond facts and laws of science: an ethnographic investigation of student engagement in scientific practices. Journal of Research in Science Teaching, 37, 237-258.

Falk, J.H., Storksdieck, M. & Dierking, L.D. (2007). Investigating public science interest and understanding: evidence for the importance of free-choice learning. Public Understanding of Science, 16, 455-469.

Fox, S. & Jones, S. (2009). The Social Life of Health Information. Pew Internet & American Life Project.

Germann, P.J. & Aram, R.J. (1996). Student performances on the science processes of recording data, analyzing data, drawing conclusions, and providing evidence. Journal of Research in Science Teaching, 33, 773-798.

Krajcik, J. & Blumenfeld, P. (2006). Project-based Learning. In R.K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences, pp. 317-334. New York, NY: Cambridge University Press.

Krajcik, J., McNeill, K.L. & Reiser, B.J. (2008). Learning-goals-driven design model: developing curriculum materials that align with national standards and incorporate project-based pedagogy. Science Education, 92, 1-32.

Land, S.M. & Greene, B.A. (2000). Project-based learning with the World Wide Web: a qualitative study of resource integration. Educational Technology Research and Development, 48, 45-67.

Li, L., Liu, X. & Steckelberg, A.L. (2010). Assessor or assessee: how student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41, 525-536.

Loughran, J. & Derry, N. (1997). Researching teaching for understanding: the students' perspective. International Journal of Science Education, 19, 925-938.

Meinwald, J. & Hildebrand, J.G., Eds. (2010). Science and the Educated American: A Core Component of Liberal Education. Cambridge, MA: American Academy of Arts and Sciences.

Millar, R., Osborne, J. & Nott, N. (1998). Science education for the future. School Science Review, 80(291), 19-24.

Moravec, M., Williams, A., Aguilar-Rocan, N. & O'Dowd, D.K. (2010). Learn before lecture: a strategy that improves learning outcomes in a large introductory biology class. CBE Life Sciences Education, 9, 473-481.

National Research Council. (1996). National Science Education Standards. National Academy Press, Washington, D.C.

Novak, A.M., McNeill, K.L. & Krajcik, J. (2009). Helping students write scientific explanations. Science Scope, 33, 54-56.

Osborne, J. (2010). Arguing to learn in science: the role of collaborative, critical discourse. Science, 328, 463-466.

Osborne, J., Simon, S. & Collins, S. (2003). Attitudes towards science: a review of the literature and its implications. International Journal of Science Education, 25, 1049-1079.

Porter, J.A., Wolbach, K.C., Purzycki, C.B., Bowman, L.A., Agbada, E. & Mostrom, A.M. (2010). Integration of information and scientific literacy: promoting literacy in undergraduates. CBE Life Sciences Education, 9, 536-542.

Roksa, J. & Arum, R. (2011). The state of undergraduate learning. Change: The Magazine of Higher Learning, 43(2), 35-38.

Seethaler, S. (2005). Helping students make links through science controversy. American Biology Teacher, 67, 265.

Smith, M.K., Wood, W.B., Adams, W.K., Wieman, C., Knight, J.K., Guild, N. & Su, T.T. (2009). Why peer discussion improves student performance on in-class concept questions. Science, 323, 122-124.

Smith, M.K., Wood, W.B., Krauter, K. & Knight, J.K. (2011). Combining peer discussion with instructor explanation increases student learning from in-class concept questions. CBE Life Sciences Education, 10, 55-63.

Toulmin, S. (1969). The Uses of Argument. Cambridge, United Kingdom: Cambridge University Press.

von Aufschnaiter, C., Erduran, S., Osborne, J. & Simon, S. (2008). Arguing to learn and learning to argue: case studies of how students' argumentation relates to their scientific knowledge. Journal of Research in Science Teaching, 45, 101-131.

Watters, D.J. & Watters, J.J. (2007). Approaches to learning by students in the biological sciences: implications for teaching. International Journal of Science Education, 29, 19-43.

Wiley, J., Goldman, S.R., Graesser, A.C., Sanchez, S.A., Ask, I.K. & Hemmerich, J.A. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46, 1060-1106.

PEGGY BRICKMAN is Associate Professor of Plant Biology at University of Georgia, Athens, GA 30602; e-mail:brickman@uga.edu. CARA GORMALLY is Director of Introductory Biology Labs at the Georgia Institute of Technology School of Biology, 310 Ferst Drive NW, Atlanta, GA 30332; e-mail: cara.gormally@biology.gatech.edu. GREG FRANCOM is Assistant Professor of E-Learning at Northern State University, 1323 S. Lincoln Street, Aberdeen, SD 57401; e-mail: greg.francom@gmail.com. SARAH E. JARDELEZA is a Research Associate in Plant Biology at Michigan State University, 166, East Lansing, MI 48824; e-mail: sarahejw@msu.edu. VIRGINIA G. W. SCHUTTE is a Ph.D. candidate in Ecology at the University of Georgia, 140 E. Green Street, Athens, GA 30602; e-mail: vschutte@uga.edu. CARLY JORDAN is Assistant Academic Professional Lecturer at the University of Wyoming, 1000 E. University Avenue, Laramie, Wyoming 82071; e-mail: cnjordan81@gmail.com. LISA KANIZAY is Postdoctoral Research Fellow of Plant Biology at the University of Georgia, 2502 Miller Plant Sciences, Athens, GA 30602; e-mail: lkanizay@plantbio.uga.edu.
Figure 2. Example project assignment: "Worst Food in America."

You will be creating a graphic to convince someone of the unhealthy
nature of a food item. The "Worst Supermarket Food" graphic from
Women's Health Magazine that was shown in class is one example, but
you are not limited to that format. Remember, you will need to find
what you think is either a healthy or unhealthy food item and also
find supporting evidence for your argument, including claim,
evidence, and warrant.

The following is a list of great Internet sites where you can get
information on nutritional content.

This website introduces you to the food label:
http://www.healthcare.uiowa.edu/fns/nutritional/foodlabel.htm

These sites can produce nutrition information for many foods:

* http://www.nutrientfacts.com/ (basic foods.)

* http://caloriecount.about.com

* http://www.fatcalories.com/ (just fast food restaurants.)

* http://www.calorieking.com/foods/
calories-in-fast-food-chains-restaurants_cY2lkPTIx.html?bid=770&sid=-l
(fast food items)

Don't forget, you will also have to find valid sources to back up
your claim that a specific food item is unhealthy.

PROJECT RUBRIC (100 POINTS)

Project Title: Expresses a clearly stated claim that is evaluated
in the project. (4 points)

Understanding and Relevance: Scientific concepts and terminology
are concisely defined; enough detail and background are provided.
Content should be targeted to your peers as an audience. It should
be meaningful, relevant, and presented in the context of larger
community issues and societal ethics (e.g., harmful or beneficial
to humans). (20 points)

Evidence: Figures/graphs and references: Evidence to support the
claim is described/provided. Figures and graphs are used to educate
and persuade. Quality and validity of scientific information
sources are evaluated. Be sure to annotate the importance of each
reference in supporting your argument, and describe how you
evaluated its quality. You must reference your sources with
literature citations. (40 points)

Synthesis: Figure legend provides reasoning through a valid
scientific argument to explain whether the evidence supports the
claim. Make a recommendation or conclusion based on your reasoning.
(20 points)

Team Reflection: (12 points) Answers the following questions:

(1) What issues did you encounter through the process of creating
this project (e.g., while making graphs and finding resources)?

(2) How did you weigh opposing evidence when developing your
recommendation?

(3) How do you anticipate using the skills you mastered doing this
project in your life?

Acknowledgments: Describes team members' contributions in terms of
project components and activities required to complete the project.
(4 points)
Gale Copyright: Copyright 2012 Gale, Cengage Learning. All rights reserved.