The relative effects of traditional lectures and guided notes lectures on university student test scores.
Abstract: Guided notes were employed in two undergraduate Psychology courses involving 71 students. The study design utilized an alternating treatments format to compare Traditional Lectures with Guided Notes lectures. In one of the two courses, tests were administered after each class lecture, whereas the same type of test was administered at the beginning of the next week's class for the second course. Regardless of test delay, the Guided Notes condition was associated with substantially higher average test performance in all courses (approximately a grade level or more). Specific implications of the present applications and outcomes as well as future research needs are discussed.

Keywords

Guided notes, active student responding, behavioral instruction
Article Type: Report
Subject: College students (Testing)
Lectures and lecturing (Research)
Lectures and lecturing (Comparative analysis)
Authors: Williams, W. Larry
Weil, Timothy M.
Porter, James C.K.
Pub Date: 01/01/2012
Publication: Name: The Behavior Analyst Today Publisher: Behavior Analyst Online Audience: Academic Format: Magazine/Journal Subject: Psychology and mental health Copyright: COPYRIGHT 2012 Behavior Analyst Online ISSN: 1539-4352
Issue: Date: Wntr, 2012 Source Volume: 13 Source Issue: 1
Topic: Event Code: 330 Product information; 310 Science & research
Product: Product Code: E197500 Students, College
Accession Number: 292504171
Full Text: Pedagogy has been a focus of behavioral research and discussion for decades (Engleman & Carnine, 1982; Keller, 1968; Lindsely, 1996; Skinner, 1968). The focus of behavioral teaching strategies is to deem student deficits as a problem of the teaching environment, rather than a shortcoming of the student. An environmental factor of considerable interest in behavioral teaching strategies is active student responding (Austin, 2000; Barbetta & Heward, 1993; Heward, 1994; Kellum, Carr, & Dozier, 2001; Malanga & Sweeney, 2008). Active student responding involves teaching strategies that increase the probability students will attend to relevant information.

One such form of active student responding, guided notes (Heward, 1994) presents a behavioral methodology that attempts to directly impact student attending with respect to variables that maximize learning in note taking situations. It also can be incorporated easily into typical lecture approaches to teaching without too much response effort on the part of the instructor. Using guided notes involves providing students with the instructor's lecture notes that contain blank spaces where students are instructed to fill in key concepts as didactic instruction proceeds. This strategy combines both active attending on the part of the student, with an inclusion of discriminative stimuli that signal a particular response is needed.

Guided notes were originally introduced for students with learning problems and were based on the notion that students who take good notes and attend to them later will perform better on tests than students who only read the material, or attend lecture during didactic instruction (Baker & Lombardi, 1985; Norton & Hartley, 1986). Subsequent research has demonstrated the effectiveness of this approach in school classrooms (Austin, Lee, Thiebealt, Carr, & Bailey, 2002; Courson, 1989; Pados, 1989; Heward, 1994; Lazarus, 1996). It is interesting that little research on guided notes has occurred in university teaching settings where didactic instruction is a primary method for the delivery of information. The use of guided notes in university classroom settings has been reported for increasing classroom responding (Austin, et al., 2002), and the effects of long versus short form Guided Notes on recall and student satisfaction (Austin & Sasson, 2001). However, not all results show significant differences in subsequent quiz performance when guided notes are used (Neef, McCord, & Ferreri, 2006). Despite variable outcomes, in a review of behavioral approaches to university teaching Austin (2000) described guided notes as an advantageous practice that has been generally shown to increase acquisition of important lecture based information and represents a methodology applicable to current lecture based practice.

Konrad, Joseph, & Eveleigh (2009) conducted a meta-analysis of the guided notes literature finding that eight studies met criteria for, at minimum, a comparison of a guided notes condition and a traditional, didactic lecture condition. Many factors were considered in this review, however, of particular interest was an analysis of the percentage of non-overlapping data points (PND) between condition. In these studies, PND was reported to range from 10%-100% overall (a lesser percentage of PND represents a greater difference between outcome scores); with a mean of 52.95%. Addition of a review component before taking a quiz was shown to decrease the gap in outcomes between conditions further. The present study sought to evaluate the relative effectiveness of guided notes as compared to traditional didactic lecture (Traditional Lecture) in two university classes with a comparison of test performance in both immediate and delayed testing conditions.

* METHOD

PARTICIPANTS AND SETTING

Undergraduate students (71) attending a large Western university participated in this study. Students were enrolled in one of two undergraduate psychology courses that met twice a week (Tuesday and Thursday). Class 1 had an enrollment of 35 students, and Class 2 had an enrollment of 36 students. Both classes followed the same class sequence/structure with the exception of when the quiz was administered. Upon starting class, the instructors would pull up the presentation on a large screen at the front of the class, ask if there were any questions or clarifications needed regarding prior material, answer those questions, and then initiate the lecture for the day. All quizzes were scored by the second author immediately following the administration of each quiz. This format lasted throughout the semester. The first author was the instructor for Class 1 and the third author was the instructor for Class 2.

MATERIALS

Throughout the semester each class consisted of a traditional style lecture delivered by the instructor and a quiz on the material delivered either at the end of the class (Class 1), or with a delay till the start of the next class (Class 2). To ensure consistency, lectures were standardized across all topics both within and between the courses. That is, material for each lecture in each course was reviewed to ensure consistency in the number and type of missing content. In addition, the material for each course was compared to ensure similar content, difficulty, and that the amount of material omitted was consistent. This was accomplished by identifying background and relevant information from each chapter in a given course's assigned text in outline format. All lectures, regardless of condition, were delivered via PowerPoint[R] presentation from the front of the classroom with the only difference being the inclusion of printed handouts during the Guided Notes condition (describe below) which consisted of the same material present on the instructor's PowerPoint[R] slides.

During the Guided Notes condition, students were provided with the full outline handout that omitted pertinent words or phrases that matched omitted text in the instructor's slide presentation. For example:

Positive reinforcement is the--of a stimulus--behavior that results in an--in future probability of that behavior in similar contexts.

Negative reinforcement is the--of a stimulus--behavior that results in an--in future probability of that behavior in similar contexts.

The instructor's slides would first show the same information omitted as the hand out notes until the class was called upon to actively provide the relevant information. During these instances, the class would be provided with a few seconds (10 s or less) to provide a response. Only individual students were permitted to respond (not the group). After a student responded, the correct information would appear on the screen regardless of whether the student responded correctly or incorrectly. That is, if a student responded correctly, the omitted material would appear on the screen in place of the blank. If the student responded incorrectly, the instructor would not call on another student, and the omitted material would appear on the screen in place of the blank. In either situation, the instructor would repeat the complete correct phrase after the omitted material appeared In addition, space was provided on the handouts for additional notes to be taken on the right-hand side by students.

DEPENDENT MEASURES

The dependent measure in this study was class quiz average across two conditions (Guided Notes & Traditional Lecture). The quiz average was computed by summing all student scores and dividing by the total number of students. Averages are reported to two decimal places.

INTER-OBSERVER AGREEMENT

A second reviewer examined 100% of quiz scores from ScanTron[R] forms as well as the grading of the short answer for each quiz across both conditions and both classes. Agreement was calculated by summing agreements for each ScanTron[R] score and short answer by quiz and dividing by the number of agreements and disagreements on scores. This score was then multiplied by 100 to compute a total percentage. Agreement was 100% for both conditions across both classes.

DESIGN

Effects of Traditional Lecture and Guided Notes were evaluated through outcome scores on quizzes that were administered at the end of the lecture (Class 1) or at the start of the following class (Class 2). The design utilized was an alternating treatments design (Barlow & Hayes, 1979). It was decided at the beginning of the study to start each class as a Traditional Lecture condition and to then randomly alternate between conditions by flipping a quarter prior to any class meeting. In addition a rule on alternation was included to ensure that there was never more than two successive presentations of either condition. In the case where a condition occurred twice in a row, it was decided that the next class meeting would occur under the other condition. Aggregates of the scores by all students were used to evaluate group performance throughout each condition. All quizzes consisted of 15 multiple choice, 5 true-false, and 1 short answer questions.

In Class 1, the quiz was delivered during the last 20 min. of the class in which the information was presented and the students were given an opportunity to review their notes for a total of 5 minutes before the quiz was administered. Class 2 quizzes were structured the same, with the exception that the quiz was delivered at the start of the following class period. This allowed for either a two-day or four-day period of time between the presentation of the material and the quiz each week. At the end of each class in Class 2, students were reminded to utilize the time between classes to study (however, the extent to which students studied between classes was not assessed).

* RESULTS AND DISCUSSION

Figures 1 and 2 show outcome scores on quizzes in Class 1 and Class 2, respectively. As can be seen for Class 1, there was an initial increasing trend in quiz scores for both teaching methods with an eventual gradual decrease over the semester. However, the Guided Notes condition resulted in higher quiz scores for students throughout the semester. Variability within condition was modest (4.28 points in the Guided Notes condition, and 5.07 points in the Traditional Lecture condition) with variability between conditions of 2.13 point advantage in the Guided Notes condition overall. As each point equates to 4% on a 100% scale, this computes to a grade differential of 8.52%, on average, throughout the semester. This results in nearly a grade level difference for students in Class 1 in the Guided Notes condition.

[FIGURE 1 OMITTED]

For Class 2, there was a mostly flat trend for both conditions with a slight decrease at the end of the semester for the Traditional Lecture condition. Variability within condition was about the same as that seen in Class 1 for the Guided Notes condition (4.11 points) but less so in the Traditional Lecture condition (2.57), with variability across conditions of 2.04 points. As discussed with Class 1, this results in nearly a full grade advantage in the Guided Notes condition (8.16%). Overall, the differentiation of quiz scores for this class is less evident than that seen in Class 1 (fig 1); however, there is a clear association between the guided notes format and higher quiz scores. Figure 2 represents a replication of Class 1 with the difference that quizzes were taken at least two days later.

[FIGURE 2 OMITTED]

This research provides convincing data on guided notes being associated with higher quiz scores due to its alternating treatments applications in two classes. Interestingly, there appeared to be an effect on overall quiz score (Class 2) when a two day delay was involved prior to testing. That is, overall quiz scores, regardless of condition, were much lower than in Class 1 where the delay to test was approximately 5 minutes. One might suggest that the two day delay would result in additional time being devoted to studying the content for the quiz; however, it appears there was little to no review of the material before the quiz was administered at the start of the following class. One possible explanation is that having the instructors' notes complete with the missing material may have provided the students with a false sense of security. This may be the case in that less studying was required as they did not have to pick through all lecture material. If students studied only the material containing omissions, much of the overall lecture material would have been ignored. Regardless, the Guided Notes condition consistently resulted in higher quiz scores than the Traditional Lecture condition, when a delay was imposed. It would seem that further study is necessary to better understand the possible benefits from attending to material as required in guided notes and delay in testing for that material.

These data provide support for the use of guided notes in University classroom lecture courses such that more effective delivery of the focal points of lecture may be made. The significance of the effects of guided notes is more obvious when looking at student's overall grades for the semester. That is, guided notes were associated with quiz score differences on the order of a full grade point. They extend the finding reported by Austin (2002) as well as previous research. Unfortunately, in research of this nature, there are a variety of confounding variables to be dealt with that would appear out of the realm of possible control. Idiosyncratic variables such as study habits, past college level performance, and competing study needs are difficult to control for. Additionally, possible causal variables such as attention, amount and type of information represented by guided note blanks (Austin & Sasson, 2001), rule formation, time to test and type of test, as well as their possible interactive effects, will need clarification in future research.

When visually analyzing the data, it appears there is an overall decreasing trend for both conditions in Class 1 and an overall decreasing trend for Traditional Lecture in Class 2. This may be a function of material becoming more difficult over the course of the semester as an artifact of the material being cumulative. If this were the case, however, it would be expected that the performance in Guided Notes condition in Class 2 would have shown a greater decrease than observed. Instead, the course permitted dropping of one quiz grade in the semester. It is assumed that the decreases observed in the final weeks of the semester was primarily due to students' deciding to drop one of the final quizzes. If a small percentage of students did not study for one of the final quizzes, the overall average would drop considerably and result in lower aggregate quiz scores.

This begs the question of whether aggregate quiz scores are acceptable as a meaningful unit of analysis for reporting on guided notes research. Another option, taken from behavior analytic method, would seem to suggest that it is sensible to conduct an individual performer analysis wherein it may be possible to ascertain changes in individual performance between Traditional Lecture and Guided Notes conditions. Data could then be reported on the proportion of students who perform at a particular level change in a Guided Notes condition quiz versus the most recent Traditional Lecture condition quizzes computed as a moving average. For instance, while arbitrarily set, a change score of a half letter grade over the moving average for Traditional Lecture quizzes for that student could be identified as an instance of improvement as a result of the Guided Notes intervention. In computing this for each student, it would be possible to report a percentage of students who performed at a half letter grade better in the Guided Notes condition.

Finally, while infrequent, it is interesting to note that several students individually performed better in the Traditional Lecture condition without the added guided notes. It is not clear why this effect would be observed, however, it can be surmised that a history of effective attending and note-taking by some students could be undermined by the relative simplicity of guided notes. In fact, it may be possible that the use of guided notes could actually lead to less attending of other material and thus, less focus on larger bodies of information. That is, the omission of words or small segments of larger pieces of information may lead a student to merely focus on those smaller parts. As a result, the student may be inclined to ignore the rest of the passage leading to a lack of context surrounding the words or phrases that are omitted.

This is, of course, only one of many possibilities and lends credence to further study of the interactional effects of speaker and listener in the context of lecture-based classes at the undergraduate level. Another possible off-shoot of the active learner concept is the analysis of distance learning materials centered on the use of guided notes technology in a train and test format.

* REFERENCES

Austin, J. L. (2000). Behavioral approaches to college teaching. In J. Austin and J E. Carr (Eds.), Handbook of applied behavior analysis (pp. 449-472). Reno, NV Context Press.

Austin, J. L., M., Thiebealt, M., Carr, J. E., & Bailey, J. S. (2002). Effects of guided notes on student responding and recall in a university classroom. Journal of Behavioral Education, 11, 243-254.

Austin, J. L., & Sasson, J. R. (2001). A comparison between long-form and short-form guided notes in a university classroom. Unpublished manuscript

Baker, L., & Lombardi, B. R. (1985). Students' lecture notes and their relation to test performance. Teaching of Psychology, 12, 28-32

Barbetta, P. M., & Heward, W. L. (1993). Effects of active student response during error correction on the acquisition and maintenance of geography facts by elementary students with learning disabilities. Journal of Behavioral Education, 3, 217-233

Barbetta, P. M., & Skaruppa, C. L. (1995). Looking for a way to improve your behavior analysis lectures? Try guided notes. The Behavior Analyst, 18, 155-160

Barlow, D.H., & Hayes, S.C. (1979). Alternating treatments desiGuided Notes: One strategy for comparing the effects of two treatments in a single subject. Journal of Applied Behavior Analysis, 12, 199-210."

Courson, F. H. (1989). Comparative effects of short- and long-form guided notes on social studies performance by seventh-grade learning disabled and at-risk students. Unpublished doctoral dissertation, The Ohio State University, Columbus.

Englemann, S., & Carnine, D. (1982). Theory of instruction: Principles and applications. New York: Irvington

Heward, W. L. (1994). Three "low-tech" strategies for increasing the frequency of active student response during group instruction. In R. Gardner III, D. M. Sainato, J O. Cooper, T. E. Heron, W. L. Heward, J. Eshleman, & T. A. Grossi (Eds.), Behavior analysis in education: Focus on measurably superior instruction (pp. 283-320) Pacific Grove, CA: Brooks/Cole

Keller, F. (1968). Good-bye, teacher. , Journal of/Applied Behavior Analysis, 1, 79-89.

Kellum, K. K., Carr, J. E., & Dozier, C. L. (2001). The effects of response-card instruction on student learning and participation in a college classroom. Teaching of Psychology, 28, 101-104.

Konrad, M., Joseph, L., & Eveleigh, E. (2009). Meta-analytic review of guided notes Education & Treatment of Children, 32(3), 421-444."

Lazarus, B. D. (1993). Guided notes: Effects with secondary and post-secondary students with mild disabilities. Education and Treatment of Children, 16, 272-289

Lazarus, B. D. (1996). Flexible skeletons: Guided notes for adolescents with mild disabilities. Teaching Exceptional Children, 28, 36-40

Lindsley, O.R. (1996). The four free operant freedoms. The Behavior Analyst, 19 199-210.

Malanga, PR., & Sweeney, W. J. (2008). Increasing active student responding in a University Applied Behavior Analysis Course: The effect of daily assessment and response cards on end of week quiz scores. Journal of Behavioral Education, 17 187-199.

Neef, N. A., McCord, B. E., & Ferreri, S. J. (2006). Effects of guided notes versus completed notes during lectures on college students' quiz performance. Journal Of Applied Behavior Analysis, 39(1), 123-130. Norton, L. S., & HarTraditional Lectureey J. (1986). What factors contribute to good examination marks? The role of note taking in subsequent examination performance. Higher Education, 15, 355-371

Pados, G. (1989). A comparison of the effects of students' own notes and guided notes on the daily quiz performance of fifth-grade students. Unpublished master's thesis, The Ohio State University, Columbus.

Skinner, B.F. (1968). The Technology of Teaching. Englewood Cliffs, New Jersey Prentice-Hall

W. Larry Williams--University of Nevada

Timothy M. Weil--University of South Florida

James C. K. Porter--Kerry's Place Autism Services

Correspondence concerning this paper should be sent to Timothy Weil, Ph.D., BCBA, Applied Behavior Analysis Program, 13301 Bruce B Downs Blvd, MHC 2115A, Tampa, Fl 33612, email: tweil@usf.edu, phone: 813.974.2408
Gale Copyright: Copyright 2012 Gale, Cengage Learning. All rights reserved.