The effects of multiple exemplar training on a working memory task involving sequential responding in children with autism.
Abstract: This study is part of a programmatic line of research into the use of basic positive reinforcement procedures for improving working memory in children with autism spectrum disorders. The authors evaluated the effects of multiple exemplar training, utilizing positive reinforcement, on performance of a "digit span backwards" task--a test of working memory that entails sequential relational responding. All three participants showed improved performance on directly trained stimuli as well as maintenance and generalization to untrained stimuli. The results provide further support for the general hypothesis that performance on working memory tasks is amenable to improvement via behavioral intervention and has implications for treating such tasks as relational operants. Implications for future research and the development of clinical interventions are discussed.

Key words: autism, working memory, executive function, positive reinforcement, relational frame theory, relational responding
Article Type: Report
Subject: Short-term memory (Physiological aspects)
Short-term memory (Demographic aspects)
Autistic children (Physiological aspects)
Psychophysiology (Research)
Authors: Baltruschat, Lisa
Hasselhorn, Marcus
Tarbox, Jonathan
Dixon, Dennis R.
Najdowski, Adel
Mullins, Ryan David
Gould, Evelyn
Pub Date: 06/22/2012
Publication: Name: The Psychological Record Publisher: The Psychological Record Audience: Academic Format: Magazine/Journal Subject: Psychology and mental health Copyright: COPYRIGHT 2012 The Psychological Record ISSN: 0033-2933
Issue: Date: Summer, 2012 Source Volume: 62 Source Issue: 3
Topic: Event Code: 310 Science & research
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 298058825
Full Text: Individuals with autism spectrum disorders (ASD) demonstrate deficits in language and socialization as well as display restricted and repetitive behaviors and interests (American Psychiatric Association [APA], 1994). A variety of other deficits have also been documented in the ASD population, including executive function (BF), in general, and working memory (WM), in particular (Geurts, Verte, Oosterlaan, Roeyers, & Sergeant, 2004; Ozonoff, 1997; Ozonoff et al., 1991; Prior & Hoffman, 1990; Rumsey & Hamburger, 1988; Verte, Geurts, Roeyers, Oosterlaan, & Sergeant, 2006; see Hill, 2004, for a more recent review). This body of literature has documented deficits in WM across a variety of tests that measure it and across a wide range of participant ages.

The term working memory generally refers to "the ability to retain information during a delay and then to make a response based on that internal representation" (Klingberg et al., 2005). An example of a task that reportedly involves working memory is hearing a sentence and then answering an unexpected question about it (e.g., What was the first letter of the third word in the sentence?) after an interval of time has elapsed, during which the individual was responding in some other way (e.g., engaging in a distractor task of some kind). The behavior-environment relations that are present in such interactions appear to be quite complex. At a minimum, these phenomena appear to involve an initial stimulus of some kind, a response to that stimulus (overt or covert), a series of other stimuli that must be responded to, and then a final stimulus that provides an opportunity to respond in some way to the very first stimulus (e.g., recall some of its features). In other words, psychological events that involve working memory entail a complex series of stimuli and responses, over some period of time, with the terminal response remaining in some way relevant to the initial stimulus.

Many everyday activities appear to consist of similar complex sequences of relations. For example, children who receive multiple-step instructions from a teacher are required to attend to the instructions and then respond to them at a later time, despite the presence of multiple distractions in the environment during the intervening period. Similarly, grocery shopping involves behavior that adjusts to current stimuli on a moment-to-moment basis, while at the same time remaining relevant to the overall task and initial stimuli (the need for particular groceries). Searching for a lost item, keeping a conversation on topic, and executing a plan require continued attention to stimuli from the recent past as well as continued response to ongoing stimulation--and all are tasks the general psychology community would suggest involve WM. In addition, a number of studies have shown that performance on tasks that measure WM positively correlate with other meaningful outcomes, such as numeracy and literacy achievement and long-term academic success (Alloway & Alloway, 2010), and development of language (Gathercole, Service, Hitch, Adams, & Martin, 1999) and negatively correlate with factors such as the amount of special education support needed (Gathercole & Pickering, 2000). These studies are merely correlational, and it would be simplistic to assume that WM is the cause of these outcomes, but it does suggest that the behavioral repertoires involved in measures of WM may participate in broader repertoires that are responsible for achieving important outcomes.

Despite the documented deficits in WM in children with ASD, relatively little research has been published on procedures for improving them. A small number of studies have evaluated approaches to improving WM, often focusing on children with attention-deficit/ hyperactivity disorder (ADHD), fetal alcohol spectrum disorder (FASD), or Down syndrome. Tatum and colleagues (2010) found that training selective, alternating, and divided attention produces WM gains in children with ADHD. Klingberg's research group found that a period of computerized WM training increased a range of cognitive abilities, including IQ test scores (Klingberg, Forssberg, & Westerberg, 2002). In an early study on rehearsal, Farb and Throne (1978) found that a rehearsal training program effectively improved the mnemonic performance of a child with Down syndrome. Subsequent research replicated their findings with typically developing peers. For example, Turley-Ames and Whitfield (2003) found that WM span scores increased as a result of using a rehearsal strategy. However, to date, little has been published on the remediation of WM deficits in individuals with ASD.

Working memory has remained the subject of little work in behavior analytic psychology. It is possible that some behavioral psychologists do not see traditionally cognitive areas, such as working memory, as a suitable subject matter for behavior analysis. However, humans are doing something during tests of WM. That is, behavior is always involved in the psychological events labeled memory (Palmer, 1991), and if behavior analytic psychology is to be conceived as a comprehensive science of the behavior of organisms (Skinner, 1938), the field must presumably include actions of these sorts. Furthermore, the exclusion of WM from behavioral investigation is unfortunate, because a behavioral approach to addressing any psychological phenomena has the practical advantage of being based on decades of research on the basic processes of learning and motivation (Catania, 1998). It also has a pragmatic approach, in which the value of a theory is assessed on the basis of how useful it is in addressing the topic (Skinner, 1974).

The expansion of behavior analytic research into phenomena labeled as working memory is important for at least two reasons. First, to further expand the scope of behavior analysis as a comprehensive science of psychology, it is important to study the full range of phenomena that general psychology (and, indeed, most people outside of psychology) believes to be important. If behavioral theories cannot account for major phenomena within psychology (e.g., memory), and, more important, if behavioral methods cannot produce meaningful change when those phenomena are disordered, then the science of behavior analysis will continue to be viewed by many psychologists as a specialist discipline, relegated to animal behavior and the simple performances of humans. One need not believe in the existence of the cognitive explanatory mechanisms posited by cognitive psychologists in an area such as WM in order to appreciate the value of addressing the behavior-environment relations labeled as such, from a behavioral perspective, with the goal of advancing (if slowly) toward a more complete behavioral account of everything organisms do.

Second, behavioral research on WM is important because the term working memory refers to a complex and interesting set of behavior-environment relations. From a behavioral perspective, the tasks involved in tests of working memory are not presumed to measure some other hypothetical cognitive or neural construct; they are legitimate behaviors in their own right. Further, they appear to be part of larger generalized behavioral repertoires that are important to everyday functioning. Consider a relatively common instruction a classroom teacher might give: "Students, please go to your cubby quietly, get your activity folder, turn to the math section, and continue where you left off yesterday." Successfully responding to this instruction requires responding differentially to multiple stimuli along the way (e.g., walking to the cubbies, finding one's backpack, avoiding bumping into other children), omitting responses to multiple distracting stimuli (e.g., jokes told by other students, events occurring outside the classroom window), and then finally responding again to the teacher's original instruction (finding the math section of the workbook), all drawn out over the course of perhaps a minute or more. The same example presumably applies to tasks in even the most rudimentary vocational environment. In short, everyday functioning constantly requires responding to multiple stimuli, in a complex manner, over extended periods of time: behavior-environment relations that share many features with common tests of working memory.

Behavior analytic principles and procedures provide a promising foundation for developing treatments to remediate WM deficits in individuals with ASD. A small number of basic studies conducted with typically developing adults demonstrated that positive reinforcement can affect performance on short-term memory tests (Cuvo, 1974; Loftus, 1972); however, little behavioral conceptual work has been published on the topic. One exception is a behavioral interpretation of memory by Palmer (1991), who posits two types of behavior involved in what are generally said to be tests of memory. Simple cases of recalling directly learned behaviors in the presence of stimuli during training are a matter of simple stimulus control and require no further explanation. Far more interesting, however, are cases wherein recall requires the emission of a behavior that has never been reinforced in the presence of the stimulus, for example, when answering the question "What did you have for breakfast three days ago?" Correct recall to such a question changes from day to day, although the topography of the question remains unchanged; that is, the correct answer cannot simply be under the direct stimulus control of the question. Palmer suggested that responding correctly in such circumstances requires the emission of intervening behaviors, potentially of great complexity, eventually providing sufficient stimulation to evoke the correct response. Further, such psychological events share much in common with problem-solving repertoires of other varieties. Space does not permit a thorough discussion of the implications of Palmer's (1991) analysis, but his paper contributed a thoroughly behavioral account of memory and identified the need for attention to be paid to intervening behaviors occurring at the covert or overt levels that facilitate correct responding in circumstances of remembering.

Our group has been developing a programmatic line of research on the use of behavioral intervention procedures for improving WM in children with autism, of which the first experiment was recently published (Baltruschat et al., 2011). In that study, positive reinforcement was used to improve performance on a WM task, referred to as a "counting span" task. The counting span task required participants to count quantities of shapes presented in a series of visual stimulus cards, and then accurately report each quantity, in the correct order in which they were presented. Positive reinforcement produced large improvements in accuracy, accompanied by maintenance after reinforcement was withdrawn, as well as generalization to untrained stimuli (untrained shapes, colors, and quantities).

The results of the initial study (Baltruschat et al., 2011) were encouraging; however, replication across additional WM tasks is needed before any more general conclusions can be made about the effectiveness of positive reinforcement in improving performances that are said to be indicative of WM. The counting span task can be thought of as a relatively straightforward task, in that it simply requires participants to count--a behavior that is under the direct stimulus control of the visual objects as discriminative stimuli, and a behavior that occurs essentially the same way each time, albeit up to different quantities. Additional research is needed on the use of positive reinforcement for improving performance on complex behavior-environment relations that are labeled as WM.

Relational frame theory (RFT) is the most well-developed contemporary behavior analytic account of cognition. A full conceptual treatment of RFT is beyond the scope of this article and has already been accomplished elsewhere (see Hayes, Barnes-Holmes, & Roche, 2001, as well as Rehfeldt & Barnes-Holmes, 2009, for a volume-length discussion of the application of RFT to developmental disabilities). However, the concept at the core of the theory is that cognition involves relating events in one's environment and that relating is generalized operant behavior, learned through a history of multiple exemplar training. From this standpoint, working memory tasks that require participants to "cognitively process" stimuli as they contact them likely involve active relational responding. When viewed in this manner, the term cognitively process does not refer to a hypothetical causal mechanism but rather to a real behavior the participant is actively engaging in, that is, relating. Furthermore, in one of the only existing behavioral conceptual accounts of executive function (EF), Hayes, Gifford, and Ruckstuhl (1996) suggested that many of the behavior-environment interactions that characterize EF are composed of relational responding. However, little or no research has been conducted on interventions that improve relational responding in tasks that are said to measure EF.

The purpose of this study was to evaluate whether the basic behavioral procedure of positive reinforcement of multiple exemplars could produce lasting and generalized improvement in a WM task that involves relating behavior. As a convenient starting point, the "digit span backwards" (DSB) task was selected for inclusion in this study. The DSB task is a commonly used procedure for assessing working memory (Aleman & van't Wout, 2008; Liu-Ambrose, Ahamed, Graf, Feldman, & Robinovitch, 2008; Zaninotto et al., 2009). The task presents a sequence of auditory stimuli, and the participant is subsequently asked to repeat the sequence of stimuli verbally, in reverse order. The task is said to measure a participant's ability to simultaneously store and process information and was developed so as to prevent the participant from verbally rehearsing the stimuli during the task (i.e., the demand to transform the order of the stimuli sequence from forward to backward internally is presumed to interfere with rehearsal). From the standpoint of RFT, this task likely involves "temporal relating" (Hayes, Barnes-Holmes, et al., 2001), in that participants are required to respond to the sequence in which stimuli are presented by responding to an instruction with a contextual cue (e.g., the word reverse or backward) for relating stimuli in the opposite order to the one in which they encountered them. The ability to relate events with respect to their relative place in time is presumably learned over the course of regular development, via repeated contact with specific exemplars, such as parents asking their children, What did you do before? What did you do after? Look, we are driving backwards, and so forth. A child who is tested on a DSB task and who has had a sufficient learning history to establish the generalized relational operant of which the task is an example would presumably perform within the normal range. However, relational operants are part of language (Hayes, Barnes-Holmes, et al., 2001), and autism is characterized by delayed language, so it is not surprising that many also perform poorly on tasks that involve relational responding, such as DSB tasks.

The encouraging feature of an RFT approach to analyzing DSB performance is the assumption that it is learned behavior and thus should be amenable to improvement via multiple exemplar training. The investigation of this possibility was the purpose of the current experiment. In this study, performance on multiple exemplars of the DSB task was directly reinforced in children with ASD. Further, maintenance was assessed after positive reinforcement was terminated, as was generalization to novel stimuli and responses, thereby allowing for an assessment of whether positive reinforcement resulted in improvements in participants' generalized ability on DSB tasks rather than on memorization of directly trained sequences of particular behaviors.

Method

Participants and Setting

The inclusion criteria for participation in the study follows: (a) participants are between 6 and 10 years of age; (b) participants are diagnosed on the autism spectrum; (c) participants must have a well-developed language repertoire (ability to follow and understand rules, follow complex instructions, etc.); (d) both the participant's family and therapy team agree that improvement in working memory is a clinical priority. All participants were diagnosed by independent psychologists, using criteria from the Diagnostic and Statistical Manual of Mental Disorders (APA, 1994) and/or the Autism Diagnostic Observation Schedule (Lord, Rutter, DiLavore, & Risi, 1999). Three boys meeting these criteria participated in the study. All participants were clients of a large-scale, community-based provider of home-based behavioral intervention services. All participants were receiving comprehensive behavioral intervention services that addressed all skill areas in which they exhibited deficits (e.g., academics, social skills, language, play, adaptive, and motor skills). The study procedures were approved by an institutional review board, and the parents of each participant gave written consent to participate before the study began. Participants are described in the next section, using aliases.

All sessions except for the Arbeitsgedaechtnis Testbatterie (described in a following section) were conducted in the participants' homes, in the typical environments in which they received behavioral intervention. Sessions occurred in rooms that contained desks, chairs, and a variety of intervention and play materials. Each session included five trials and occurred two-to-four times per day, two-to-three days per week. During each session, the first author and the child were present.

Experimental Design

Since each participant was expected to display individual differences, a single-subject design was implemented to detect and evaluate intervention effects at the level of the individual. It was hoped that the effects of treatment would remain after treatment was discontinued (i.e., not reverse), so a multiple baseline across participants design was selected.

Response Measurement and Interobserver Agreement

During all tabletop phases (i.e., all phases except for the Arbeitsgedaechtnis Testbatterie), accuracy data were collected for each trial and were summarized and analyzed as percent correct. Correct responding consisted of repeating the stimuli presented in the trial, in reverse order. A second trained observer independently collected data during 80%, 86%, and 67%, of sessions for Joe, Adam, and Ken, respectively. Interobserver agreement (IOA) was calculated by dividing the number of trials for which both observers scored exactly the same data by the total number of trials for which two observers scored data, and the resulting decimal was multiplied by 100, thereby converting it into a percentage. Mean IOA was 87% (range = 60%-100%), 93% (range = 80%-100%), and 99% (range = 80%-100%) for Joe, Adam, and Ken, respectively.

Arbeitsgedaechtnis Testbatterie

At the start and completion of the study, each participant completed the Arbeitsgedaechtnis Testbatterie (AGTB; Hasselhorn, et al., in press), a computerized German test of working memory. The AGTB is conducted with a touch-sensitive computer monitor and is said to assess the WM of children. It contains the following six subscales: (1) Complex Span, (2) Color Span, (3) Digit Span Backwards, (4) Stroop-Like, (5) Go/No Go, and (6) Counting Span. The AGTB computer program was translated into English for use with English-speaking children. The subscale most relevant to this study was the Digit Span Backwards subtest. During that task, the computer presented a recorded human voice, stating a series of numbers, and the participant was required to recall them in reverse order and state the series vocally back to the experimenter. The number of digits in each sequence started with two and was increased by one digit contingent on two consecutive correct trials.

The pre- and postcomputerized AGTB assessment sessions were conducted in a room at the clinic. The room contained two desks, two chairs, and a computer equipped with a touch-screen monitor. Each AGTB session lasted approximately 1 hour.

Procedures

During the tabletop phases (baseline, positive reinforcement, maintenance, and generalization), random sequences of letters were presented vocally by an experimenter, and participants were asked to recall them in reverse order and state them vocally back to the experimenter. Stimuli were selected from a pool of 16 letters. The 16 letters were randomized and then split in half: half for use during baseline, positive reinforcement, and maintenance phases (letters A, C, E, G, I, K, M, and 0) and half for use during generalization probes (letters B, D, T, H, Y, L, N, and P).

Prebaseline evaluation. Digit span backwards tasks range in difficulty, depending on the number of stimuli presented in the sequence, so it was necessary to identify a number of stimuli to present during baseline that produced consistently low levels of accuracy. To determine this number, we presented trials with only two stimuli in the sequence, with the number of stimuli gradually increasing on successive trials, contingent on two consecutive correct trials, until the participant made two consecutive errors. We then chose the number of stimuli at which errors occurred on two consecutive trials for inclusion in baseline (three for Joe and Ken, four for Adam).

Baseline. Before the first baseline trial was conducted, the experimenter provided the instructions to the participant and practiced the task with the child twice. On each baseline trial, the experimenter presented letter sequences vocally, and the child was asked to recall them in reverse order. Specifically, after presenting the sequence of letters, the experimenter asked, "What letters do you remember, backwards?" No differential consequences or feedback was provided for correct or incorrect responding, the experimenter simply said "Okay, let's do the next one," and moved on to the next trial, regardless of the participant's response to a trial.

Positive reinforcement. Before each session, a brief multiple stimulus preference assessment was conducted, wherein participants were asked to select an item from an array of highly preferred items (e.g., video game, candy, movie, drawing material, stickers). Participants' parents were asked to restrict access to the pool of highly preferred items outside of experimental sessions. Contingent on each correct response during a session, participants were given the selected reinforcer for that session (i.e., 1 minute access for tangible items or a small bite of food for edible items). Contingent on an incorrect response, participants were given descriptive feedback (e.g., "No, that's not right, let's try again"), and the next trial was initiated. When more than one session was conducted consecutively, participants were given 5-min breaks between sessions. At the beginning of each session, the contingencies were explained to the participant (e.g., "Every time you get an answer right, you get to play your video game for 1 minute.").

For one participant (Ken), the positive reinforcement procedure did not lead to a stable increase in performance and was therefore modified into a token economy on Session 40. The contingencies in place for the token economy were as follows: (a) each correct response earned one token, (b) the number of tokens Ken was required to earn before receiving the backup reinforcer and a break from work began at two, (c) incorrect responses resulted in loss of all tokens on the token board (correct responses had to be consecutive in order to earn the backup reinforcer). The number of tokens required to fill the token board (and thus earn the backup reinforcer and a break) was gradually increased in increments of one, until mastery was achieved with four tokens. Thus, when four tokens were required to earn reinforcement, the participant had to achieve at least four out of five trials in a session correct (i.e., 80% correct or higher). Therefore, when the criterion for reinforcement was four tokens, it amounted to a differential reinforcement contingency, wherein Ken could only receive the backup reinforcer if he demonstrated at least 80% correct on a given session. The backup reinforcer was determined in the same manner as the reinforcers during the positive reinforcement phase.

Maintenance. Visual inspection (without any particular quantitative criterion) was used to determine if the positive reinforcement condition resulted in a large and stable increase in correct responding. If an increase was observed, participants were then exposed to the maintenance condition. This condition was identical to baseline. Participants were given no feedback as to whether their responses were correct or incorrect.

Generalization. During the maintenance phase, if visually inspected data revealed stable levels of accuracy, and these levels were consistently higher than those observed during baseline, generalization probes were conducted. These sessions were identical to baseline sessions, with the exception that novel stimuli were used, which were never included in any training sessions.

Results

Figure 1 depicts the percentage of correct responding during baseline, positive reinforcement, and maintenance phases for all participants. The top panel depicts Joe's data. During the baseline phase, correct responding was consistently low for both baseline/ training stimuli (M = 20%) and generalization stimuli (M = 13%). When the positive reinforcement phase was initiated, Joe's correct responding immediately increased to 100% and then stabilized at 80% (M = 80%), at which time the maintenance phase was initiated, wherein reinforcement was discontinued. Joe's correct responding maintained between 60% and 80% (M = 75%). Joe's accuracy in generalization sessions was slightly lower than his responding to the trained stimuli in the maintenance phase (M = 67%) but was nonetheless substantially higher than baseline.

[FIGURE 1 OMITTED]

The middle panel depicts Adam's data. During baseline, Adam's responding was low to the training stimuli (M = 23%) and the generalization stimuli (M = 26%). Once positive reinforcement was introduced, Adam's accuracy increased across successive sessions, stabilizing at 100% (M = 86%). When the maintenance phase was introduced, Adam's accuracy remained high for both the training stimuli (M = 86%) and generalization stimuli (M = 100%).

The bottom panel depicts Ken's data. During baseline, Ken demonstrated low accuracy with both training stimuli (M = 26%) and generalization stimuli (M = 20%). When positive reinforcement was introduced, his performance increased immediately to 100% but remained variable (M = 65%). Therefore, the positive reinforcement procedure was modified into a token economy on Session 40. Ken's responding increased and became stable at 80% (M = 78%). Ken displayed high accuracy during the maintenance phase to trained stimuli (M = 75%) and on generalization probes (M = 76%).

Secondary Analysis: AGTB Results

Figure 2 shows the pre- and posttest scores for all participants on the Digit Span Backwards subtest of the AGTB. They axis depicts the mean length of the last four series correctly recalled, out of a total of 10 for each participant. Joe's score improved from recalling an average of 1.75 letters before treatment to 2.75 letters after treatment. Adam recalled on average 3.75 letters after treatment compared to 3 on average before treatment. Ken's recall increased from 2 letters to 3.75 letters.

[FIGURE 2 OMITTED]

Discussion

The results of this study provide further support for the general notion that the behavioral repertoires labeled as working memory may be amenable to modification in some high-functioning children who have autism via basic behavioral intervention procedures, such as positive reinforcement. The current study extends the findings of Baltruschat et al. (2011) by demonstrating that positive reinforcement can improve performance on yet another test that is said to measure WM, one that involves more complex behavior-environment relations. The current results also have implications for an RFT analysis of working memory. It was not the purpose of the study to identify whether performance on DSB tasks is indeed a genuine instance of derived relational responding--to do so would require far more in terms of the various controls which are generally used in RFT research (Hayes, Barnes-Holmes, et al., 2001). Nevertheless, it is reasonable to analyze performance on DSB tasks as sequential relational responding, given that the tasks require participants to respond to the cues backward and/or reverse by relating the relative sequential or temporal position of the stimuli presented in the task. An RFT analysis of such behavior would suggest that it is a learned operant and that it should be possible to strengthen it in the same way that virtually any other operant can be strengthened--positive reinforcement of multiple exemplars. The results of the study provide preliminary evidence in support of this possibility.

The findings of maintenance and generalization in the study's data warrant discussion. The improvement in performance that reinforcement produced maintained after reinforcement was withdrawn for all three participants. This is an important finding because it suggests that reinforcement may have produced a strengthening in the overall operant class rather than simply providing the participant with more motivation to respond than was present during baseline. The rapid increase in accuracy--from 20% to 100% correct for Joe--which occurred on the first session of the positive reinforcement phase, could potentially suggest that the intervention simply provided motivation that was lacking in baseline. While this is possible, the more gradual increases in accuracy observed for the other two participants, as well as the fact that high accuracy maintained after reinforcement was terminated, suggest that the effect of the intervention was not simply due to increasing motivation during testing.

Perhaps the most promising finding of this study is that the improvement in performance did not only maintain in the presence of stimuli during training but generalized to novel stimuli as well (i.e., completely different letters that were not present during training). Moreover, the improved performance was apparent in the AGTB posttest, which also involved stimuli that were never present during training (numbers), but the AGTB findings must be replicated with an experimental design that allows for the demonstration of experimental control of the change in AGTB scores. These findings provide further evidence that positive reinforcement may have strengthened the overall operant class involved in the task, not only the reinforcement of specific responses to specific stimuli.

One potential limitation of the current study is that we did not attempt to identify how many exemplars, if any, needed to be trained. That is, it is possible that positively reinforcing one exemplar repeatedly could have produced generalization to novel exemplars. However, this seems unlikely, given what is known about the importance of multiple exemplar training (Stokes & Baer, 1977). It is equally possible that practice across multiple exemplars alone, without the use of positive reinforcement, could have produced generalization. In other words, the current study implemented a treatment package containing two components, and no attempt was made to identify which component was necessary. Future research should attempt to evaluate positive reinforcement and practice across multiple exemplars separately. Another potential limitation is the possibility that the procedure used in this study did not strengthen the relating behavior involved in the WM task but rather simply strengthened the behavior of attending to the task. Because attending to stimuli is a behavior that is necessarily precurrent to relating those stimuli, it may be difficult to tease apart the separate contributions of strengthening relational operants versus strengthening attending, but future research should attempt to do so. Finally, we did not attempt to measure or analyze any potential remembering strategies that participants may have been using. Correct responding to the DSB task, particularly in the case of generalization probes that contained untrained stimuli, involved correct emission of previously unreinforced responses, a performance that may have been facilitated by intervening behaviors, such as rehearsal (Palmer, 1991). As suggested by Palmer, these behaviors may be the most interesting and important behavioral components of remembering and should be directly studied in future research.

The results of the Baltruschat et al. (2011) study and this study appear to provide at least initial evidence that positive reinforcement can improve WM in some children with autism, a population with demonstrated deficits in WM. However, much work still needs to be done in translating these initial findings into real-life clinical interventions. No attempt was made in this line of research to demonstrate changes in WM in the everyday lives of the participants. These studies are probably best conceptualized as "bridge" studies, in that they applied a basic learning process (i.e., positive reinforcement) to a basic test of working memory (i.e., DSB), in a clinical population (i.e., autism). This strategy is in line with the general inductive or "ground-up" approach commonly taken in behavior analytic research to tackling problems of social significance (Hayes, Blackledge, & Barnes-Holmes, 2001). However, it cannot be considered a substitute for testing the effectiveness of real-life interventions; it is only a starting-point from which to develop such interventions. Future research will need to evaluate whether bridge procedures such as the one studied here produce a generalized effect in the participants everyday lives and examine how interventions should be tailored to the exigencies of day-to-day clinical intervention.

This study was conducted in partial fulfillment of Lisa Baltruschat's doctoral degree in psychology.

Correspondence concerning this article should be addressed to Jonathan Tarbox, Center for Autism and Related Disorders, 9019 Ventura Blvd, 3rd Floor, Tarzana, CA 91356. E-mail: j.tarbox@centerforautism.com

References

ALEMAN, A., & VAN'T WOUT, M. (2008). Repetitive transcranial magnetic stimulation over the right dorsolateral prefrontal cortex disrupts digit span task performance. Neuropsychobiology, 57, 44-48.

ALLOWAY, T. P., & ALLOWAY, R. G. (2010). Investigating the predictive roles of working memory and IQ in academic attainment. Journal of Experimental Child Psychology, 80, 606-621.

ALLOWAY, T. R, GATHERCOLE, S. E., WILLIS, C., & ADAMS, A. M. (2005). Working memory and special educational needs. Educational and Child Psychology, 22, 56-67.

AMERICAN PSYCHIATRIC ASSOCIATION. (1994). Diagnostic and statistical manual or mental disorders (4th ed.). Washington, DC: Author.

BALTRUSCHAT, L., HASSELHORN, M., TARBOX, J., DIXON, D. R., NAJDOWSKI, A. C., MULLINS, R. D., & GOULD, E. R. (2011). Adressing working memory in children with autism through behavioral intervention. Research in Autism Spectrum Disorders, 5, 267-276, CATANIA, A. C. (1998). Learning. Upper Saddle River, NJ: Prentice Hall.

CUVO, A. J. (1974). Incentive level influence on overt rehearsal and free recall as a function of age. Journal of Experimental Child Psychology, 18, 167-181.

FARB, J., & THRONE, J. (1978). Improving the generalized mnemonic performance of a Down syndrome child. Journal of Applied Behavioral Analysis, 11, 413-419.

GATHERCOLE, S. E., & PICKERING, S. J. (2000). Assessment of working memory in six-and seven-year old children. Journal of Educational Psychology, 92, 377-390.

GATHERCOLE, S. E., SERVICE, E., HITCH, G. J., ADAMS, A.-M., & MARTIN, A. J. (1999). Phonological short-term memory and vocabulary development: Further evidence on the nature of the relationship. Applied Cognitive Psychology, 13, 65-77.

GEURTS, H., VERTE, S., OOSTERLAAN, J., ROEYERS, H., & SERGEANT, J. (2004). How specific are executive functioning deficits in attention deficit hyperactivity disorder and autism? Journal of Child Psychology & Psychiatry, 45,836-854.

HASSELHORN, M., SCHUMANN-HENGSTELER, R., GRUBE, D., KONIG, J., MAHLER, C., SCHMID, I., ... ZOELCH, C. (in press). Arbeitsgedachtnistestbatterie fur kinder von 5 bis 12 jahren [Working memory test battery for children 5 to 12 years]. Gottingen, Germany: Hogrefe--Verlag.

HAYES, S. C., BARNES-HOLMES, D., & ROCHE, B. (2001). Relational frame theory: A post-Skinnerian account of language and cognition. New York, NY: Kluwer Academic.

HAYES, S. C., BLACKLEDGE, J. T., & BARNES-HOLMES, D. (2001). Language and cognition: Constructing an alternative approach within the behavioral tradition. In S. C.

Hayes, D. Barnes-Holmes, & B. Roche (Eds.), Relational frame theory: A post-Skinnerian account of language and cognition. New York, NY: Kluwer Academic.

HAYES, S. C., GIFFORD, E. V., & RUCKSTUHL, L. E. (1996). Relational frame theory and executive function: A behavioral approach. In G. R. Lyon & N. A. Krasnegor (Eds.), Attention, memory, and executive function. Baltimore, MD: Brookes.

HILL, E.L. (2004). Evaluating the theory of executive dysfunction in autism. Developmental Review, 24, 189-233.

KLINGBERG, T., FERNELL, E., OLESEN, P. J., JOHNSON, M., GUSTAFFSON, P., DAHLSTROM, K., ... WESTERBERG, H. (2005). Computerized training of working memory in children with ADHD--A randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 177-186.

KLINGBERG, T., FORSSBERG, H., & WESTERBERG, H. (2002). Training of working memory in children with ADHD. Journal of Clinical and Experimental Neuropsychology, 24, 781-791.

LIU-AMBROSE, T., AHAMED, Y., GRAF, R, FELDMAN, F., & ROBINOVITCH, S. N. (2008). Older fallers with poor working memory overestimate their postural limits. Archives of Physical Medicine and Rehabilitation, 89, 1335-1340.

LOFTUS, G. R. (1972). Eye fixation and recognition memory for pictures. Cognitive Psychology, 3,525-551.

LORD, C., RUTTER, M., DILAVORE, P. C., & RISI, S. (1999). Autism diagnostic observation schedule. Los Angeles, CA: Western Psychological Services.

OZONOFF, S. (1997). Components of executive function in autism and other disorders. In J. Russell (Ed.), Autism as an executive disorder (pp. 179-211). Oxford, England: Oxford University Press.

OZONOFF, S., PENNINGTON, B. F., & ROGERS, S. J. (1991). Executive function deficits in high-functioning autistic individuals: Relationship to theory of mind. Journal of Child Psychology and Psychiatry, 32, 1081-1105.

PALMER, D. C. (1991). A behavioral interpretation of memory. In L. J. Hayes & P. N. Chase (Eds.), Dialogues on verbal behavior (pp. 261-279). Reno, NV: Context Press.

PRIOR, M., & HOFFMAN, W. (1990). Neuropsychological testing of autistic children through exploration with frontal lobe tests. Journal of Autism and Developmental Disorders, 20, 581-590.

REHFELDT, R. A., & BARNES-HOLMES, Y. (2009). Derived relational responding: Applications for learners with autism and other developmental disabilities: A progressive guide to change. Reno, NV: Context Press.

RUMSEY, J., & HAMBURGER, S. (1988). Neuropsychological findings in high functioning men with infantile autism, residual state. Journal of Clinical and Experimental Neuropsychology, 10, 201-221.

SKINNER, B. F. (1938). The behavior of organisms. Acton, MA: Copley Publishing Group.

SKINNER, B. F. (1974). About behaviorism. New York, NY: Vintage Books.

STOKES, T. F., & BAER, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 10, 349-367.

TAMM, L., HUGHES, C., AMES, L., PICKERING, J., SILVER, C. H., STAVINOHA, R, ... EMSLIE, G. (2010). Attention training for school-aged children with ADHD: Results of an open trial. Journal of Attention Disorders, 14, 86-94.

TURLEY-AMES, K. J., & WHITFIELD, M. M. (2003). Strategy training and working memory task performance. Journal of Memory and Language, 49, 446-468.

VERTE, S., GEURTS, H. M., ROEYERS, H., OOSTERLAAN, J., & SERGEANT, J. A. (2006). The relationship of working memory, inhibition, and response variability in child psychopathology. Journal of Neuroscience Methods, 151, 5-14.

ZANINOTTO, A. L. C., BUENO, 0. F. A., PRADELLA-HALLINAN, M., TUFIK, S., RUSTED, J., STOUGH, C., & POMPE, S. (2009). Acute cognitive effects of donepezil in young, healthy volunteers. Human Psychopharmacology: Clinical and Experimental, 24, 453-464.

Lisa Baltruschat

Center of Research on Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt am Main, and the Center for Autism and Related Disorders

Marcus Hasselhorn

German Institute for International Educational Research, Center of Research on Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt am Main

Jonathan Tarbox

Center for Autism and Related Disorders, Autism Research Group

Dennis R. Dixon, Adel Najdowski, Ryan David Mullins, and Evelyn Gould

Center for Autism and Related Disorders
Joe was 7 years old and had been receiving behavioral intervention
  for 5 years. At the time of the study, Joe was receiving 35 hours per
  week of behavioral intervention, consisting of 20 hours per week of

  school shadowing and 15 hours per week of in-home therapy. Joe was
  in good general health, with no history of seizures, accidents, or
  hospitalization. Joe was a participant in the Baltruschat et al.
  (2011) study but had no other prior training in WM.

  Adam was 9 years old, had received behavioral intervention services
  for approximately 6 years, and was receiving between 9 and 16 hours
  per week of behavioral intervention at the time of the study. Adam
  was in good general health, with no history of seizures, accidents,
  or hospitalizations. At the time of the study, Adam was taking
  25 mg of Zoloft (prescribed for his obsessive behaviors)
  and 1 mg of Tenex (prescribed off-label for his hyperactivity)
  per day and continued to do so throughout the study. Joe was also a
  participant in the Baltruschat et al. (2011) study but had no other
  prior training in WM.

  Ken was 6 years old and had been diagnosed with autistic disorder
  when he was 29 months old. Ken had received approximately 5 years of
  behavioral intervention services and was receiving 18 hours per week
  at the time of the study. Ken was in good general health, with no
  history of seizures, accidents, or hospitalizations. Ken had received
  training in one prior WM program wherein classification responses
  hierarchical relating) were trained in sequences, which required
  subsequent recall of classified stimuli. In this task, a sequence
  of visual stimuli was presented to Ken. In response to each stimulus,
  Ken was asked to provide a classification response that identified
  a function of the object (e.g., Can you eat it?).
  At the conclusion of the sequence, Ken was asked to list the
  stimuli in the order in which they were initially presented.
  Although this kind of task would be labeled as WM by the general
  psychological community, from a functional standpoint it did not
  resemble the task in the current study. It involved hierarchical
  (categorization) rather than sequential (one digit following the
  other) relating.
Gale Copyright: Copyright 2012 Gale, Cengage Learning. All rights reserved.