A quantitative analysis of language interventions for children with autism.
Abstract: Research and services continue to expand to community-based programs serving individuals diagnosed with autism. A focus of great interest in those efforts is that of language acquisition and functional usage. For the purpose of this evaluation, language acquisition interventions are grouped into two broad categories, contrived and naturalistic. Contrived interventions include those in tightly controlled settings with specific instructional sequences (e.g. discrete trials), whereas naturalistic interventions are defined as less structured and with instructional sequences that change with the child's interest. This meta-analysis focuses on interventions of spoken language developed for children diagnosed with autism. Contrived teaching approaches were compared to naturalistic approaches during intervention, generalization, and maintenance conditions by calculating the percentage of non-overlapping data (PND). The results illustrate that naturalistic interventions are more effective than contrived interventions for teaching language skills and for maintaining the skills learned. The data also indicate that contrived approaches result in greater generalization than naturalistic approaches. Implications and future directions for the acquisition and generalization of acquired language skills and the importance of treatment integrity are discussed. Keywords: autism, contrived teaching, discrete trial teaching, DTT, language interventions, meta-analysis, natural environment teaching, naturalistic, NET
Article Type: Report
Subject: Autism (Care and treatment)
Autism (Research)
Children (Health aspects)
Children (Psychological aspects)
Special education (Usage)
Authors: Kane, Meghan
Connell, James E.
Pellecchia, Melanie
Pub Date: 03/22/2010
Publication: Name: The Behavior Analyst Today Publisher: Behavior Analyst Online Audience: Academic Format: Magazine/Journal Subject: Psychology and mental health Copyright: COPYRIGHT 2010 Behavior Analyst Online ISSN: 1539-4352
Issue: Date: Spring, 2010 Source Volume: 11 Source Issue: 2
Topic: Event Code: 310 Science & research
Product: Product Code: E121920 Children; 8294000 Education of Handicapped; 9105115 Special Education Programs NAICS Code: 61111 Elementary and Secondary Schools; 92311 Administration of Education Programs
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 244271252
Full Text: Autism is not only of the most prevalent developmental disabilities, but it is also the fastest growing according to the Autism Society of America, (2008). The Center for Disease Control and Prevention statistics indicate that 1 in 150 8-year-old children in the United States have an autism spectrum disorder (CDC; 2007). The rising incidence may be due to increased awareness, early identification markers and screenings, and more sensitive and specific assessment diagnostic instruments. As the number of children with the disorder rises, so too does the need for qualified therapists and effective interventions to maximize each child's full potential. As such, the CDC recommends that individuals diagnosed with autism receive evidence-based, early intervention services as soon as possible (CDC; 2007).

According to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV; 1994), autism spectrum disorder (ASD) is described by significant deficits in three behavioral domains: 1) qualitative impairment in social interaction, 2) qualitative impairment in communication, and 3) restricted, repetitive and stereotyped patterns of behavior, interests, and activities. Communication deficits include a delay in the development of spoken language. When language does develop, impairments in conversational language occur with high frequency (DSM-IV; 1994). Many instructional models (e.g. discrete trial teaching, incidental teaching, pivotal response training) specifically teach spoken language as part of the instructional sequence to remediate these debilitating communication deficits.

As stated above, there are multiple approaches used to teach individuals with developmental disabilities spoken language just within the field of applied behavior analysis (ABA). Historically, the approach most associated with ABA is discrete trial teaching (DTT). Discrete trial teaching is a systematic and structured teaching methodology, consisting of "discrete" trials. A discrete trial consists of one concise instruction, a learned response, and a consequence highly controlled by an instructor. Discrete trial teaching sessions generally occur at an isolated table in a designated area of a home or school and thus the model has received significant criticism over the years (Steege, Mace, Perry, & Longnecker, 2007). Therefore, for purpose of this investigation, all research using DTT is referred to as contrived approaches because the instructional strategy is not "typical" of a naturalistic setting.

In addition to the setting and approach being contrived, discrete trial critics have argued that there is a lack of skill generalization, that the instructional approach only produces rote responding, and that there is an inability to teach sequential chains since instruction only occurs as discrete trials (Steege et al., 2007; Sundberg & Partington, 1998). For these reasons, critics have referred to DTT as an analog training condition and not likely to generalize to natural contingencies of reinforcement.

Autism interventions have also evolved to address some of the criticisms of DTT and to better address the needs of the population. Some researchers worked toward altering teaching strategies in hopes of achieving more promising outcomes. For instance, Koegel, O'Dell, and Koegel (1987) conducted a study in which they manipulated teaching variables to include more functional teaching stimuli, naturalistic reinforcers, and teaching within the natural environment. The results of this study suggest that these teaching methods resulted in greater generalization of language skills. In addition, other ABA approaches have emerged that specifically focus on training in the natural environment and are thus called, naturalistic approaches. Naturalistic approaches include incidental teaching (Hart & Risley, 1975, 1982), natural environment teaching (Sundberg & Partington, 1998), pivotal response training (Koegel, Koegel, Harrower, & Carter, 1999), and enhanced milieu teaching (Hancock & Kaiser, 2002). The term naturalistic is used because the stimuli, prompts, and reinforcer availability occur naturally in the student's environment. Naturalistic programming is generally considered more child-centered than contrived approaches. For example, instructors will often follow the child's lead in a play situation and look for occasions to provide instruction in language skills not yet secure in the student's repertoire. Additionally, there is an assumption with this approach that the acquired skills will generalize more readily than contrived approaches because the reinforcement for correct responding is accessible in the natural environment (Cowan & Allen, 2007; Sundberg & Partington, 1998). Finally, proponents of naturalistic approaches argue that acquired language is less robotic and rote than language acquired through a contrived approach.

There is little extant research evaluating the relative effectiveness of the approaches mentioned above. Additionally, a comprehensive search reveals no quantitative analyses comparing these approaches. Comparative reviews typically fall into two broad categories, literature reviews and meta-analyses. Literature reviews synthesize and summarize previous evaluations but offer no quantitative analyses. Whereas, a meta-analysis is a method of synthesizing, quantifying and summarizing the results from independently conducted evaluations. Glass (1976) argued that a meta-analysis is the statistical approach to review extant data that is superior to narrative reviews. He explained that meta-analysis is the "analysis of analyses."

Scruggs, Mastropieri, and Castro (1987) outlined a procedure for quantitatively evaluating the effectiveness of treatments for single-subject research. The percentage of nonoverlapping data points (PND) is a non-parametric measure that examines the difference between the baseline and treatment phases of individual studies. Kazdin (1978) suggested that evaluating non-overlapping data between baseline and intervention phases is a reliable method of evaluating treatment effectiveness.

Scruggs, Mastropieri, Forness, and Kavale (1988) conducted a meta-analysis using PND to assess early language interventions for children identified as handicapped. The investigation focused on the effects of language interventions on frequency of verbalizations. The researchers concluded that interventions which included specific generalization procedures produced more favorable outcomes than interventions involving the "train and hope" philosophy for increasing frequency of verbalizations for that population.

Other researchers have used PND to evaluate other topics related to skill acquisition and programming for individuals diagnosed with autism. Bellini, Peters, Benner, and Hopf (2007) analyzed social skills interventions for children with autism in school settings. The authors concluded that social skills interventions in schools were minimally effective. Bellini, and Akullian (2007) compared video modeling and video self-modeling for the ASD population by computing PND. They concluded that video modeling and video self-modeling were both effective for social skills acquisition and maintenance. An evaluation of the Picture Exchange Communication System (PECS) literature reported effectiveness for mand initiations (Devis & Tincani, 2008). Using the percentage of non-overlapping data (PND) method, the authors reported that of the studies that were included, PECS proved to be effective in establishing mands (i.e., functional communication).

The present meta-analysis evaluates the relative effectiveness of interventions for children with autism that lead to the acquisition of spoken language. The extant literature included in the investigation was categorized into the two broad approaches described above, contrived and naturalistic. Intervention, maintenance, and generalization effects were measured by calculating the percentage of non-overlapping data in order to answer three questions: a) do contrived approaches demonstrate a greater change from baseline to intervention as expected? b) do naturalistic approaches show comparatively greater generalization?, and c) do the two approaches differ comparatively over time?

Method

Search Criteria

Relevant articles were searched using PsycINFO and Educational Resources Information Center (ERIC) databases by using various combinations of the following search terms: analog, Asperger's, autism, autoclitic acquisition, communication, contrived, discrete trial, discrete trial instruction, discrete trial teaching, DTI, DTT, echoic acquisition, incidental learning, incidental teaching, intraverbal acquisition, language, language acquisition, mand acquisition, milieu, natural, natural environment teaching, naturalistic teaching, PDD, pervasive developmental disorder, pivotal response, pivotal response training, structured teaching, tact acquisition, and verbal behavior. In addition, manual searches of the Journal of Applied Behavior Analysis (JABA), and the Journal of Autism and Developmental Disorders were conducted in order to locate additional studies that were not captured by the original database searches. Finally, the reference sections of the studies captured by the original searches were reviewed, and all research relevant to this meta-analysis were retrieved.

Next, the first author ensured that the evaluations found from the searches described above met the focus of the study (i.e., inclusion/exclusion criteria) which was established a-priori. The inclusion/exclusion criteria was established to include only those studies that fell within the scope of the questions raised above, and to include: only research demonstrating experimental manipulations of language acquisition; only research using single-subject designs; multiple baseline, multiple probe, or ABA designs; research participants between the ages of 2 years and 13 years in order to specifically focus on children; research participants that were diagnosed with ASD, Pervasive Developmental Disorder (PDD), or Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS); and finally each study had to present a graphic display of the data in order for PND values to be calculated (see Figure 1).

Some of the most well known discrete trial evaluations (Lovaas, 1987; McEachin, Smith, & Lovaas, 1993) did not meet the inclusion criteria for the meta-analysis. These studies were excluded largely because they used a group design, rather than a single-subject design. Additional studies were excluded if they didn't specify the diagnosis of the research participants, didn't provide graphic displays of data, didn't involve measures of spoken language, or focused on research participants that were dually diagnosed.

Categorization of Included Evaluations (Independent Measures)

Each study was then classified as either "contrived", "naturalistic" or both "contrived and naturalistic". Contrived approaches (as described above) were operationally defined as highly structured teaching that consists of 1:1 instruction that occurred in an isolated location in a tightly controlled instructional environment. Studies using this approach involved the key components of a discrete trial as outlined by Koegel, Russo, and Rincover (1977). Those components include the presentation of a discriminative stimulus (Sd), a target response (or approximation), the programmatic and systematic delivery of a reinforcing consequence, and a specified inter-trial interval. In this approach, reinforcement for correct responding is not necessarily found in the student's natural environment and the instructor controls access to reinforcers (praise, tangibles, etc.). In contrived approaches, the emphasis is usually placed on the topography of the response (i.e. what the child says), and not necessarily on the function of the response.

"Naturalistic" teaching was operationally defined as a student-centered approach in which the instructor follows the student's lead, and bases instructions on the student's interests. Compared to contrived approaches, naturalistic approaches occur in a more natural setting (e.g., school, home, playground) with an array of play items available for the child to sample. The instructional arrangements are then "naturally occurring" in that the instructor waits for the student to engage in one of the play items, and then uses that item to shape or maintain verbal repertoires. For example, a naturalistic approach to teaching mands for information might include hiding a piece from a student's favorite puzzle. Once the child asks, "where is the puzzle piece?" the instructor provides information leading to the missing item. The function, rather than topography, of language is heavily emphasized in naturalistic approaches. Pivotal Response Training (PRT), Natural Environment Teaching (NET), mand training, milieu teaching, and incidental teaching are all classified as "naturalistic" for the purposes of this study.

Dependent Measures

Examples of dependent variables for contrived approaches included labeling items such as color, size, shape, and verbs (Krantz, Zalenski, Hall, Fenske, & McClannahan, 1981), and acquisition of intraverbal antonyms (Perez-Gonzalez, Garcia-Ansenjo, Williams, & Carnerero, 2007). For naturalistic approaches, dependent variables ranged from conversational utterances (Hancock & Kaiser, 2002) to peer manding (McGee, Almeida, Sulzer-Azaroff, & Feldman, 1992). Some researchers compared both contrived and naturalistic approaches with the same dependent measures. Examples include the study by Woods (1984) which compared both approaches to evaluate the effects on verbal tacting, and the Endicott and Higbee (2007) study which compared both approaches to evaluate the effects on mands for information (see Figure 1).

Data Analysis: Percentage of Non-overlapping Data Points

The PND values were calculated using the procedure outlined by Scruggs et al. (1987). That is, a ruler was horizontally placed at the highest baseline data point. Then, the number of data points in the treatment phase that exceeded that point were counted. The value of the number of data points in the treatment phase exceeding the highest baseline data point was then divided by the number of total data points in the treatment phase. That value was then multiplied by 100 in order to obtain a percentage (PND = number of treatment data points exceeding the highest baseline data point / total number of treatment data points * 100). For example, if three points in the treatment phase exceed the highest data point in the baseline, and there are four data points total in the treatment phase, the PND value is 75% (3/4 * 100 = 75%). Scruggs and Mastropieri (1998) establish cutoff criteria for evaluating quantitative data using PND: very effective treatments (over 90 PND), effective treatments (between 70 and 90 PND), questionable effectiveness (between 50 and 70 PND), and ineffective treatments (below 50 PND).

Percentage of non-overlapping data scores were calculated from baseline to intervention for each research participant, across all dependent variables, in each of the 22 studies. Additionally, the PND scores were calculated for the generalization and maintenance (i.e., follow-up) conditions when applicable by comparing the appropriate condition (i.e. generalization or maintenance) to baseline conditions (see Figure 1). Research studies that compared both types of approaches were included in the mean PND by first categorizing the interventions in each study, then calculating the values from that intervention using the procedure described above.

Inter-observer Agreement

Two researchers independently evaluated each of the 22 studies to ensure that the inclusion/exclusion criteria were met. The researchers were in 100% agreement that all studies met criteria. Next, both researchers independently classified each experiment into one of three categories (based on the operational definitions above): 1) contrived programming, 2) naturalistic programming, or 3) both contrived and naturalistic programming. The researchers agreed 100% on the categorical grouping.

Finally, PND inter-observer agreement (IOA) was calculated by dividing the number of agreements by the sum of agreements and disagreements, and multiplying that value by 100 (agreements/agreements + disagreements * 100). IOA was calculated for all intervention, generalization, and maintenance effects. The IOA for PND calculations was 90.2%.

Results

A total of 22 studies were found using the search keywords and then met the inclusion/exclusion criteria. Of the 22 studies, 21 were published in peer-reviewed journals between the years of 1981 and 2007. One doctoral dissertation met the inclusion criteria (Jennett, 2005). The 22 studies included an aggregate total of 65 research participants. The mean age of the research participants for the included studies was 6.2 years, with a range of 2 13 years (see Figure 1).

The majority of the studies included were multiple-baseline designs (n=19). Two studies used a variation of the multiple baseline design, described as multiple probe designs. One study used an ABA design. Ten studies were classified as contrived, six were classified as naturalistic, and six of the studies compared both types of interventions within the same experiment (Figure 1).

Quantitative Analysis

A total of 103 PND scores were calculated, across participants and dependent variables, for contrived interventions from the 22 included studies (Figure 1). The PND scores for contrived interventions ranged from 0--100%, with a median of 90%. The mode for this distribution was 100% (see Figure 2). The mean PND score for contrived interventions was 65% (see Figure 3).

For naturalistic interventions, a total of 49 PND scores were calculated, across participants and dependent variables, from the 22 included studies (see Figure 1). The PND scores for naturalistic interventions ranged from 0--100%. The median for the distribution of PND scores for naturalistic interventions was 98%, and the mode was 100% (see Figure 2). The mean PND value for naturalistic interventions was 83% (see Figure 3).

PND scores were also calculated to determine generalization effects for both contrived and naturalistic approaches to teaching spoken language skills. Seven of the 22 included studies involved graphic displays of generalization effects for contrived programming (see Figure 1). Within those seven studies, 35 PND scores were calculated across participants and dependent variables. The range of PND values for generalization for contrived approaches was 0--100%, with a median of 97%. The mode for this distribution was 100% (see Figure 2). The mean PND score for generalization for contrived teaching was 87% (see Figure 3).

Six of the 22 included studies provided graphic displays of generalization results for naturalistic programming (see Figure 1). Of those six research studies, 30 PND scores were calculated across subjects and dependent variables. The range of PND scores for generalization of naturalistic approaches was 0--100%, with a median of 95%. The mode for this distribution was 100% (see Figure 2). The mean PND for generalization for naturalistic approaches was 72% (see Figure 3).

Finally, PND scores were calculated for maintenance effects for both contrived and naturalistic approaches to teaching spoken language skills. Only 3 of the 22 included studies provided measures of maintenance for contrived approaches (see Figure 1). Of those three studies, 11 PND scores were calculated across subjects and dependent variables. The range of PND scores for maintenance for contrived approaches was 0--100%, and the mode was 100%. The median for this distribution was 100% (see Figure 2). The mean PND for maintenance for contrived programming was 82% (see Figure 3).

Four of the 22 included studies provided measures of maintenance for naturalistic approaches to teaching spoken language skills to children with autism (see Figure 1). Twenty PND scores were calculated from those four studies. The range of PND scores for maintenance for naturalistic approaches was 0-100%. The median PND score was 100%, and the mode was 100% (see Figure 2). The mean PND score for maintenance for naturalistic programming was 88% (see Figure 3).

Figure 3 compares the mean PND for intervention, generalization, and maintenance effects for contrived and naturalistic approaches to teaching spoken language skills. For intervention effects, contrived approaches scored a mean PND of 65%, while the mean PND for naturalistic language interventions was 83%. For generalization effects, the mean PND for contrived programming was 87%, and the mean PND for naturalistic programming was 72%. The mean PND score for maintenance for contrived approaches was 82%, compared to a mean PND of 88% for maintenance with naturalistic approaches.

Discussion

The results for this meta-analysis reveal that when comparing mean PND scores, naturalistic interventions were more effective than contrived interventions from baseline to intervention and baseline to follow-up conditions, but less effective when comparing the baseline to generalization conditions. These results were somewhat surprising. It was anticipated that contrived approaches would show the greatest change from baseline to intervention, but less change from baseline to generalization. That is, it was expected that the tightly controlled environment of the contrived settings would demonstrate a superior instructional arrangement in that the new skills would be acquired faster and with less variability than the naturalistic approaches. And, that the variability of the naturalistic approaches would promote the generalization of the newly acquired skills, after skill mastery was established, and therefore be apparent in the PND between baseline and generalization conditions comparatively. In other words, naturalistic approaches are designed to mimic conditions in which language is learned naturally, and therefore generalize to natural settings more readily. For example, the first time an infant babbles a sound that resembles "mama" in the presence of his/her mom, mom reacts favorably, and that sound is selected over others in the future. This natural contingency is deliberately mimicked in the naturalistic approaches.

The generalization effects reported for the meta-analysis suggest that the ASD population generalizes spoken language skills more readily with contrived teaching approaches, rather than naturalistic approaches. The mean PND scores for both naturalistic and contrived generalization measures were considered effective by the scale provided by Scruggs and Mastropieri (1998), although the mean for contrived generalization was 15% higher than the mean for naturalistic generalization. This finding is contrary to what the authors predicted at the onset of this analysis. It was anticipated that language skills learned through naturalistic approaches would readily generalize across settings, people, and/or stimuli, and that contrived approaches would not. The reason for this conjecture is that the instructions, stimuli, and reinforcers used in contrived interventions are highly specialized; thus, the skills learned through contrived programming would not likely generalize to natural contexts.

One possibility for the unexpected results for generalization effects found in the meta-analysis is that a bias exists in the studies that included measures of generalization for contrived teaching. The fact that researchers were testing for generalization effects could imply that programming included specific features conducive to generalization (e.g. fading contrived reinforcers for naturalistic ones; including responding in other settings in mastery criteria). At the very least, the researchers recognized the importance of generalization; hence, generalization effects were being measured. For example, Jones and Feeley (2007) focused on teaching spontaneous responses to verbal and non-verbal cues. The authors clearly describe the intervention approaches as discrete trial teaching. However, the generalization procedures consisted of fading edibles (i.e., contrived reinforcers) for more naturalistic ones, such as the teacher speaking louder in response to the student saying "what?". Generalization probes were done in a novel setting with novel teachers, and correction procedures were not used. The measures taken for generalization purposes were conducive to generalization of spoken language skills, and all contrived teaching should include such measures. In fact, the generalization procedures described in Jones et al. (2007) closely resemble naturalistic teaching, and the outcome for the research participants was favorable. Another possibility for these surprising results is that the repetition that is characteristic of contrived approaches is more conducive to generalization. In naturalistic teaching methods, the instructor often follows the student's lead and capitalizes on any teaching opportunities. It is unlikely that there are many repetitive opportunities to practice target skills in naturalistic approaches. However, repetition is intrinsic to contrived teaching approaches.

Although contrived approaches scored a higher mean PND for generalization, naturalistic approaches scored a higher mean PND for maintenance effects. Maintenance measures involve the occurrence of a response over time (Alberto & Troutman, 2006), which is a critical factor when teaching spoken language skills. Children with autism have a tendency to lose language skills over time (Goldberg, Osann, Filipek, Laulhere, Jarvis, Modhal, Flodman, & Spence, 2003). Therefore, research involving acquisition of spoken language should include maintenance conditions and measures of skill maintenance.

A possibility for stronger maintenance effects for naturalistic programming is that the naturally occurring contingencies used during intervention phases continue to be available within the environment over time. The reinforcers in a contrived teaching approach would be difficult to sustain over time, which may lead to lower rates of the target behavior. However, for this analysis, the mean PND for maintenance effects for both types of programming were considered effective according to the scale provided by Scruggs and Mastropieri (1998). The difference between the two means was 5.7%.

Future Directions

Children with autism do not readily learn spoken language without a specific intervention. Today, many interventions for children with autism begin with contrived teaching approaches during acquisition. Clinicians are more likely to move to a more naturalistic approach to train for generalization of language skills. However, the results of the current meta-analysis suggest this method is not optimal for teaching spoken language skills. These findings suggest that perhaps it would be more effective to devise language interventions utilizing naturalistic teaching approaches for acquisition with a progression to more contrived approaches to train for generalization. At the least, further investigation is warranted in order to examine the differences between naturalistic and contrived interventions for learners with autism. It may also be useful to examine different variables (i.e. attention span, stereotypical behaviors, number of established reinforcers, etc.) for this population in order to determine the most appropriate language interventions for individuals. Ultimately, clinicians and practitioners should use research in this area to develop an assessment tool to determine which teaching strategies are most beneficial for individual learners.

Another direction for future research is to investigate to what extent researchers are actually measuring generalization in studies involving spoken language acquisition. For instance, can the child respond to various instructions? Or, are the discriminative stimuli for the target response a set of highly specialized instructions? Another area of interest is for researchers to address the functional nature of spoken language learned through contrived teaching. When a child learns to say "cookie" after being asked, "what do you want?" in a contrived approach, is the word "cookie" functioning as a mand with a specific reinforcer (i.e., access to the cookie)? Or is the word "cookie" functioning as a learned intraverbal response? Another aspect of examining the function of spoken language learned through contrived programming is to address the criticisms that rote, robotic language emerges.

An additional direction for future research is to consider the intervention, generalization, and maintenance effects of various skills taught through contrived and naturalistic approaches. For instance, it would be interesting to examine if skills such as eye contact or imitation are more readily learned through naturalistic programming. It is possible that the best approach varies depending on the skill being taught.

Although the findings of this meta-analysis are important to consider when devising a language intervention for a child with autism, one must consider that outcomes can be idiosyncratic. That is, each child benefits the most from an individualized approach, based on the principles of applied behavior analysis. It is possible that a comprehensive approach is best, including aspects of both naturalistic and contrived teaching approaches (see Sundberg & Partington, 1999). However, each intervention should consist of continuous assessment, empirically validated teaching methods, and data collection. These interventions should also heavily emphasize generalization and maintenance of skills learned. As with any teaching strategy, the skill level of the practitioners is of upmost importance, as well as treatment integrity.

Limitations: Analysis Methods

A standard outcome metric for analyzing single-subject research has not been identified. PND has been criticized in the literature for not representing treatment outcomes accurately (Allison & Gorman, 1994; Levin, 1992). However, as Scruggs and Mastropieri (1987) explain, PND can be easily computed through graphic displays of data, and it is an evaluative method that provides a measure of treatment effectiveness in most cases. PND also allows for a greater number of studies to be included in the analysis than parametric approaches because there aren't constraints regarding normality, homogeneity, and independence of single-case data. Scruggs et al. (1987) remind the reader that there is not one single approach for evaluating data of singlesubject designs, just as there is not one approach for evaluating research involving group designs. PND measures are valuable because they can be easily interpreted by the reader. For the purposes of this meta-analysis, PND was the most appropriate metric for evaluating treatment effectiveness.

References

Allison, D.B., Gorman, B.S. (1994). "Make things as simple as possible, but no simpler": A rejoinder to Scruggs and Mastropieri. Behaviour Research and Therapy, 32, 885-890.

Alberto, P.A., & Troutman, A.C. (2006). Applied Behavior Analysis for Teachers, Seventh Edition. Upper Saddle River, NJ: Pearson Prentice Hall.

Autism Society of America (2008). Facts and Statistics. Retrived November 24, 2008, www.autism-society.org

Bellini, S., Peters, J.K., Benner, L., & Hopf, A. (2007). A Meta-Analysis of SchoolBased Social Skills Interventions for Children With Autism Spectrum Disorders. Remedial and Special Education, 28, 3, 153-162.

Bellini, S., & Akullian, J. (2007). A Meta-Analysis of Video Modeling and Video Self Modeling Interventions for Children and Adolescents With Autism Spectrum Disorders. Council for Exceptional Children, 73, 3, 264-287.

Buffington, D.M., Krantz, P.J., McClannahan, L.E., & Poulson, C.L. (1998). Procedures for Teaching Appropriate Gestural Communication Skills to Children with Autism. Journal of Autism and Developmental Disorders, 28, 6, 535-545.

Center for Disease Control and Prevention (2007). Autism information center. Retrieved October 26, 2008, www.cdc.gov/ncbddd/autism/

Cowan, R.J., & Allen, K.D. (2007). Using Naturalistic Procedures to Enhance Learning in Individuals with Autism: A Focus on Generalized Teaching Within the School Setting. Psychology in the Schools, 44, 7, 710-715.

Devis, K.Z., & Tincani, M. (2008). Quantitative Synthesis of Single-Subject Studies on the Picture Exchange Communication System. Manuscript submitted for publication.

Diagnostic and Statistical Manual of Metal Disorders, Fourth Edition (DSM-IV; 1994). American Psychiatric Association.

Endicott, K., & Higbee, T.S. (2007). Contriving Motivating Operations to Evoke Mands for Information in Preschoolers with Autism. Research in Autism Spectrum Disorders, 1, 210-217.

Esch, B.E., Carr, J.E., and Michael, J. (2005). Evaluating Stimulus-Stimulus Pairing and Direct Reinforcement in the Establishment of an Echoic Repertoire of Children Diagnosed with Autism. Analysis of Verbal Behavior, 21, 43-58.

Finkel, A.S., & Williams, R.L. (2001). A Comparison of Textual and Echoic Prompts on the Acquisition of Intraverbal Behavior in a Six-Year-Old Boy with Autism. Analysis of Verbal Behavior, 18, 61-70.

Glass, G.V. (1976). Primary, Secondary, and Meta-analysis of Research. Educational Researcher, 5, 3-8.

Goldberg, W.A., Osann, K., Filipek, P.A., Laulhere, T., Jarvis, K., Modahl, C., Flodman, P., & Spence, M.A. (2003). Language and Other Regression: Assessment and Timing. Journal of Autism and Developmental Disorders, 33, 6, 607-616.

Hancock, T.B., & Kaiser, A.P. (2002). The Effects of Trainer-Implemented Enhanced Milieu Teaching on the Social Communication of Children with Autism. Topics in Early Childhood Special Education, 22, 1, 39-54.

Hart, B., & Risley, T.R. (1975). Incidental Teaching of Language in the Preschool. Journal of Applied Behavior Analysis, 8, 411-420.

Hart, B., & Risley, T.R. (1982). How to use incidental teaching for elaborating language instruction. Austin, TX: PRO-ED.

Jennett, H.K. (2005). The effects of discrete trial instruction and mand training for teaching children with autism to make requests for items. Unpublished doctoral dissertation, Rutgers University, New Brunswick, NJ.

Jones, E.A., Feeley, K.M., & Takacs, J. (2007). Teaching Spontaneous Responses to Young Children with Autism. Journal of Applied Behavior Analysis, 40, 3, 565-570.

Kazdin, A.E. (1978). Methodological and Interpretive Problems of Single-Case Experimental Designs. Journal of Consulting and Clinical Psychology, 46, 629-642.

Koegel, R.L., Camarata, S., Koegel, L.K., Ben-Tall, A., & Smith, A.E. (1998). Increasing Speech Intelligibility in Children with Autism. Journal of Autism and Developmental Disorders, 28, 3, 241-251.

Koegel, L.K., Carter, C.M., Koegel, R.L. (2003). Teaching Children with Autism SelfInitiations as a Pivotal Response. Top Language Disorders, 23, 2, 134-145.

Koegel, L.K., Koegel, R.L., Harrower, J.K., & Carter, C.M. (1999). Pivotal Response Intervention I: Overview of Approach. The Journal of the Association for Persons with Severe Handicaps, 24, 3, 174-185.

Koegel, R.L., O'Dell, M.C., & Koegel, L.K. (1987). A Natural Language Teaching Paradigm for Nonverbal Autistic Children. Journal of Autism and Developmental Disorders, 17, 2, 187-200.

Koegel, R.L., Russo, D.C., & Rincover, A. (1977). Assessing and Training Teachers in the Generalized Use of Behavior Modification with Autistic Children. Journal of Applied Behavior Analysis, 10, 2, 197-205.

Krantz, P. J., Zalenski, S., Hall, L.J., Fenske, E.C., & McClannahan, L.E. (1981). Teaching Complex Language to Autistic Children. Analysis and Intervention in Developmental Disabilities, 1, 259-297.

Kroeger, K.A., & Nelson, W.M. (2006). A Language Programme to Increase the Verbal Production of a Child Dually Diagnosed with Down Syndrome and Autism. Journal of Intellectual Disability Research, 50, 2, 101-108.

Levin, J.R. (1992). Single-case research design and analysis: Comments and concerns. In T.R. Kratchowll & J.R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 213-242). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Lovaas, O.I., (1987). Behavioral Treatment and Normal Educational and Intellectual Functioning in Young Autistic Children. Journal of Consulting and Clinical Psychology, 55, 1, 3-9.

McEachin, J.J., Smith, T., & Lovaas, O.I. (1993). Long-term Outcome for Children with Autism Who Received Early Intensive Behavioral Treatment. American Journal on Mental Retardation, 97, 4, 359-372.

McGee, G.G., Almeida, C., Sulzer-Azaroff, B., & Feldman, R.S. (1992). Promoting Reciprocal Interactions Via peer Incidental Teaching. Journal of Applied Behavior Analysis, 25, 1, 117-126.

McGee, G.G., & Daly, T. (2007). Incidental Teaching of Age-Appropriate Social Phrases to Children with Autism. Research & Practice for Persons with Severe Disabilities, 32, 2, 112-123.

McGee, G.G., Krantz, P.J., & McClannahan, L.E. (1985). The Facilitative Effects of Incidental Teaching on Preposition Use by Autistic Children. Journal of Applied Behavior Analysis, 18, 1, 17-31.

Miranda-Linne, F., & Melin, L. (1992). Acquisition, Generalization, and Spontaneous Use of Color Adjectives: A Comparison of Incidental Teaching and Traditional Discrete-Trial Procedures. Research in Developmental Disabilities, 13, 191-210.

Naoi, N., Yokoyama, K., Yamamoto, J. (2007). Intervention for Tact as Reporting in Children with Autism. Research in Autism Spectrum Disorders, 1, 171-184.

Perez-Gonzalez, L.A., Garcia-Asenjo, L., Williams, & Carnerero (2007). Emergence of Intraverbal Antonyms in Children with Pervasive Developmental Disorder. Journal of Applied Behavior Analysis, 40, 4, 697-701.

Pierce, K., & Schreibman, L. (1995). Increasing Complex Social Behaviors in Children with Autism: Effects of Peer-Implemented Pivotal Response Training. Journal of Applied Behavior Analysis, 28, 3, 285-295.

Pierce, K., & Schreibman, L. (1997). Multiple Peer Use of Pivotal Response Training to Increase Social Behaviors of Classmates with Autism: Results from Trained and Untrained Peers. Journal of Applied Behavior Analysis, 30, 1, 157-160.

Sarokoff, R.A., Taylor, B.A., Poulson, C.L. (2001). Teaching Children with Autism to Engage in Conversational Exchanges: Script Fading with Embedded Textual Stimuli. Journal of Applied Behavior Analysis, 34, 1, 81-84.

Scruggs, T.E., & Mastropieri, M.A. (1998). Synthesizing Single Subject Research: Issues and Applications. Behavior Modification, 22, 221-242.

Scruggs, T.E., Mastropieri, M.A., & Castro, G. (1987). The Quantitative Synthesis of Single-Subject Research: Methodology and Validation. Remedial and Special Education, 8, 24-33.

Scruggs, T.E., Mastropieri, M.A., Forness, S.R., & Kavale, K.A. (1988). Early Language Intervention: A Quantitative Synthesis of Single-Subject Research. The Journal of Special Education, 22, 3, 259-283.

Steege, Mace, Perry, & Longnecker (2007). Applied Behavior Analysis: Beyond Discrete Trial Teaching. Psychology in the Schools, 44, 91-99.

Sundberg, M., & Partington, J. (1998). Teaching language to children with autism and other developmental disabilities. Pleasant Hill, CA: Behavior Analysts Inc.

Sundberg, M. L., & Partington, J.W. (1999). The need for both discrete trial and natural environment language training for children with autism. In P.M. Ghezzi, W.L. Willaims, & J.E. Carr (Eds.), Autism: Behavior analysticperspectives (pp. 139-156). Reno, NV: Context Press.

Taylor, B., & Harris, S.L. (1995). Teaching Children with Autism To Seek Information: Acquisition of Novel Information and Generalization of Responding. Journal of Applied Behavior Analysis, 28, 1, 3-14.

Woods, T.S. (1984). Generality In the Verbal Tacting Of Autistic Children As A Function Of "Naturalness" In Antecedent Control. Journal of Behavior Therapy & Experimental Psychiatry, 15, 1, 27-32.

Yamamoto, J, & Mochizuki, A. (1988). Acquisition and Functional Analysis of Manding with Autistic Students. Journal of Applied Behavior Analysis, 21, 1, 57-6

Author Contact Information

Meghan Kane, M.S.Ed., BCBA

meghan_kane@elwyn.org

James E. Connell, PhD., BCBA-D

jconnell@temple.edu

Melanie Pellecchia, M.S.Ed., BCBA

melaniep@temple.edu
Figure 1--Details pertaining to studies included in the meta-
analysis. PDD = pervasive developmental disorder; PDD-NOS = pervasive
developmental disorder-not otherwise specified.

Authors/Year      Research    Participants
                  Design      Number/Age/Diagnosis

Krantz,           Multiple    4 children with autism
Zalenski, Hall,   baseline    (age 7-13)
Fenske,&
McClannahan
(1981)

Woods (1984)      Multiple    2 children with autism
                  baseline    (age 6)

McGee,            Multiple    3 children with autism
Krantz, &         baseline    (age 8-11)
McClannahan
(1985)

Yamamoto, &       Multiple    3 children with autism
Mochizuki         baseline    (age 10-11)
(1988)

Miranda-          Multiple    2 children with autism
Linne, & Melin    baseline    (age 10-12)
(1992)

McGee,            Multiple    3 children with autism
Almeida,          baseline    (age 3-5)
Sulzer-
Azaroff, &
Feldman
(1992)

Pierce, &         Multiple    2 children with autism
Schreibman        baseline    (age 10)
(1995)

Taylor, &         Multiple    3 children with autism
Harris (1995)     baseline    (age 5-9)

Pierce, &         Multiple    2 children with autism
Schreibman        baseline    (age 7-8)
(1997)

Koegel,           ABA         5 children with autism
Camarata,         design*     (age 3-7)
Koegel, Ben-
Tall, & Smith
(1998)

Buffington,       Multiple    4 children with autism
Krantz,           baseline    (age 4-6)
McClannahan,
& Poulson
(1998)

Finkel, &         multiple    1 child with autism (age
Williams          baseline    6)
(2001)

Sarokoff,         multiple    2 children with autism
Taylor, &         baseline    (age 8-9)
Poulson
(2001)

Hancock, &        multiple    4 children with autism or
Kaiser (2002)     baseline    PDD (age 2-4)

Jennett           multiple    6 children with autism or
(2005)            probe       PDD-NOS (age 3-5)

Koegel,           multiple    2 children with autism
Carter, &         baseline    (age 4-6)
Koegel
(2003)

Esch, Carr, &     multiple    3 children with autism
Michael           baseline    (age 6-8)
(2005)

Endicott, &       multiple    4 children with autism
Higbee            baseline    (age 3-5)
(2007)

Perez-            multiple    2 children with autism
Gonzalez,         baseline    (age 6-8)
Garcia-
Asenjo,
Williams, &
Carnerero
(2007)

McGee, &          multiple    3 children with autism
Daly (2007)       baseline    (age 4-5)

Naoi,             multiple    3 children with autism
Yokoyama, &       baseline    (age 4-7)
Yamamoto
(2007)

Jones,            multiple    2 children with autism or
Feeley, &         probe       PDD-NOS (age 3)
Takacs
(2007)

Authors/Year      Dependent          Type of           Phases Included
                  Measures           Intervention      in Analysis

Krantz,           use of label,      contrived         Baseline
Zalenski, Hall,   color, size,                         Intervention
Fenske,&          shape, &
McClannahan       verb
(1981)

Woods (1984)      verbal             compare           Baseline
                  tacting            contrived &       Intervention
                                     naturalistic

McGee,            preposition        compare           Baseline
Krantz, &         use                contrived &       Intervention
McClannahan                          naturalistic      Generalization
(1985)

Yamamoto, &       "give me"          contrived         Baseline
Mochizuki         and "that's                          Intervention
(1988)            not it"                              Generalization
                  responses                            Maintenance

Miranda-          use of color       compare           Baseline
Linne, & Melin    adjectives         contrived &       Intervention
(1992)                               naturalistic      Maintenance

McGee,            peer               naturalistic      Baseline
Almeida,          manding                              Intervention
Sulzer-
Azaroff, &
Feldman
(1992)

Pierce, &         initiating and     naturalistic      Baseline
Schreibman        maintaining                          Intervention
(1995)            conversation                         Generalization
                  with peers                           Maintenance

Taylor, &         mands for          compare           Baseline
Harris (1995)     information        contrived &       Intervention
                                     naturalistic

Pierce, &         initiating and     naturalistic      Baseline
Schreibman        maintaining                          Intervention
(1997)            conversation                         Generalization
                  with peers                           Maintenance

Koegel,           intelligibility    compare           Baseline
Camarata,         of various         contrived &       Intervention
Koegel, Ben-      speech             naturalistic
Tall, & Smith     sounds
(1998)

Buffington,       pairing            contrived         Baseline
Krantz,           verbal                               Intervention
McClannahan,      responses                            Generalization
& Poulson         with
(1998)            gestures

Finkel, &         intraverbal        contrived         Baseline
Williams          responses                            Intervention
(2001)                                                 Maintenance

Sarokoff,         conversationa      naturalistic      Baseline
Taylor, &         l exchanges                          Intervention
Poulson
(2001)

Hancock, &        conversationa      naturalistic      Baseline
Kaiser (2002)     l utterances                         Intervention
                                                       Maintenance

Jennett           frequency of       compare           Baseline
(2005)            mands              contrived &       Intervention
                                     naturalistic      Generalization

Koegel,           correct use of     naturalistic      Baseline
Carter, &         targeted                             Intervention
Koegel            morphemes                            Generalization
(2003)

Esch, Carr, &     echoic             contrived         Baseline
Michael           responses                            Intervention
(2005)

Endicott, &       mands for          compare           Baseline
Higbee            information        contrived &       Intervention
(2007)                               naturalistic      Generalization

Perez-            use of             contrived         Baseline
Gonzalez,         intraverbal                          Intervention
Garcia-           antonyms
Asenjo,
Williams, &
Carnerero
(2007)

McGee, &          use of social      naturalistic      Baseline
Daly (2007)       phrases                              Intervention
                                                       Generalization

Naoi,             verbal tacting     contrived         Baseline
Yokoyama, &                                            Intervention
Yamamoto                                               Generalization
(2007)

Jones,            intraverbal        contrived         Baseline
Feeley, &         responses                            Intervention
Takacs                                                 Generalization
(2007)

Figure 2--Median and Mode PND scores for contrived and naturalistic
approaches to teaching lanaguage skill. Data we collected from the
distribution of intervention, generalization, and maintenance effects.
PND = Percent of Non-overlapping Data points.

                              Percent of Non-overlapping Data Points

                                Median and Mode PND scores

                                  Median      Mode

Contrived Intervention              90        100
Naturalistic Intervention           98        100
Contrived Generalization            97        100
Naturalistic Generalization         95        100
Contrived Maintenance               100       100
Naturalistic Maintenance            100       100

Note: Table made from bar graph.

Figure 3--Mean PND scores for both contrived and naturalistic
approaches to teaching language skills across intervention,
generalization, and maintenance effects. PND = Percent of
Non-overlapping Data.

                       Contrived     Naturalistic

Intervention              64.8          83.1
Generalization            87.0          71.8
Maintenance               81.8          87.5

Note: Table made from bar graph.
Gale Copyright: Copyright 2010 Gale, Cengage Learning. All rights reserved.