Early identification of behavioral and emotional problems in youth: universal screening versus teacher-referral identification.
Abstract: Universal screening is one strategy to enhance the early identification of behavioral and emotional problems among youth. Although it appears to be effective, it is unclear if universal screening is more or less effective than current teacher referral practices. Thus, the purpose of this study was to compare the effectiveness of a teacher-rated, universal screener and typical teacher-referral methods in identifying youth at risk for emotional and behavioral problems. Results indicated that of the 24 students identified as at-risk by the universal screening measure, only 11 were previously identified through current teacher referral practices - highlighting the potential benefit of universal screening to enhance early identification. Furthermore, results indicated that academic achievement and student engagement outcomes were significantly correlated with at-risk status by identification method. The strengths and limitations of this study, as well as implications for practice, are discussed.
Article Type: Report
Subject: Emotional problems of children (Research)
Child psychology (Research)
Authors: Eklund, Katie
Renshaw, Tyler L.
Dowdy, Erin
Jimerson, Shane R.
Hart, Shelley R.
Jones, Camille N.
Earhart, James
Pub Date: 01/01/2009
Publication: Name: The California School Psychologist Publisher: California Association of School Psychologists Audience: Academic Format: Magazine/Journal Subject: Psychology and mental health Copyright: COPYRIGHT 2009 California Association of School Psychologists ISSN: 1087-3414
Issue: Date: Annual, 2009 Source Volume: 14
Topic: Event Code: 310 Science & research
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 264174765
Full Text: A significant number of children are at-risk for or are currently experiencing emotional and behavioral problems (Ringel & Sturm, 2001; United States Public Health Service, 2000). The consequences for schools are clear such that emotional and behavioral problems have been well-documented to be significant barriers to learning (Catalano, Haggerty, Osterle, Fleming, & Hawkins, 2004). Children with an early onset of behavior problems are at elevated risk for academic failure, peer rejection, substance abuse, and delinquency (Reinke, Herman, Petras, & Ialongo, 2008). Furthermore, national longitudinal studies show that more than half of the students identified with emotional or behavioral problems drop out of school, 75% achieve below expected grade levels in reading, and 97% achieve below expected grade levels in math (Bradley, Doolittle, & Bartolotta, 2008). Such findings overwhelmingly indicate that emotional and behavioral problems are associated with deleterious outcomes in youth.

Despite a significant number of students experiencing emotional and behavioral problems, the majority of these students remain unidentified and consequently untreated (Ringel & Sturm, 2001; United States Public Health Service, 2000). This is detrimental to student outcomes, considering that the longer a child's emotional and behavioral problems go unidentified and untreated, the more stable his or her maladaptive trajectory is likely to be (Gottlieb, 1991). Early identification via screening is particularly important as it could help trigger early intervention, resulting in a disruption of the maladaptive trajectory. Research has also documented that recognition of a mental health problem increases the likelihood that children and their parents will seek help for that problem (Cauce, et al., 2002). In this way, universal screening efforts may ultimately lead to early intervention for students' current problems as well as prevention of future problems.

Within the school setting there is emerging evidence that early identification, combined with early and comprehensive prevention and intervention, can decrease the likelihood of academic failure and future life difficulties (Lane & Menzies, 2003; Walker & Shinn, 2002). Thus, as schools aim to serve all students regardless of risk level, through both special and general education supports, early identification via screening is a means for increasing the likelihood that more students are healthy, thriving, and progressing toward optimal development. Just as the fields of medicine and public health have repeatedly demonstrated the potential of early identification and intervention to prevent and alleviate disease and sickness (Fox, Halpern, & Forsyth, 2008), the fields of psychology and education are beginning to demonstrate that using a similar approach within schools can do the same for children's emotional and behavioral problems. Indeed, there is growing evidence that school-based screening can alter deleterious developmental trajectories and enhance positive outcomes (Report of the Alliance for School Mental Health, 2005).

A key feature of early identification is a focus on assessments that are useful for identifying progressive levels of risk among all students, not only among those with profound difficulties or problems (Glover & Albers, 2007). However, current methods of early identification vary widely, with many still focused solely on identifying students at the highest level of need. Such methods include teacher referral, parent referral for assessment and services at school or through primary care settings, youth self-referral, and universal screening. This study compares a common method for early identification-teacher referral to what may be a more underused and novel method universal screening.

Traditional Teacher Referral

Given that children spend countless hours at school, teachers are an invaluable resource for referring students in need of behavioral, emotional, and academic intervention. General education teachers are the primary link between students exhibiting problematic behavior and receiving access to school-based services. However, contemporary research indicates that teachers do not refer students based on behavior problems at the same rate as other academic concerns (Walker, Nishioka, Zeller, Severson, & Feil, 2000). In addition, many teachers perceive children's emotional and behavioral problems as someone else's responsibility (Severson, Walker, Hope-Doolittle, Kratochwill, & Gresham, 2007) - possibly contributing to lower referral rates. Furthermore, previous studies have found a significant time lag between initial symptoms and referral to services by teachers (Duncan, Forness, & Hartsough, 1995). In general, such findings suggest that teacher referral may not be the most effective method of early identification. Thus, the effectiveness of alternative early identification methods, like universal screening, warrants further examination.

Universal screening

School-based universal screening is conducted with all students in a given classroom, school, or district, to identify those at risk of academic failure and/or behavioral difficulties and may therefore benefit from intervention (e.g., Severson & Walker, 2002). This approach toward early identification allows for the provision of evidence-based prevention and early intervention services delivered through a multi-tiered intervention approach (Kratochwill, Albers, & Shernoff, 2004). It is recommended that this multi-tiered approach be accomplished via a multi-gated screening strategy. However, what remains to be resolved is whether the information obtained at each gate adds significantly to the prediction equation, justifying the additional time and resources required. In addition, key concerns regarding universal screening have been raised including the overidentification of students in need of school-based and community mental health services, as well as the potential need for multi-informant assessments of youth's behavioral problems (Levitt, Saka, Romanelli, & Hoagwood, 2007). Thus, further exploration is needed regarding the utility and feasibility of universal screening within schools.

Current study

The purpose of this study was to evaluate the ability of a universal screening measure to identify students who might otherwise go undetected through a traditional identification paradigm (i.e., teacher referral for special education, child study team, or other school-based service provisions). Utilizing data from a cohort of third- and fourth-grade students, the differences between students referred and not referred for evaluation or intervention based on the different referral systems was examined. Specifically, the aims of this study were twofold:

(1) Explore the differences between students identified as at-risk for behavioral and emotional difficulties by a universal screening measure compared to those identified via teacher referral.

(2) Examine the relationship of students' academic achievement and engagement indicators to results from the early-identification method (i.e., universal screening or teacher referral outcomes).

METHOD

Participants

Participants were 26 third-graders and 22 fourth-graders from two elementary schools in a suburban community, within the same school district, located on California's central coast. During the 2008-2009 school year, the total enrollment of one school was 286 students, and the total enrollment of the other was 421 students. During that time, the demographic make up of both schools was comparable, with approximately 73% of students identifying as Hispanic or Latino, 18% as White, and 9% as other or multiple ethnic groups. Approximately 68% of the students were classified as socioeconomically disadvantaged, 40% as English language learners, and 14% as students with disabilities. Using class-wide data collection procedures, the demographics of the participants in the present study (N = 48) were representative of the student population in these schools.

Measures

BASC-2 Behavioral and Emotional Screening System (BESS), Teacher Form. The BESS teacher form (child/adolescent version) is a screening instrument used to identify behavioral and emotional strengths and weaknesses of students in grades K-12 (Kamphaus & Reynolds, 2007). It assesses a wide range of behavioral problems and strengths, such as internalizing and externalizing problems, school problems, and adaptive skills. It is designed to be completed in 5 minutes or less. Respondents rate each item on a 4-point scale--never, sometimes, often, or almost always. The sum of the items generates a total T-score with high scores reflecting more problems. Scores of 20-60 suggests a "normal" level of risk; 61-70 suggests an "elevated" level of risk; and 71 or higher suggests an "extremely elevated" level of risk. The BESS was normed with a sample of 12,350 teacher, parents, and students, collected from 233 cities in 40 states. Results from the norming process indicated that the psychometric properties of the BESS (across all forms) are generally acceptable, having good split-half reliability (.90-.96), test-retest reliability (.80-.91), inter-rater reliability (.71-.83), sensitivity (.44-82), and specificity (.90-.97). Furthermore, the measure has demonstrated acceptable convergent validity with the Achenbach System of Empirically Based Assessment (.71-.77), Conner's Rating Scales (.51-.78), Vineland Adaptive Behavior Scales (.32-.69), Children's Depression Inventory (.51), and the Revised Children's Manifest Anxiety Scale (.55; Kamphaus & Reynolds, 2007).

Report Cards. Student report cards include academic and student engagement indicators, as graded by their teachers. The academic indicators comprised 6 total subject areas--Listening, Reading, Writing, Math, History, and Science--and corresponded to California state educational standards. Each indicator was graded on a scale of 1-4 (1 = has difficulty with standard, 2 = approaches standard, 3 = meets and applies standard, 4 = exceeds standard), indicating teachers' perceptions of students' present levels of achievement. For the purposes of this study, each subject area was conceptualized as a sub-composite, making up a total Academic Achievement composite. A behavioral engagement indicator accompanied each subject area, wherein the teachers graded the amount of "effort" students exhibited in meeting academic standards, using the same grading scale. Because these engagement indicators were unidimensional and few in number, they were summed into a total Engagement composite for the purposes of this study.

Traditional Teacher Referral Data. Additional data was collected on each student to determine which students had previously been identified by teachers as being at-risk and needing additional behavioral or emotional evaluation or intervention. Noted indicators included: (a) referral to the school's child study team, (b) testing for special education eligibility, (c) receipt of current special education services, and (d) receipt of other, non-special-education interventions (e.g., general-education counseling or classroom environment alterations). These indicators were combined as dichotomous variables and students were classified as either "at-risk" or "normal" based on teacher referral for services.

Procedures

During the first quarter of the school year, the BESS teacher form was completed for all third-grade students attending one school and for all fourth-grade students attending the other school. For each grade, screening outcomes indicated that the majority of students were in the "normal" range, several students were in the "elevated" range, and relatively few students were in the "extremely elevated" range. Thus, for the purposes of this study, students receiving the latter two classifications were grouped together, resulting in dichotomized risk-level classification outcomes: "normal" (T scores 20-60) or "at-risk" (T scores 61 and above). Screening results indicated that 20 third-graders and 13 fourth-graders had BESS outcomes in the "at-risk" range. In an attempt to create matched groups, the 13 "at-risk" third-graders were selected to participate in the study, matched with a random selection of 13 "normal" third-graders. A random selection of 13 "at-risk" fourth-graders was then conducted, matched with a random selection of 13 "normal" fourth-graders. During the course of the study, however, 2 "at-risk" fourth-graders were transferred to another school, and so the matched pairs were reduced to 13 third-graders and 11 fourthgraders in each group (N = 48).

After the sample participants were established, additional data was collected from school records and teacher interviews to establish which students were referred for additional services and previously identified by teachers as "at-risk." Next, the students' first quarter report cards - graded within a few weeks of BESS completion - were examined and coded. The Listening, Reading, Writing, Math, History, and Science sub-composites were generated and weighted by summing the indicators associated with each subject area and then dividing that total by the respective number of indicators. The Academic Achievement and Student Engagement composites were derived via the same process as the sub-composites, using their respective indicators.

Analyses

All analyses were conducted using SPSS version 17.0. BESS scores and teacher referral for services were the two variables used to create the four proposed groups (see Table 1). Students were classified as "at-risk" via the BESS according to published norms for this measure (Kamphaus & Reynolds, 2007), and dichotomized for the purposes of this study. Students were identified as "at-risk" through teacher referral methods if they met at least one of four criterions: they were either (a) referred to the school child study team, (b) tested for special education eligibility, (c) currently receiving special education services, and/or (d) receiving other, non-special education intervention services. All analyses were conducted using these aggregate grouping variables for BESS identification and teacher referral.

Participant scores on the BESS and teacher referral were used to place individuals into one of the four groups. As shown in Table 1, Group 1 is labeled as Both Identified and consists of students classified as "at-risk" by both the BESS and teacher referral. Group 2 is labeled as BESS Identified and consists of students classified as "at-risk" by the BESS but not by teacher referral. Group 3 is labeled as Teacher Identified and consists of students identified as "at-risk" through teacher referral but not identified by the BESS. Group 4 is labeled as Not Identified and consists of students not identified as "at-risk" by the BESS or via teacher referral.

RESULTS

Results indicated that 23% percent of individuals were classified as Both Identified (n = 11); 27% of students identified as at-risk by the BESS (n = 13; BESS Identified) were not concurrently identified by their teachers as needing additional services; 8% of students were only identified as at-risk by teachers (n = 4; Teacher Identified) but not by the BESS; and the remaining 42% of students (n = 20; Not Identified) represent a group that appears to be relatively free of behavioral and emotional concerns - not being identified via the BESS or teacher referral (see Table 1).

A between-subjects multivariate analysis of variance (MANOVA) was conducted to investigate the differences between groups on academic functioning and student engagement. Using the Wilks' criterion, there was a significant effect for group membership, (F = 3.88,p < .001), indicating that academic achievement and school engagement systematically differ among elementary students according to referral method. Results of follow-up analyses using Tukey's tests are shown in Table 2, along with means and standard deviations for each group on academic achievement and student engagement variables. Students in the Both Identified group had significantly lower academic achievement than students who were not identified as at-risk and those only identified as at-risk through teacher referral. Academic achievement mean differences were not significantly different among any of the remaining three groups: BESS Identified, Teacher Identified, and Not Identified. For indicators of student engagement, all students identified as at-risk by the BESS (in Both Identified and BESS Identified groups) had significantly lower scores than the Teacher Identified or Not Identified groups.

DISCUSSION

These results provide initial evidence that universal screening may identify students not previously detected, or identified earlier than typically demonstrated, by current teacher referral practices. With an equal number of students identified as "at-risk" and "normal" by the BESS as part of this matched sample, it was anticipated that teacher referral practices would reflect these same rates. However, 13 of the 24 students identified as at-risk by the BESS were not identified as at-risk by teachers. Thus, the BESS seems to have enhanced early identification potential - and possibly increased sensitivity - over teacher referral methods. Given that either the BESS or teacher referral (not both) could potentially serve as a first step in a multiple-gated early identification approach, it would be important that all students with potential emotional or behavioral risk are identified. While additional gates of assessment can further delineate which children are truly in need of services, children not identified via this initial gate are unlikely to receive the intervention services they need, and may continue untreated for a critical period of time before identification occurs. Given this conceptual framework, the results suggest that universal screening, when compared to traditional teacher referral methods, may serve as a more comprehensive method for a first-gate assessment.

Significant mean differences were found between groups on measures of academic achievement and student engagement. Students identified as at-risk by the BESS and by teacher referral (the Both Identified group) had significantly lower academic achievement scores than those not identified or those only identified by teacher referral. As the Both Identified group and BESS Identified group had statistically similar lower mean scores on academic achievement outcomes, this may further highlight the link between behavioral problems and lower academic outcomes. Students identified by the BESS (regardless of teacher referral) had lower academic achievement scores, suggesting that a universal screener may identify students at-risk for emotional, behavioral, and academic problems.

On measures of student engagement, two separate groups emerged. All students identified as atrisk by the BESS (either Both Identified or BESS Identified) had significantly lower student engagement scores than all students classified as "normal" on the BESS. The remaining Teacher Identified and Not Identified groups had significantly higher student engagement scores. These results indicate that teacher reports of student engagement levels may be an important consideration when identifying students who might be at-risk for future behavioral and emotional problems. Results obtained from a systemic screening measure, aligned with pre-established school indicators (e.g., report cards), may assist teachers and school personnel in making well-informed data-based decisions.

Strengths and Limitations

Strengths of this study include the comparison of an already established method of student referral (i.e., teacher methods) to an understudied, relatively new method of student referral (universal screening via the BESS). This study demonstrated that universal screening potentially offers a more sensitive and efficient method of identifying students at-risk for behavioral, emotional, and academic problems.

Limitations are present within the current study. The generalizability of the findings may be limited due to the small sample size, and the fact that both methods for comparison used the same rater (i.e., teacher). Further replication studies are needed with larger sample sizes and utilizing different rating forms (e.g., parent or self report) and methods (e.g., observations). Also, individual teacher-referral methods may vary widely. Some teachers refer students for minor issues, while other teachers only refer students with severe emotional or behavioral problems. The BESS may be used as one element of a screening process to help make referral methods more consistent.

Implications for Practice

Results from the present study, alongside previous results investigating screening for emotional and behavioral risk, reveal that universal screening may be a viable approach to early identification of students at-risk for behavioral, emotional, and academic problems. However, given limited financial resources, competing demands on time, and already high reporting requirements, many teachers and school districts may be skeptical of additional requests for student assessment. One approach, perhaps more acceptable to school personnel, might be to integrate universal screening for emotional and behavioral problems into schools' preexisting Response to Intervention (Rtl) paradigms. Data on emotional and behavioral problems could be collected alongside academic data to document which children are at-risk and could benefit from additional prevention or intervention. In this way, RtI and multiple-gating screening procedures could be integrated to improve the acceptability, precision, accuracy, and efficiency of early identification, intervention, and prevention. Optimally, a continuum of school-based psychological services, starting with and grounded in universal screening to identify risk levels, could support students' academic, emotional, and behavioral needs. However, as demonstrated by this study, universal screening may identify additional students not otherwise identified and may do so earlier than traditional methods. School personnel must be prepared to conduct further assessment and/or provide services for students in need.

REFERENCES

Bradley, R., Doolittle, J., & Bartolotta, R. (2008).Building on the data and adding to the discussion: The experiences and outcomes of students with emotional disturbance. Journal of Behavioral Education, 17, 4-23.

Catalano, R.F., Haggerty, K.P., Osterle, S., Fleming, C.B., & Hawkins, J.D. (2004). The importance of bonding to school for healthy development: Findings from the social development research group. Journal of School Health, 74, 252-261.

Cauce, A.M., Domenech-Rodriguez, M., Paradise, M., Cochran, B.N., Shea, J.M., Serbnik, D., & Baydar, N. (2002). Cultural and contextual influences in mental health help seeking: A focus on ethnic minority youth. Journal of Consulting and Clinical Psychology, 70(1), 44-55.

Duncan, B., Forness, S.R., & Hartsough, C. (1995). Students identified as seriously emotionally disturbed in day treatment: Cognitive, psychiatric, and special education characteristics. Behavioral Disorders, 20, 238-252

Fox, J. K., Halpern, L. F., & Forsyth, J. P. (2008). Mental health checkups for children and adolescents: A means to identify, prevent, and minimize suffering associated with anxiety and mood disorders. Clinical Psychology: Science and Practice, 15(3), 182-211.

Glover, T.A., & Albers, C.A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117-135.

Gottlieb, G. (1991). Experiential canalization of behavioral development: Theory. Developmental Psychology, 27, 4-13.

Kamphaus, R.W., & Reynolds, C.R. (2007). BASC-2 Behavioral and Emotional Screening System Manual. Circle Pines, MN: Pearson.

Kratochwill, T.R., Albers, C.A., & Shernoff, E. (2004). School-based interventions. Child and Adolescent Psychiatric Clinicals of North America, 13, 885-903.

Lane, K.L., & Menzies, H.M (2003). A school-wide intervention with primary and secondary levels of support for elementary students: Outcomes and considerations. Education and Treatment of Children, 26, 431-451.

Levitt, J.M., Saka, N., Romanelli, L.H., & Hoagwood, K. (2007). Early identification of mental health problems in schools: The status of instrumentation. Journal of School Psychology, 45, 163-191. Report of the Alliance for School Mental Health (2005). Center of Child and Adolescent Mental Health, Columbia University.

Reinke, W.M., Herman, K.C., Petras, H., Ialongo, N.S. (2008). Empirically derived subtypes of child academic and behavior problems: Co-occurrence and distal outcomes. Journal of Abnormal Child Psychology, 36, 759-770.

Ringel, J., & Sturm, R. (2001). National estimates of mental health utilization and expenditure for children in 1998. Journal of Behavioral Health Services and Research, 28, 319-332.

Severson, H.H., & Walker, H.M. (2002). Pro-active approaches for identifying children at risk for socio-behavioral problems. In. K.L. Lane, F.M. Gresham, & T.E. O'Shaughnessy (Eds.), Interventions for children with or at-risk for emotional and behavioral disorders, (pp. 33-54). Boston: Allyn & Bacon.

Severson, H.H., Walker, H.M., Hope-Doolittle, J., Kratochwill, T.R., & Gresham, F.M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193-223.

United States Public Health Service (2000). Report of the surgeon general's conference on children's mental health: A national agenda. Washington, DC: Department of Health and Human Services.

Walker, H.M., Nishioka, V.M., Zeller, R., Severson, H.H., & Feil, E.G. (2000). Causal factors and potential solutions for the persistent under-identification of students having emotional or behavioral disorders in the context of schooling. Assessment for Effective Intervention, 26, 29-40.

Walker, H.M., & Shinn, M.R. (2002). Structuring school-based interventions to achieve integrated primary, secondary, and tertiary prevention goals for safe and effective schools. In M.R. Shinn, H.M. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II: Preventative and remedial approaches, (pp. 1-25). Bethesda, MD: National Association of School Psychologists.

Correspondence may be sent to Katie Eklund, UCSB, GGSE, CCSP, Santa Barbara, CA 93106-9490 or e-mail: keklund@education.ucsb.edu or edowdy@education.ucsb.edu

Katie Eklund, Tyler L. Renshaw, Erin Dowdy, Shane R. Jimerson,

Shelley R. Hart, Camille N. Jones, and James Earhart

University of California, Santa Barbara
TABLE 1: Studen identification by referral method

                   BESS Teacher Ratings
                   At-Risk               Normal
Teacher Referral   T >60                 T <60

At-Risk            I. Both Identified    III. Teacher Identified
>1 referral

Normal             II. BESS Identified   IV. Not Identified
0 referrals

Note. BESS indicates score from the Behavior and Emotional Screening
System (Kamphaus & Reynolds, 2007); Teacher referral indicates
referral by general education teacher for additional assessment and/
or services.

TABLE 2: Mean Levels of Academic Achievement and Student Engagement by
Method of Identification (N = 48)

                         Method of Identification
                       Both Identified   BESS Identified
                       (e = 11)          (e = 13)

Dependent Variable     T           SD     T            SD

Academic Achievement   1.81 (a)     .44    2.16 (ab)    .45
Student Engagement     10.36 (a)   2.20   11.00 (a)    1.78

                       Teacher Identified   No Identified
                       (e = 4)               (e = 20)

Dependent Variable     T           SD     M            SD

Academic Achievement   2.41 (b)     .54    2.56 (b)     .30
Student Engagement     14.50 (b)   1.73   13.90 (b)    2.47

Note. Tukey comparisons were employed to analyze group means in cases
of significant F tests. Significant differences (p < .01) between
group means are indicated by different letters. Means having the same
subscript are not significantly different.
Gale Copyright: Copyright 2009 Gale, Cengage Learning. All rights reserved.