Vocational services research: recommendations for next stage of work.
Subject: Veterans (Health aspects)
Vocational rehabilitation (Analysis)
Employment (Analysis)
Authors: Drebing, Charles E.
Bell, Morris
Campinell, E. Anthony
Fraser, Robert
Malec, James
Penk, Walter
Pruitt-Stephens, Laura
Pub Date: 01/01/2012
Publication: Name: Journal of Rehabilitation Research & Development Publisher: Department of Veterans Affairs Audience: Academic Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2012 Department of Veterans Affairs ISSN: 0748-7711
Issue: Date: Jan-Feb, 2012 Source Volume: 49 Source Issue: 1
Product: Product Code: E198380 Veterans; 8336000 Vocational Rehabilitation; 9108130 Jobs & Employment; E220000 Employment NAICS Code: 62431 Vocational Rehabilitation Services; 92611 Administration of General Economic Programs SIC Code: 8331 Job training and related services
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 283834660
Full Text: INTRODUCTION

Scientific efforts to advance the practice of vocational rehabilitation (VR) for adults with the full range of disabilities have made remarkable progress over the past two decades. The number of published evaluations of vocational interventions has grown at an encouraging pace, and though the number of clinical trials has been fairly small, it is steadily increasing. Moreover, the level of methodological and statistical sophistication has improved substantially. To a significant extent, these trends have been driven by research studies seeking to carefully evaluate the use of the Individual Placement and Support model of supported employment (IPS SE) for adults with psychiatric disorders. There have been more than 20 clinical trials of IPS SE over the past 20 years, and more than 25 percent of all empirical evaluations of vocational services (VS) published in 2009 represent evaluations of IPS SE. These efforts have coincided with growing agreement among policymakers, clinicians, and researchers that clinical programming should be guided by principles of evidence-based practice (EBP) [1-2]. While IPS SE is one of the practices with a sufficient evidence base to be included on the list of EBPs, relatively few of the range of common vocational interventions have been the focus of controlled trials. Of those that have been, the populations studied have typically been a subset of the disability groups using VS.

As VS research matures, it is necessary for the field to review its progress and identify any important gaps in measurement and methodology that may hamper its ability to answer key questions. Moreover, articulating larger strategic issues may help direct future research toward particularly relevant and compelling concerns. To encourage progress, we have identified (1) ways to increase consistency in measurement of employment outcomes, (2) emerging patterns and lingering gaps in the range of variables and measures commonly used in VS research, (3) broader methodological patterns and needs in the area of study design and sampling, (4) interventions that warrant additional study, and (5) broad strategies to increase the overall amount and quality of VS research. We make recommendations in each of these five areas. These recommendations vary in terms of the audience they are relevant to and the potential benefit they may produce. Given these variations in focus, audience, and potential benefit, we have not tried to prioritize them in terms of importance but instead offer them all as potential ways to enhance different aspects of current research. Note that this article is not intended to be a comprehensive review of the literature, measures, or methodology in VS research. Instead, we have tried to highlight trends in the literature and make recommendations for the field as it moves forward. In particular, the final section of this article focusing on broad strategies for enhancing research could easily include extensive discussion of each strategy--an approach that is beyond the scope of this article.

COMPETITIVE EMPLOYMENT OUTCOMES AND MEASURES

As a field, VS profits from the relative luxury of having a single primary outcome: competitive employment. Competitive employment is defined by the Department of Labor (DOL) as work in the competitive labor market that is performed on a full-time or part-time basis in an integrated setting and for which an individual is compensated at or above the minimum wage [3]. While the stated goals of vocational interventions may be framed in a range of ways, the overall objective for most is to help participants obtain and maintain their own competitive job in the community.

The singular focus on competitive employment as the primary outcome of VS and as a key outcome for rehabilitation should continue for a number of reasons. Deterioration in functioning in competitive employment is a central element in the definition of disability (see the World Health Organization's International Classification of Functioning, Disability, and Health [4]), as evidenced by its central use in the compensation determination processes. It is also reflected in the diagnostic criteria for many of the most common disabling conditions (see Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, diagnostic criteria [5]). In the field of psychiatric rehabilitation, foundational documents such as the President's New Freedom Commission on Mental Health emphasize the primacy of helping participants resume valued roles like employment in a way that is fully integrated into the community [6]. In this framework, clinical care, and specifically those clinical efforts designed primarily to reduce clinical symptoms, serves the overall goal of helping adults move toward or maintain full integration in the community. Similarly, the current philosophical emphasis on client-centered approaches to healthcare emphasizes the need for all clinical services to serve the goals of clients [7]. A growing body of evidence indicates that most adults with disabilities want to be employed in the community [8]. Finally, growing support exists for the long-standing view of many VS providers and researchers that "work is therapy" [9-11]. Participation in employment can have important clinical benefits that rival the benefits of many common clinical interventions. These benefits are wide ranging, and while not achieved by every participant, are generally experienced by most participants across most work settings and diagnostic groups. These include the benefits of physical activity; learning and cognitive activity; social contact and engagement; enhanced opportunities to play valued social roles, including a valued family role as "provider"; a valued societal role as a "worker"; a broader sense of purpose and meaning; opportunities to use and develop skills; opportunities for distraction from clinical symptoms such as anxiety; and the indirect benefits of earning income, such as paid leisure time and employer-supported healthcare benefits [9-11]. These benefits are

rarely the primary goal of VS but rather constitute secondary benefits that bolster a continued clinical focus on employment.

What Qualifies as Competitive Employment?

Within this singular focus on competitive employment, substantive variations in definition exist that warrant review. For many studies, competitive employment is defined primarily using common DOL criteria of a "regular community job that anyone can apply for, with nondisabled coworkers, paying minimum wage or higher," as distinguished from agency-contracted community employment, employment at a business owned and run by the rehabilitation agency, and employment in a sheltered workshop [12-14]. Variations between studies occur in whether to include "casual" or intermittent employment, employment with the assistance of job coaching or other supports, and the range of self-employment options or jobs supported by other programs such as state-managed business enterprise programs. Some studies use definitions that vary from the DOL criteria in small--and sometimes large--ways, while others lack specific definitions or criteria for competitive employment [15]. Some of the most common variations fall primarily along the lines of what the minimum number of hours worked per week qualifies as competitive employment; what the minimum length of time worked is; and whether competitive employment includes day labor, seasonal labor, or other temporary work. The data used to identify who is employed also vary, including the use of VR records [16], self-report data [15], employer information, or some combination of these [12].

A wide range also exists in whether ongoing competitive employment activity is included as an outcome and how it is measured. Common approaches include the use of variables such as any employment during follow-up (yes or no); total number of hours, weeks, or months worked; annualized weeks or months worked; number of weeks with [greater than or equal to] 20 hours worked; months with [greater than or equal to] 40 hours of paid employment; days or weeks to first job obtained; mean number of jobs obtained; mean tenure per job (total number of hours, weeks, or months on the job); longest held job; average pay rate; total earned income; average weekly pay; and total number of hours worked. These variables reflect the underlying interest in key dimensions of employment, including the extent of participation in any employment, the nature of the job(s) acquired, and the degree of earned income and other benefits. Measuring ongoing employment activity is particularly important as evidence mounts that acquiring initial employment is relatively easy compared with maintaining employment, which is more integral to sustained recovery. For example, in studies of employment services for clients with traumatic brain injury (TBI), although return to work rates across samples with mixed injury severities are as high as 70 percent [17], the rate of maintaining employment for individuals with primarily moderate to severe injuries appears to be more in the range of 34 to 46 percent [18]. The emerging notion of "steady worker" as an outcome category reflects researchers' interest in documenting who maintains engagement in the role of employee across jobs versus within a specific job, both of which provide a different aspect of the outcome picture [19]. Across the field, variation in how ongoing employment is measured makes the comparison of both employment and individual job maintenance rates among studies more difficult.

While it is beyond the scope of this article to provide a complete rationale for any one definition of employment or any single approach to measuring employment across the full range of VS outcome research targeting diverse interventions and diverse disability groups, we do make the following broad recommendations for researchers, journal reviewers and editors, and grant reviewers:

1. All studies should emphasize the primacy of competitive employment outcomes in the way that they present their outcome data.

2. All future studies of vocational interventions should include the complete definition that the investigators use for competitive employment. Unless specifically justified, this definition should include the DOL criteria for competitive employment. In addition, it should also specifically state whether the following are included in the definition: (1) casual and intermittent employment activities, (including seasonal labor, temporary labor, and day labor), (2) employment with supports, and (3) range of self-employment options. It must also include specific criteria in terms of the minimum number of hours worked per week and weeks worked over the follow-up period. This allows for greater opportunities to compare results between studies. At a future point, the field may wish to convene a blue-ribbon panel to come to a consensus about one or more definitions to be used across all studies.

3. While studies may choose to define competitive employment in somewhat different ways, they should all provide sufficient data to allow the reader to identify (1) the number of jobs that are full-time, half-time or greater, and less than half-time; (2) the number of jobs that last [greater than or equal to] 3 months and <3 months; and (3) employment rates both with and without casual or intermittent employment, set-aside jobs, and productive activity other than paid employment.

4. All studies should report data on a minimum set of common employment variables reflecting job acquisition, employment activity, and earnings. This minimum data set should include (1) the mean number of jobs acquired, mean duration of each job acquired, mean pay rate, and total pay per job; (2) the total weeks worked and total hours worked during the follow-up period; and (3) the total earned income. This data set should include outcome data in terms of both the sample and the individual participant. Finally, these individual variables should also be reported for the subset of participants who obtain one or more jobs. Each of these indicators contributes a unique element to the overall outcome picture and will support comparisons between studies. Examples of tables reporting many of these variables can be found in several recent articles [12,20], and these tables may serve as a common format for reporting these outcomes.

5. The existing literature evaluating the validity of common measures of competitive employment is inadequate, including the method of data collection (self-report vs employer data vs clinician report), and additional study should be encouraged.

What Other Outcomes Beyond Competitive Employment Should be Routinely Examined and What Other Variables and Measures Included Across Vocational Intervention Outcome Studies Should be Routinely Considered?

While the singular focus on competitive employment should continue to be a central theme of VS research, evidence suggests that consistently adding additional outcomes is desirable. "Work activity" and "paid employment" [12] are terms that have been used to refer to any type of paid activities, including competitive employment, paid work in a therapeutic setting such as transitional employment, and set-aside jobs such as those found in diversified placement settings or a sheltered workshop. Relative to competitive employment, this type of work is not always fully integrated into the community, may not be "owned" by the participant, may not pay at or above minimum wage, and typically does not offer benefits such as paid vacation, health benefits, or payment into Social Security retirement. Emerging data from research with psychiatric populations suggest that these types of work activities do have some advantages that are likely relevant for the range of disability groups. Because they are in the control of providers, participants can begin engaging in work fairly quickly and can often work more and earn more than when they must first find a competitive job [20]. In Bond et al.'s randomized controlled trial (RCT) [12] comparing IPS SE and Diversified Placement Approach (DPA) services, IPS SE resulted in better competitive employment and while DPA services resulted in a higher rate of work activity with quicker onset, more weeks worked, and more income earned. Penk et al. noted a similar finding when comparing a Department of Veterans Affairs (VA) transitional work experience (TWE) intervention with job placement [20]. They point out that while TWE was supposed to be a stepping stone to competitive employment, a role it appeared to perform poorly, many participants and referral sources also saw it as a valued intervention that facilitates structured activity, social contact, and support for participants engaged in relatively intensive clinical care for substance use and psychiatric disorders. It is also a means of "acquiring immediate income for participants, many of whom are homeless isolated adults with little or no money" [20].

Some of the same benefits could be cited for volunteer activity. Involvement in volunteer roles in the community has been thought of as a stepping stone to competitive employment for those who may be ambivalent about seeking employment directly [21-23]. A small number of studies do suggest that volunteer activity has positive effects on mental health functioning by providing structure, social engagement and support, community involvement, and enhanced self-esteem [21-23]. It has been used successfully in the rehabilitation of adults with TBI and for older adults for whom there are disincentives for competitive employment [23].

Educational activities can also offer structured activity, social contact, and a meaningful social role. They may also offer opportunities for skill development as well as the chance to earn credentials that can lead to more desirable competitive employment opportunities. In this way, education can address one of the criticisms of many vocational interventions--that participants often end up working in low-wage jobs that have limited personal reward, which in turn leads to poor job tenure [24]. The development of "supported education" as a rehabilitation intervention sharing many of the elements of supported employment has helped spur interest in the inclusion of education services and goals into vocational planning [25].

Work activity, volunteer activity, and participation in education and training have traditionally been elements in some VS models. They have been justified as steps toward employment. As prior studies have noted, these activities offer the immediate benefits of structured involvement that can allow greater accommodations than many competitive jobs and can function as supportive activities for participants engaged in generally intense clinical treatment. They are also less threatening than many competitive jobs may be for many participants who may be anxious or ambivalent about competitive work [26]. Finally, they may be a valued goal in themselves for participants who want these health and therapeutic benefits but do not want a competitive job [27].

Including these activities as common outcome variables in vocational intervention evaluations does have some risk. Their inclusion may be misinterpreted by some to suggest they are equivalent in value to competitive employment, potentially leading to a reduced focus on competitive employment by the field and/or participants. With respect to participants, proponents of IPS SE have emphasized the importance of avoiding pre-employment activities such as training and education specifically because of the potential that they may delay and distract participants from job acquisition [12]. Despite the strong desire of participants for competitive employment [28], two separate studies found evidence that in some situations, intermediate work activities such as transitional employment placements can represent a distraction or disincentive to advancing to competitive employment and that participants in these activities may be less interested in pursuing competitive jobs than they were initially [20,29].

A broader philosophical issue also underlies the questions of whether to expand employment outcomes and how to value different outcomes relative to each other. The patient-centered model and the recovery model of care, two models that are highly influential in the current design of clinical services, both emphasize the centrality of client choice. While evidence exists that many potential participants in VS would like competitive employment, there is also evidence that at least in some situations, some would prefer other opportunities, such as transitional employment, education, and volunteer services [27]. While some of those sentiments reflect simple preferences, anecdotal evidence suggests that for some, those preferences reflect anxiety about potential failure in competitive jobs, given personal histories of multiple job losses [30]. If some client choices and preferences primarily reflect perceived low self-efficacy and high anxiety about failure, it is important to study how client choice functions in VS with a range of service options. Perception of self-efficacy, anxiety about failure, and other psychological barriers to employment may also respond to rehabilitation interventions (for example, cognitive behavior therapy combined with work therapy [31-32]) so that preferences may change over time.

If we look at the broader frame of reference in VS research, a wide range of baseline, moderating, mediating, and outcome variables and their respective measures can be found among intervention studies. This variety reflects a number of factors, including the nature of the sample, the focus of the intervention, the design of the study, and the ever-changing status of available outcome measures. While this variation may add depth and range to the scientific literature, it creates challenges for those comparing studies and for those attempting meta-analytic studies of treatment effects.

While allowing for scientific originality in the types of baseline measures examined (e.g., neurocognitive, social cognitive, symptom, personality, vocational, psychosocial, community function, and entitlement variables), all studies should carefully characterize their sample using commonly accepted measures appropriate to the clinical population. The intervention needs to be carefully described and fidelity and treatment integrity need to be determined. While a range of immediate and long-term outcome variables may be employed, studies should include those common variables that will make their results comparable with the broader literature.

In order to promote interpretation across studies, we have identified some common variables and measures that researchers may want to consider. The Table contains some of the variables and measures that are commonly used in VS research and have substantial evidence to support their validity. While it is not intended to be an exhaustive list, it does identify common measures that should be considered in developing new studies. There are also variables of interest for which either no valid measures could be identified or simply no measures of any kind could be identified. These include the degree to which jobs reflect participants' preferences and interests, measures of whether jobs obtained are direct or indirect products of the target intervention, and measures of treatment fidelity for many common interventions. The Figure lists some of these variables, with the hope that additional developmental research in the area of measurement can help address some of these gaps.

One growing trend in measurement has been the development and utilization of objective measures of the intervention being evaluated. Again, this has been led by research efforts to evaluate IPS SE. A clear model of IPS SE service was published in 1993 [33], followed by a measure of treatment fidelity developed and validated in 1997 [34]. That measure has been used in a wide variety of studies both to document that the interventions being evaluated were representative of the model and to examine correlates of the degree of fidelity [35]. Koop et al. subsequently developed and evaluated a measure of treatment fidelity for DPA services [36] that has been used in at least one clinical trial as well [12]. Both of these measures document fidelity at the program level. Unfortunately, no published fidelity measures exist for the other vocational interventions commonly used and no fidelity measures exist that measure the intervention received at the individual VS participant's level such that variations in intervention fidelity betweenparticipants could be examined. Published studies on interventions, such as selective or direct placement as funded by state and Federal VR programs, social enterprise, and self-employment programs [20,37] and other non-IPS SE programs, have little or no data to document the degree to which the services received by participants in the study actually match the targeted intervention model.

Fidelity is not the only variable needed to fully understand the evaluation of a vocational intervention. Studies should consider how to routinely include data regarding key intervention parameters, including measures of the time from enrollment to initiation of the intervention, the length and intensity of the intervention, the degree to which the intervention was integrated with other services, other clinical services received during the time of the intervention, provider characteristics, and the client-provider relationship.

Employment outcomes are also related to a range of environmental factors, and consistent documentation of those factors will allow for easier comparison between studies. A well-documented set of systems factors that affect employment outcomes include systems incentives and disincentives for work. While the most common of these are subsidies such as Social Security disability pensions or VA disability income, others embedded in other government benefits and clinical services have the potential to affect decisions about employment. For example, some housing programs such as the Veterans Health Administration's (VHA) Transition Residence programs require participants to be working in order to participate, resulting in participants entering employment primarily to facilitate participation in this housing program. Studies that fail to capture active contingencies of this type or program may miss key determinants of employment outcomes.

Similarly, the degree to which the vocational intervention is integrated into the larger system of clinical and social services is emerging as a key predictor of efficacy. In the IPS SE literature, integration is a key element of treatment fidelity and has been shown to be among the best predictors of employment outcomes change [38]. Data suggesting that other clinical services, service providers, and friends and families can play a key role in supporting or discouraging a return to employment underline the importance of interventions that coordinate closely with other services and communicate with the participant's larger social network [39]. For example, Gowdy et al. found that the degree of optimism about successful employment outcomes within providers and programs as a whole was predictive of the rate of successful program outcomes over and above the type of intervention being offered [40]. It seems likely that other program variables such as staffing, staff experience and turnover, program accreditation, program funding, and budget constraints may also prove to be important moderating factors that will help explain additional aspects of intervention outcomes.

If we look to a broader framework, environmental variables such as laws and governmental policies [41], healthcare systems variables [42], local unemployment rates [43], urban versus rural settings, the availability of additional VS and healthcare services, and needed resources such as transportation all likely contribute to outcome. To be effective, comparisons between studies will require a more complete description of the environment in which participants are looking for work and in which providers are assisting them.

With respect to the broader range of study variables and measures, we make the following recommendations:

1. While presenting outcome data in a way that emphasizes the primacy of competitive employment as a desired outcome, studies should routinely include distinct outcome data on the prevalence and degree of work activity, educational activity, and volunteer activity of all participants.

2. Measures of fidelity should become an expectation of all VS studies. This will require the development of new measures for some common services (see Hart et al. for tracking of standardized vocational interventions across five service sites in TBI [44]), including state and Federal VS, social enterprise, self-employment programs, transitional employment, and non-IPS SE. Development of measures of fidelity assessed at the participant level is needed.

3. Other measures of the intervention, in addition to fidelity, should be routinely included in all vocational intervention studies. Specifically, all studies should include data regarding the time to intervention initiation, length and intensity of intervention, and degree of integration of the intervention with other clinical and social services.

4. Measures of at least some key environment variables should be routinely included. Specifically, all studies should include data regarding the presence and nature of incentives and disincentives for employment, including data regarding the type of subsidies and contingencies on employment and the specific contingencies (e.g., potential earnings cap) data about the local employment market, including the local unemployment rate and data about use of additional clinical services that could substantially influence outcomes.

5. Continued efforts in measurement development and validation are needed to ensure the range of relevant variables can be included in studies using psychometrically sound measures.

Design and Sampling Issues

While RCTs have been underutilized, the trend in the literature is toward a growing use of RCTs. For a simple examination of methodology within the existing literature, we conducted searches in the PsychInfo database in 2009, 1999, and 1989 using the terms "vocational rehabilitation," "vocational services," and "supported employment." In 2009, RCTs represented 22 percent of the evaluation studies as compared with 5 percent of the studies in 1999 and 0 percent of the studies in 1989. The majority of evaluation studies continue to use nonexperimental or quasi-experimental designs, most often providing pre- and posttest data on a sample of participants in one intervention or archival data describing large numbers of participants in poorly described programs. The trend toward greater use of RCTs reflects the growing availability of funding for this type of work, as well as the increased value placed on rigorous research that can qualify interventions as EBPs.

The use of RCT designs does present challenges. For example, Macias et al. point out the importance and challenges of addressing the potential for differential initial interest in, and thus attrition from, different treatment assignments within a study with evidence that this may be a confounding factor in some past studies of IPS SE [45]. While it is true that attrition can be a meaningful outcome variable, it also has the potential to be a threat to the internal validity of a study. Careful selection of a comparison condition that is well described and carefully implemented also helps to ensure that results are meaningful. Malec points out that RCT designs are less suited to evaluate interventions that seek to modify the physical or social environment, as well as important nonspecific intervention factors such as the therapeutic alliance or optimism of the provider [46].

Despite the design challenges presented by using RCTs, the trend toward greater use, when appropriate, should continue. However, given the inherent obstacles to conducting RCTs in vocational settings (i.e., difficulties minimizing the potential for experimenter effects, ethics of withholding interventions identified as best practices despite limited experimental support), observational and other alternative designs will also continue to play an important role in the literature.

A critical need exists for continued focused study of aspects of cost and outcome of vocational interventions, as well as more frequent inclusion of cost data in the range of VS studies. There have been a number of cost-benefit studies of VS [14,47-49]. Efficiency studies can provide some of the most useful and influential data for funders and policymakers. Interventions designed to move participants from the ranks of the unemployed to the employed have some of the most tangible benefits for key stakeholders. More studies are needed to examine efficiency from the range of cost "frames," including the participants, healthcare funders and providers, government, and society. Existing studies primarily examine efficiency from the perspective of the healthcare system, leaving the other views underexplored. In general, efficiency analyses present a number of challenges, including the development of the range of relevant cost data, varing methodologies within the field, and the limited number of evaluators adequately trained to complete this type of analysis. Such analyses also need to address cost-shifting, where an intervention may appear cost-effective from one perspective (healthcare cost) but actually cost society more (shifted to criminal justice costs).

Apart from the IPS SE literature, relatively few model-testing studies compare and discuss the relative advantages of well-defined models of service in or across specific disability groups. An example of this type of approach would be the systematic review by Fadyl and McPherson [50], who identified three primary approaches to VR after brain injury (BI): (1) program-based VR, i.e., VR in the context of a comprehensive postacute rehabilitation program; (2) supported employment model; (3) case coordination model. Model 1 is exemplified by comprehensive-integrated day programs, such as those developed by BenYishay et al. [51] and Prigatano et al. [52], as well as by Community Reintegration programs [53-56]. The work of Wehman et al. [57-58] exemplifies model 2. Vocational case coordination [59-60] and resource facilitation (RF) [61] are examples of model 3. The authors document the features of these three approaches that are common and those that delineate the strategic value of each approach within a clinical context. Discussions such as this are relatively rare in the literature and should be more common as comparable data about specific models become more common.

Across published outcome studies from the past 20 years, archival studies using state and Federal administrative databases have been a common presence. For example, the RSA-911 (Rehabilitation Service Administration Case Service Report) database has been used in at least 35 studies in the past 10 years with a range of disability groups (see Bruyere and Houtenville [62] for an overview of this work), while the VHA and other databases are also being used to study entire populations of service recipients [63-64]. The advantage is the presence of existing data sets that represent all or most of the participants in commonly used programs and in specific disability subgroups. These data sets have varying availability to community researchers and have limits in terms of the range of data collected and evidence of the known validity of those data. Despite these limitations, when used carefully, they have the potential to provide important information about existing practices, and so continued efforts to explore and expand their usage is warranted.

In terms of populations and samples, it is noteworthy that over the past 20 years, the VS literature has shifted in primary focus from participants with developmental disorders to those with mental health disorders. In 2009, almost 60 percent of the published evaluations were of interventions with adults with mental health disorders compared with just 20 percent in 1989. In contrast, the largest portion of published evaluations in 1989 focused on adults with developmental disorders (40%) compared with just 10 percent in 2009. A need exists to expand the range of participant populations represented in new studies to evaluate outcomes of various interventions for adults with the range of disabilities and limitations, including the full range of mental health disorders, TBI, spinal cord injuries, sensory loss, limb loss and other physical disabilities, developmental disorders, other neurological disorders, prison re-entry populations, intimate partner violence populations, sex-offender populations, and others.

For many VR participants whose functional impairments manifest early in their adult life, their initiation into the world of the disabled begins with application for disability benefits. In the VA system, with its gradations of service-connected disability (from 0% to 100%), obtaining compensation can become an ongoing task of proving just how disabled the person can be. Veterans often describe themselves as "working on their disability." Once people receive their disability pension, it becomes a powerful disincentive to return to competitive employment. Their "disability career" can become self-perpetuating, because people with disabilities associate with each other and have fewer contacts in the working world. Studies are needed that examine these psychological processes, particularly at the time of application for disability, and interventions need to be developed that frame the pension process within a broader view of the individual's recovery through rehabilitation. While people certainly benefit from the financial security of disability compensation, the process need not propel people into disability careers.

The "treatment career framework" proposed by Hser et al. also offers a framework for understanding treatment outcome and recovery for the range of relapsing disorders common in VR settings [65]. This perspective assumes that "individuals progress through complex developmental patterns by stages within which skills, attitudes, and behaviors evolve" [65-67]. Internal factors, such as motivation to change [68], and external social forces, including family, peer, and external contingencies for specific behaviors, vary over time and have direct influence on the course and outcome of treatment careers. From this perspective, treatment entry and re-entry, relapse, recidivism, and dropout represent common predictable elements in the larger picture of recovery. While not desirable, relapse and recidivism viewed apart from the overall pattern of the treatment career should not be seen as evidence that interventions are not necessarily working. From this perspective, the key outcome is success in the larger treatment career framework.

The treatment career framework highlights the larger issue of variation in the length of follow-up. Finding a job takes time even in the best situations. If we add the need to document job maintenance after acquisition, it is clear that a significant period of time is needed to observe the degree to which participants are able to achieve the desired outcomes and how long it will last. If we consider the most salient methodological issues for vocational evaluations, we must consider inadequate follow-up periods to be one of the major concerns in relation to understanding return to work. For example, for services for adults with TBI, documented return-to-work rates range from 12 to 70 percent. Not only do these rates reflect case mix variables, but the follow-up periods vary widely, ranging from 6 weeks to 7 years [17]. Clearly, longer follow-up periods are more labor intensive, more expensive, and require longer grant funding cycles. However, they provide a better opportunity to document the development and the duration of any treatment effects produced. In two recent RCTs in which weekly employment outcome data were reported across 12 [20] and 24 [12] months of follow-up, initial trends in treatment effects did not stabilize until at least the 6-month follow-up and group differences continued to change significantly for the remainder of the follow-up period in both studies. Unfortunately, there are no established conventions for a minimum follow-up period. Of the follow-up periods for six RCTs published in 2009, two used 6 months, two used periods from 12 to 18 months, and two followed participants for [greater than or equal to] 2 years. Again, this degree of variation limits the conclusions that can be drawn across studies.

With respect to methodology, we make the following recommendations:

1. Researchers and program evaluators should continue to seek ways to use more experimental and quasi-experimental designs whenever possible while recognizing that the full range of designs will continue to be needed to answer pressing questions in this field.

2. Researchers and funders should seek ways to expand developmental research and create funding mechanisms that are well suited to this type of work. Similarly, researchers and funders should also seek ways to expand the use of archival and model-testing studies.

3. Researchers should attempt to include measures of various aspects of cost, while funders should attempt to expand funding for studies focusing explicitly on cost-benefit and cost-efficiency issues.

4. If possible, researchers and funders should broaden the range of populations studied to ensure progress in interventions for adults facing the full range of disabilities and other barriers to employment. Continued research attention is needed to address evidence of ongoing disparities in VS participation and outcomes.

5. Given the amount of time it takes to obtain employment and then demonstrate sufficient maintenance of employment, researchers should allow adequate follow-up periods to examine these variables. In general, for studies using sustained employment as a key outcome, a minimum of 24 months of follow-up should be encouraged by journals and funding sources. There should also be a concerted effort to study the long-term effects of vocational interventions over follow-up periods of at least 3 to 5 years.

6. Studies that use the frame of reference of either "disability careers" and/or "treatment careers" should be encouraged, as should developmental research on the value of these frames.

Interventions that Warrant Additional Study

The last 10 years have been a time of innovation in VS, spurred by the growing number of empirical studies documenting both the efficacy of existing models and the considerable room for improvement in even the most effective models. The next 10 years are likely to produce research that evaluates a number of adaptations of existing models, either for improvements in general outcomes or for improved services to target populations. Entirely new approaches will also be developed, evaluated, and disseminated. While most existing models have been developed within specific clinical settings and with specific populations, their efficacy with the range of different target populations needs to be explored. We discuss some of the most promising directions next.

Further Study of Individual Placement and Support Model of Supported Employment

A wide range of important questions warrants further investigation of IPS SE, including whether it can be effectively applied to populations outside adults with major mental illness, ways to enhance entry and participation, and ways to further enhance employment maintenance. A number of efforts have enhanced the existing model in order to either improve outcomes or adapt it for other populations, such as adults with TBI. Supported employment has been paired with a range of additional interventions, including cognitive rehabilitation [69-70], motivational interviewing, social skills training [71], and supported education [72]. This trend will hopefully continue in an effort to improve the outcomes and broaden the application of this well-established model. There is also a need to study how IPS SE leads to successful outcomes and what elements are most closely related to outcomes.

Customized Employment

Described by some authors as "the natural evolution of supported employment," customized employment is an emerging intervention that is beginning to develop a base of empirical support [73]. The model emphasizes an extensive job-development process to meet individualized job goals that reflect the unique needs of the employment seeker. Small caseloads reflect the effort to spend more time understanding emerging participant interests and goals as well as a key focus on employment facilitated by additional funding and support resources. The data from initial empirical evaluations are quite limited at this point [73-74].

Diversified Placement Approach and Transitional Work Experience

With the publication of fidelity guidelines for services [36], a group of common services are likely to be increasingly studied. The DPA most closely describes VS common in clubhouse settings typically serving adults with severe mental illnesses. It is also similar to VHA TWE services, which are also fairly common across the country and with a range of disability groups [20]. Existing evidence suggests that these models have been relatively ineffective at helping participants obtain competitive employment. However, they are relatively effective at helping participants engage in work activity. The value of work activity and its role in helping participants return to competitive employment is one aspect that needs further study. With 100 VHA vocational programs, including both DPA-type services alongside IPS SE, further need exists for investigation about how these services can most effectively interface.

Resource Facilitation

Developed in the field of BI rehabilitation, this approach has the potential for successful application with the full range of clinical groups. The RF [75] approach involves a coordinator providing assistance and advocacy to "break down barriers, increase access, and facilitate timely, coordinated management of resources" to return the individual with BI to full participation in family and community life [61]. RF seeks to increase access to community services and supports. A primary goal of RF is to develop a service support network that not only directly supports return to work (e.g., job search, placement, supported employment, transportation to work) but also provides a network of social support for work while giving work meaning. The RF coordinator is an advocate who assists the participant to develop a self-directed plan for community re-entry, identify needed services and supports, and develop a sustainable network of these services and supports. Building this network requires the education of both involved parties about BI. In some cases, a family member is a very able RF coordinator. However, in many cases, family members are not sufficiently knowledgeable about community systems or able to be effective advocates, and RF is best provided by a professional.

Interventions to Enhance Employers' Involvement

Major interventions used across disability groups to enhance employers' involvement include (1) liaison with employers soon after injury, (2) employer education, and (3) long-term employer support. In one study, while 80 percent of participants with BI returned to full- or part-time employment or education overall, almost 40 percent returned to their preinjury employment, although not necessarily at the same level [59]. This was orchestrated through early contact with the employer soon after injury and through maintaining this relationship over the succeeding months while the client engaged in rehabilitation to the point where return to work became a more realistic goal. Employer education has been identified as critical to vocational re-entry after BI [59,76]. Such education includes both general information dispelling employer myths about BI and specific information about the client's needs for physical and cognitive accommodations. Ongoing employer support [59,61,76-77] has also been identified as critical. Such support begins with regular follow-up that becomes increasingly less frequent as confidence in the durability of the placement increases. In the long term, a contact person remains available into the indefinite future for assistance and problem-solving as unanticipated difficulties arise.

Paid Coworkers as Trainers

This model involves the selection of a well-established senior lead- or journey-level worker to mentor the VS participant. Mentorship involves training, observation, self-management concerns, and advocacy. Coworkers are paid on an hourly basis for their training activity (e.g., before work, over lunch, on break, end of day) and receive 2 to 4 hours of training to learn training tools for both themselves and the participant. The model was developed by Curl et al. [77-78] and has been used with adults with learning and behavioral disorders, developmental disabilities, and TBI but may have promise for other disability groups as well. The model is particularly helpful when the level of work is at a semiskilled or skilled level. Inappropriate for the generic job coach, the training is only needed intermittently or in relation to a specific aspect of the job or if the company is unreceptive to a job coach.

Work Trials With or Without Pay

Work trials are time-limited job placements to assess the client's ability to succeed at a specific job and in a specific work environment. Work trials may be paid or unpaid and typically include elements of supported employment [77-78]. Work trials provide a means of assessing the client's ability to manage many aspects of the employment, such as the specific work skills required by the job, time demands and other expectations for performance, and the interpersonal and physical environment of the workplace. These latter aspects of work that are not directly related to job skills are often the most challenging for individuals with vocational problems. Since the early 1990s, it has been recognized that such on-the-job assessments are of greater value in assessing the ability of the client to succeed on the job than standardized job skill or interest assessments [79-80].

Another area likely to attract interest is the use of paid internships as a stepping stone to competitive work. In some states, VR agencies have developed interventions in which they pay a client's salary to an employer for a paid internship, involving 2 to 6 months of paid training and experience. This is done under a "good faith" agreement with the company that the intern will be hired post internship period. This arrangement provides a margin of safety for all parties and allows training and acculturation to the workplace.

Psychological Interventions to Enhance Vocational Outcomes

Preliminary research supports adding psychological interventions to VS to address relevant psychological processes that are known predictors of work performance and vocational outcome. These interventions employ neurocognitive and social cognitive retraining [69-70,81], cognitive behavior therapy that targets dysfunctional beliefs related to work [32], detailed work feedback and goal setting [82], or work-related social skills training (e.g., workplace fundamentals [83]). These interventions, alone and in combination, may improve vocational outcomes within the full range of VS participants. They also provide a more comprehensive rehabilitation approach that may synergistically increase the overall therapeutic effect of the rehabilitation experience on clinical outcomes, quality of life, and the recovery process [84].

Contingency Management Integrated with Vocational Services

Contingency management has primarily been used to enhance substance abuse treatment, but at least two RCTs have documented its efficacy at enhancing the outcomes of transitional employment [85-86]. Both job acquisition and maintenance goals were rewarded, with the result that participants were more active in job search and moved to competitive employment more quickly and at higher rates. The "therapeutic workplace" is a unique variation on this theme, using employment and a structured therapeutic work setting to reinforce abstinence among unemployed adults with substance use disorders [87]. Though substantial empirical data supports its efficacy at establishing abstinence, the model has not been applied widely [88-89].

Self-Employment and Social Enterprise Interventions

Self-employment and microenterprise development interventions have a number of advantages over interventions that result in placements in traditional jobs for VR participants. Self-employment typically focuses on jobs that more closely reflect the personal interests and skills of the individual. Self-employment also offers a greater degree of autonomy and flexibility. Adults who work for themselves have a greater ability to shift their work activities and schedules to address their other needs, including attending clinical appointments. Criminal records can represent a significant barrier to being employed in many companies, so some adults seek self-employment as a more viable means of work. Finally, self-employment also offers the potential for higher pay rates for those who are successful. In these ways, self-employment can be a means of raising the value of being employed for some people and so may lead to enhanced tenure. It does pose some risks as well, including the potential of less job stability and reliable pay, greater stress, and greater range of skill requirements. Clearly, interventions like the supported self-employment program developed and currently being evaluated within the VHA [90] designed to promote or support self-employment among populations with a range of disabilities are not the ideal service for everyone, but rather are a valuable option for a significant fraction of VR participants.

Social enterprises, also referred to as "social firms" or "affirmative businesses," are businesses created specifically to employ adults with disabilities, most commonly psychiatric disabilities. This model was developed in Europe and has spread to North America and Asia. The model is designed to provide a number of advantages, including the development of a stronger sense of community in the work setting, greater empowerment of participants, and the infusion of work with a stronger sense of personal mission. Like self-employment interventions, social enterprise interventions are just beginning to be evaluated [91].

Family and Clinical Provider Interventions

The growing evidence documenting the key role of stakeholders such as family, friends, and healthcare providers in VS outcomes has begun to spawn a range of new interventions designed to influence these stakeholders to support return-to-work efforts. Motivational interviewing interventions designed specifically to enhance support from family and friends for IPS SE are currently under evaluation [92]. Contingency management approaches that reward support for employment outcomes among vocational and nonvocational healthcare providers [93] are also being studied and may well be found to have a powerful effect on employment outcomes.

Broad Strategies to Increase Overall Amount of Research Focusing on Vocational Services

The literature examining vocational interventions has grown steadily over the past 20 years, both in terms of raw number of published studies and in degree of sophistication. IPS SE has been established as one of the most well-supported EBPs and other VS models are increasingly being evaluated and standardized. Despite these advances, a great need for additional study exists in order to improve program success rates and develop innovative new models that will better serve the needs of the full range of potential consumers. With the VHA in particular, the expansion of vocational intervention research would reflect the VHA's desire to focus its services on rehabilitation and recovery-oriented models and to expand its own VS in an EBP manner. In order to facilitate the needed growth in vocational intervention research, we make the following range of recommendations:

1. Expand research that supports ongoing efforts to help healthcare policymakers, administrators, clinicians, and consumers recognize the centrality of employment outcomes across disability groups. Earlier in this article, we identified a number of reasons that competitive employment should be a primary outcome for rehabilitation services. Those reasons also represent a substantive rationale for employment and recovery outcomes to take a more central role in healthcare. Efforts to move healthcare toward a patient-centric, recovery-oriented model necessarily elevate the focus on employment. Cost-benefit analyses from the societal perspective also highlight the importance of employment as a desirable outcome for healthcare consumers and systems. Researchers and funders could better support this effort by ensuring that questions related to this agenda remain the focus of empirical study. Compelling research findings have played a key role in changing the healthcare system's orientation, and they will likely provide a key impetus for continued change.

2. Develop a new generation of professionals interested in and capable of conducting VS research through new and existing career development funds and funded research training fellowships. The research field operates on very basic behavioral psychology principles: the efforts of new researchers are directly shaped by the reinforcers of funding and career advancement. Research funders should ensure that there are adequate incentives to gain the attention and guide the efforts of young researchers trying to establish their careers. Targeted career development awards programs and funded research fellowship programs can provide particular assistance to help young investigators gain the experience and mentoring they need to build a sustainable research career focusing on VS.

3. Increase broader opportunities for well-designed VS research by expanding targeted funding sources and developing opportunities for focused meetings and conferences for VS researchers. The National Institutes of Health, VA, and other funders of clinical research should exert the powerful influence of the purse to help expand the amount and quality of VS research. Targeted requests for applications that specifically encourage well-designed studies focusing on key strategic questions for VS for a wide range of disability groups will likely have the greatest potential for making specific advances in the field. A need also exists for more regular sharing of research results and ideas. Currently, no single natural venue exists for that type of communication, because VS researchers are spread across professions and organizations. Until such a venue develops from existing or new professional groups, organizations with a vested interest in advancing this field (VA, National Institute on Disability and Rehabilitation Research, or U.S. Psychiatric Rehabilitation Association) should develop an annual meeting specifically for VS researchers and their trainees.

4. Create research centers of excellence focusing on VS. The development of issue-focused research centers has been an effective method of advancing targeted study in a range of fields. In particular, the VA has developed a range of rehabilitation research and development Centers of Excellence, Mental Illness Research Education and Clinical Centers, Quality Enhancement Research Initiatives, and Research Enhancement Award Programs focused on important clinical issues. Unfortunately, none have been developed with a focus on VS.

5. Develop collaborative projects that involve natural VS partners. The fragmentation of providers in the VS field represents a major barrier to advancing intervention outcomes. For example, within the VA, VS provided by the Veterans Benefits Administration and VHA are rarely integrated in any clinical or research effort. Similarly, important opportunities exist for VA and other state and Federal VS to collaborate on interventions that would drive up successful outcome rates for both providers. Joint efforts would benefit not only from shared expertise but also from shared resources. This will likely require considerable political willpower and may not happen without outside facilitation, because the history of noncollaboration is long and likely reflects a range of bureaucratic barriers.

CONCLUSIONS

Employment services research has been growing and maturing as a field, resulting in greater advances in clinical practice. This progress is both a cause and a consequence of the increased recognition of the central role of employment in the process of rehabilitation and recovery. While much of the recent rigorous research has been done within the subset of mental health services, the broader field of VS research is ready for clearer coherence in shared expectations and standards for research so that the field can consolidate it gains on its way to helping greater numbers of people successfully return to rewarding jobs in the community. This article is an effort to advance this process.

JRRD at a Glance

As the field of vocational services research matures, it is necessary to review its progress and identify any important gaps in measurement and methodology that may hamper future efforts. To encourage progress, we have identified ways to increase consistency in measurement of employment outcomes, emerging patterns and lingering gaps in the range of variables and measures commonly used in vocational services research, broader methodological patterns and needs in the area of study design and sampling, interventions that warrant additional study, and broad strategies to increase the overall amount and quality of research on vocational services. The goal of this article is to assist the field in achieving clearer coherence in shared expectations and standards for research so that the field can consolidate its gains as it assists people to return successfully to rewarding jobs in the community.

ACKNOWLEDGMENTS

Author Contributions:

Study concept and design: C. E. Drebing, M. Bell, E. A. Campinell, R. Fraser, J. Malec, W. Penk, L. Pruitt-Stephens.

Drafting of manuscript: C. E. Drebing, M. Bell, E. A. Campinell, R. Fraser, J. Malec, W. Penk, L. Pruitt-Stephens.

Critical revision of manuscript for important intellectual content: C. E. Drebing, M. Bell, E. A. Campinell, R. Fraser, J. Malec, W. Penk, L. Pruitt-Stephens.

Financial Disclosures: The authors have declared that no competing interests exist.

Funding/Support: This material is based on work supported by the VHA Office of Research and Development and the New England Mental Illness Research Education and Clinical Center.

Additional Contributions: We also wish to thank Dr. Gary Bond for his support and helpful feedback in the preparation of this article.

Abbreviations: BI = brain injury, DOL = Department of Labor, DPA = Diversified Placement Approach, EBP = evidence-based practice, IPS SE = Individual Placement and Support model of supported employment, RCT = randomized controlled trial, RF = resource facilitation, TBI = traumatic brain injury, TWE = transitional work experience, VA = Department of Veterans Affairs, VHA = Veterans Health Administration, VR = vocational rehabilitation, VS = vocational services.

REFERENCES

[1.] Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn't. BMJ. 1996;312(7023):71-72. [PMID: 8555924] http://dx.doi.org/10.1136/bmj.312.7023.71

[2.] Woolf SH. Practice guidelines, a new reality in medicine. II. Methods of developing guidelines. Arch Intern Med. 1992;152(5):946-52. [PMID: 1580720] http://dx.doi.org/10.1001/archinte.1992.00400170036007

[3.] Job Training Program Act, disability grant program funded under Title III, Section 323, and Title IV, Part D, Section 452. Washington (DC): Department of Labor; 1998.

[4.] International classification of functioning, disability and health: ICF. Geneva (Switzerland): World Health Organization; 2001.

[5.] Diagnostic and statistical manual of mental disorders: Fourth edition text revision. DSM-IV-TR. Washington (DC): American Psychiatric Association; 2002.

[6.] President's new freedom commission on mental health. Washington (DC): Mental Health Commission; 2002.

[7.] Stewart M, Brown JB, Donner A, McWhinney IR, Oates J, Weston WW, Jordan J. The impact of patient-centered care on outcomes. J Fam Pract. 2000;49(9):796-804. [PMID: 11032203]

[8.] McQuilken M, Zahniser JH, Novak J, Starks RD, Olmos A, Bond GR. The work project survey: Consumer perspectives on work. J Vocat Rehabil. 2003;18(1):59-68.

[9.] Mueser KT, Becker DR, Torrey WC, Xie H, Bond GR, Drake RE, Dain BJ. Work and nonvocational domains of functioning in persons with severe mental illness: A longitudinal analysis. J Nerv Ment Dis. 1997;185(7):419-26. [PMID: 9240359] http://dx.doi.org/10.1097/00005053-199707000-00001

[10.] Bond GR, Resnick SG, Drake RE, Xie H, McHugo GJ, Bebout RR. Does competitive employment improve non-vocational outcomes for people with severe mental illness? J Consult Clin Psychol. 2001;69(3):489-501. [PMID: 11495178] http://dx.doi.org/10.1037/0022-006X.69.3.489

[11.] Siu PS, Tsang HW, Bond GR. Nonvocational outcomes for clients with severe mental illness obtaining employment in Hong Kong. J Vocat Rehabil. 2010;32:15-24.

[12.] Bond GR, Salyers MP, Dincin J, Drake R, Becker DR, Fraser VV, Haines M. A randomized controlled trial comparing two vocational models for persons with severe mental illness. J Consult Clin Psychol. 2007;75(6):968-82. [PMID: 18085913] http://dx.doi.org/10.1037/0022-006X.75.6.968

[13.] Fraser VV, Jones AM, Frounfelker R, Harding B, Hardin T, Bond GR. VR closure rates for two vocational models. Psychiatr Rehabil J. 2008;31(4):332-39. [PMID: 18407883] http://dx.doi.org/10.2975/3L4.2008.332.339

[14.] Lawer L, Brusilovskiy E, Salzer MS, Mandell DS. Use of vocational rehabilitative services among adults with autism. J Autism Dev Disord. 2009;39(3):487-94. [PMID: 18810627] http://dx.doi.org/10.1007/s10803-008-0649-4

[15.] Schneider J, Slade J, Secker J, Rinaldi M, Boyce M, Johnson R, Floyd M, Grove B. SESAMI study of employment support for people with severe mental health problems: 12-month outcomes. Health Soc Care Community. 2009;17(2):151-58. [PMID: 18800980] http://dx.doi.org/10.1111/j.1365-2524.2008.00810.x

[16.] McKay C, Johnsen M, Stein R. Employment outcomes in Massachusetts Clubhouses. Psychiatr Rehabil J. 2005; 29(1):25-33. [PMID: 16075694] http://dx.doi.org/10.2975/29.2005.25.33

[17.] Shames J, Treger I, Ring H, Giaquinto S. Return to work following traumatic brain injury: Trends and challenges. Disabil Rehabil. 2007;29(17):1387-95. [PMID: 17729084] http://dx.doi.org/10.1080/09638280701315011

[18.] Kreutzer JS, Marwitz JH, Walker W, Sander A, Sherer M, Bogner J, Fraser R, Bushnik T. Moderating factors in return to work and job stability after traumatic brain injury. J Head Trauma Rehabil. 2003;18(2):128-38. [PMID: 12802222] http://dx.doi.org/10.1097/00001199-200303000-00004

[19.] Bond GR, Drake RE, Becker DR. Beyond evidence-based practice: Nine ideal features of a mental health intervention. Res Soc Work Pract. 2010;20(5):493-501. http://dx.doi.org/10.1177/1049731509358085

[20.] Penk W, Drebing CE, Rosenheck RA, Krebs C, Van Ormer A, Mueller L. Veterans Health Administration transitional work experience vs. job placement in veterans with co-morbid substance use and non-psychotic psychiatric disorders. Psychiatr Rehabil J. 2010;33(4):297-307. [PMID: 20374988] http://dx.doi.org/10.2975/33.4.2010.297.307

[21.] Black W, Living R. Volunteerism as an occupation and its relationship to health and wellbeing. Brit J Occupation Ther. 2004;67(12):526-32.

[22.] Ouellet MC, Morin CM, Lavoie A. Volunteer work and psychological health following traumatic brain injury. J Head Trauma Rehabil. 2009;24(4):262-71. [PMID: 19625865] http://dx.doi.org/10.1097/HTR.0b013e3181a68b73

[23.] Warburton J, Peel NM. Volunteering as a productive ageing activity: The association with fall-related hip fracture in later life. Eur J Ageing, 2008;5(2):129-36. http://dx.doi.org/10.1007/s10433-008-0081-9

[24.] Hagner D. Primary and secondary labor markets: Implications for vocational rehabilitation. Rehabil Couns Bull. 2000;44(1):22-29. http://dx.doi.org/10.1177/003435520004400104

[25.] Gutman SA, Kerner R, Zombek I, Dulek J, Ramsey CA. Supported education for adults with psychiatric disabilities: Effectiveness of an occupational therapy program. Am J Occup Ther. 2009;63(3):245-54. [PMID: 19522133] http://dx.doi.org/10.5014/ajot.63.3.245

[26.] Drebing CE, Hebert M, Mueller LN, Van Ormer EA, Herz L. Vocational rehabilitation from a behavioral economics perspective. Psychol Serv. 2006;3(3):181-94. http://dx.doi.org/10.1037/1541-1559.3.3.181

[27.] Drebing CE, Van Ormer EA, Schutt RK, Krebs C. Client goals for vocational rehabilitation: Distribution, sources, and consequences. Psychiatr Rehabil J. 2004;47(3):162-72.

[28.] Bedell JR, Draving D, Parrish A, Gervey R, Guastadisegni P. A description and comparison of experiences of people with mental disorders with supported employment and paid prevocational training. Psychiatr Rehabil J. 1998;21(3): 279-83.

[29.] McGurk SR, Mueser KT, DeRosa TJ, Wolfe R. Work, recovery, and comorbidity in schizophrenia: A randomized controlled trial of cognitive remediation. Schizophr Bull. 2009;35(2):319-35. [PMID: 19269925] http://dx.doi.org/10.1093/schbul/sbn182

[30.] Drebing CE. Paying people to get better: Incentive enhancement and vocational rehabilitation. Proceedings of the Annual Convention of the American Psychological Association; 2005; Washington, DC.

[31.] Lysaker PH, Bond G, Davis LW, Bryson GJ, Bell MD. Enhanced cognitive-behavioral therapy for vocational rehabilitation in schizophrenia: Effects on hope and work. J Rehabil Res Dev. 2005;42(5):673-82. [PMID: 16586193] http://dx.doi.org/10.1682/JRRD.2004.12.0157

[32.] Lysaker PH, Davis LW, Bryson GJ, Bell MD. Effects of cognitive behavioral therapy on work outcomes in vocational rehabilitation for participants with schizophrenia spectrum disorders. Schizophr Res. 2009;107(2-3):186-91. [PMID: 19046856] http://dx.doi.org/10.1016/j.schres.2008.10.018

[33.] Becker DR, Drake RE. A working life: The Individual Placement and Support (IPS) program. Concord (NH): New Hampshire-Dartmouth Psychiatric Research Center; 1993.

[34.] Bond GR, Becker DR, Drake RE, Vogler KM. A fidelity scale for the individual placement and support model of supported employment. Rehabil Couns Bull. 1997;40(4): 265-84.

[35.] Bond GR, Drake RE, Becker DR. An update on randomized controlled trials of evidence-based supported employment. Psychiatr Rehabil J. 2008;31(4):280-90. [PMID: 18407876] http://dx.doi.org/10.2975/3L4.2008.280.290

[36.] Koop JI, Rollins AL, Bond GR, Salyers MP, Dincin J, Kinley T, Shimon SM, Marcelle K. Development of the DPA Fidelity Scale: Using fidelity to define an existing vocational model. Psychiatr Rehabil J. 2004;28(1):16-24. [PMID: 15468632] http://dx.doi.org/10.2975/28.2004.16.24

[37.] Chandler D, Levin S, Barry P. The menu approach to employment services: Philosophy and five-year outcomes. Psychosoc Rehabil J. 1999;23(1):24-33.

[38.] Cook JA, Blyler CR, Leff HS, McFarlane WR, Goldberg RW, Gold PB, Mueser KT, Shafer MS, Onken SJ, Donegan K, Carey MA, Kaufmann C, Razzano LA. The employment intervention demonstration program: Major findings and policy implications. Psychiatr Rehabil J. 2008; 31(4):291-95. [PMID: 18407877] http://dx.doi.org/10.2975/3L4.2008.291.295

[39.] Conley R, Conroy JW. The Florida Freedom Initiative on employment of people with significant disabilities: Lessons to be learned. J Vocation Rehabil. 2009;31(1):19-27.

[40.] Gowdy EA, Carlson LS, Rapp CA. Organizational factors differentiating high performing from low performing supported employment programs. Psychiatr Rehabil J. 2004; 28(2):150-56. [PMID: 15605751] http://dx.doi.org/10.2975/28.2004.150.156

[41.] Shankar J. Policy barriers to the employment of people experiencing psychiatric disabilities. Soc Work Ment Health. 2008;7(1-3):271-82. http://dx.doi.org/10.1080/15332980802072603

[42.] Campbell K, Bond GR, Gervey R, Pascaris A, Tice S, Revel G. Does type of provider organization affect fidelity to evidence-based supported employment? J Vocat Rehabil. 2007;27(1):3-11.

[43.] Cook JA, Mulkern V, Grey DD, Burke-Miller J, Blyler CR, Razzano LA, Onken SJ, Balser RM, Gold PB, Shafer MS, Kaufmann CL, Donegan K, Chow CM, Steigman PA. Effects of local unemployment rate on vocational outcomes in a randomized trial of supported employment for individuals with psychiatric disabilities. J Vocat Rehabil. 2006; 25(2):71-84.

[44.] Hart T, Dijkers M, Whyte J, Braden C, Trott CT, Fraser R. Vocational interventions and supports following job placement for persons with traumatic brain injury. J Vocat Rehabil. 2010;32(3):135-50.

[45.] Macias C, Gold PB, Hargreaves WA, Aronson E, Bickman L, Barreira PJ, Jones DR, Rodican CF, Fisher WH. Preference in random assignment: Implications for the interpretation of randomized trials. Adm Policy Ment Health. 2009;36(5):331-42. [PMID: 19434489] http://dx.doi.org/10.1007/s10488-009-0224-0

[46.] Malec JF. Ethical and evidence-based practice in brain injury rehabilitation. Neuropsychol Rehabil. 2009;19(6): 790-806. [PMID: 19626559] http://dx.doi.org/10.1080/09602010903031203

[47.] Cimera RE. The outcomes and costs of public vocational rehabilitation of consumers with mental illness. J Appl Rehabil Counsel. 2009;40(2):28-33.

[48.] Bush PW, Drake RE, Xie H, McHugo GJ, Haslett WR. The long-term impact of employment on mental health service use and costs for persons with severe mental illness. Psychiatr Serv. 2009;60(8):1024-31. [PMID: 19648188] http://dx.doi.org/10.1176/appi.ps.60.8.1024

[49.] Chalamat M, Mihalopoulos C, Carter R, Vos T. Assessing cost-effectiveness in mental health: Vocational rehabilitation for schizophrenia and related conditions. Aust N Z J Psychiatry. 2005;39(8):693-700. [PMID: 16050923] http://dx.doi.org/10.1080/j.1440-1614.2005.01653.x

[50.] Fadyl JK, McPherson KM. Approaches to vocational rehabilitation after traumatic brain injury: A review of the evidence. J Head Trauma Rehabil. 2009;24(3):195-212. [PMID: 19461367] http://dx.doi.org/10.1097/HTR.0b013e3181a0d458

[51.] Ben-Yishay Y, Silver SM, Piasetske E, Rattok J. Relationship between employability and vocational outcome after intensive holistic cognitive rehabilitation. J Head Trauma Rehabil. 1987;2(1):35-48. http://dx.doi.org/10.1097/00001199-198703000-00007

[52.] Prigatano GB, Klonoff PS, O'Brien KP, Altman IM, Amin K, Ciapello DA. Productivity after neuropsychological oriented milieu rehabilitation. J Head Trauma Rehabil. 1994;9(1):91-102. http://dx.doi.org/10.1097/00001199-199403000-00011

[53.] Cope DN, Cole JR, Hall KM, Barkan H. Brain injury: Analysis of outcome in a post-acute rehabilitation system. Part 1: General analysis. Brain Inj. 1991;5(2):111-25. [PMID: 1873600] http://dx.doi.org/10.3109/02699059109008083

[54.] Cope DN, Cole JR, Hall KM, Barkan H. Brain injury: Analysis of outcome in a post-acute rehabilitation system. Part 2: Subanalyses. Brain Inj. 1991;5(2):127-39. [PMID: 1908341] http://dx.doi.org/10.3109/02699059109008084

[55.] Evans RW, Jones ML. Integrating outcomes, value, and quality. An outcome validation system for post-acute rehabilitation programs. J Insur Med. 1991;23(3):192-96. [PMID: 10147766]

[56.] Mills VM, Nesbeda T, Katz DI, Alexander MP. Outcomes for traumatically brain-injured patients following post-acute rehabilitation programmes. Brain Inj. 1992;6(3):219-28. [PMID: 1581745] http://dx.doi.org/10.3109/02699059209029663

[57.] Wehman P, West M, Fry R, Sherron P, Groah C, Kreutzer J, Sale P. Effect of supported employment on the vocational outcomes of persons with traumatic brain injury. J Appl Behav Anal. 1989;22(4):395-405. [PMID: 2515185] http://dx.doi.org/10.1901/jaba.1989.22-395

[58.] Wehman P, West M, Sherron P, Roah C, Kreutzer J. Return to work: Supported employment strategies, costs, and outcome data. In: Thomas DF, Menz FE, McAless DC, editors. Community-based employment following traumatic brain injury. Menomonie (WI): University of Wisconsin Stout; 1993.

[59.] Malec JF, Buffington AL, Moessner AM, Degiorgio L. A medical/vocational case coordination system for persons with brain injury: An evaluation of employment outcomes. Arch Phys Med Rehabil. 2000;81:1007-15. [PMID: 10943747] http://dx.doi.org/10.1053/apmr.2000.6980

[60.] Malec JF, Moessner AM. Replicated positive results for the VCC model of vocational intervention after ABI within the social model of disability. Brain Inj. 2006;20(3):227-36. [PMID: 16537264] http://dx.doi.org/10.1080/02699050500488124

[61.] Trexler LE, Trexler LC, Malec JF, Klyce D, Parrott D. Prospective randomized controlled trial of resource facilitation on community participation and vocational outcome following brain injury. J Head Trauma Rehabil. 2010;25(6): 440-46. [PMID: 20220530] http://dx.doi.org/10.1097/HTR.0b013e3181d41139

[62.] Bruyere SM, Houtenville AJ. Use of statistics from national data sources to inform rehabilitation program planning, evaluation, and advocacy. Rehabil Counsel Bull. 2006;50(1):46-58. http://dx.doi.org/10.1177/00343552060500010601

[63.] Resnick SG, Drebing CE, Rosenheck R. What makes vocational rehabilitation effective?: Program characteristics and employment outcomes nationally in VA. Psychol Serv. 2006; 3(4):239-48. http://dx.doi.org/10.1037/1541-1559.3.4.239

[64.] Greenberg GA, Rosenheck R. Compensation of veterans with psychiatric or substance abuse disorders and employment and earnings. Mil Med. 2007;172(2):162-68. [PMID: 17357771]

[65.] Hser YI, Evans E, Huang D, Anglin DM. Relationship between drug treatment services, retention, and outcomes. Psychiatr Serv. 2004;55(7):767-74. [PMID: 15232015] http://dx.doi.org/10.1176/appi.ps.55.7.767

[66.] Humphreys K, Moos RH, Cohen C. Social and community resources and long-term recovery from treated and untreated alcoholism. J Stud Alcohol. 1997;58(3):231-38. [PMID: 9130214]

[67.] McLellan AT, Lewis DC, O'Brien CP, Kleber HD. Drug dependence, a chronic medical illness: Implications for treatment, insurance, and outcomes evaluation. JAMA. 2000; 284(13):1689-95. [PMID: 11015800] http://dx.doi.org/10.1001/jama.284.13.1689

[68.] Prochaska JO, DiClemente CC. Toward a comprehensive model of change. In: Miller WR, Heather N, editors. Treating addictive behaviors: Processes of change. New York (NY): Plenum Press; 1986.

[69.] Bell MD, Zito W, Greig T, Wexler BE. Neurocognitive enhancement therapy with vocational services in schizophrenia: Work outcomes at two year follow-up. Schizophr Res. 2008;105(1-3):18-29. [PMID: 18715755] http://dx.doi.org/10.1016/j.schres.2008.06.026

[70.] McGurk SR, Mueser KT, Pascaris A. Cognitive training and supported employment for persons with severe mental illness: One-year results from a randomized controlled trial. Schizophr Bull. 2005;31(4):898-909. [PMID: 16079391] http://dx.doi.org/10.1093/schbul/sbi037

[71.] Chan AS, Tsang HW, Li SM. Case report of integrated supported employment for a person with severe mental illness. Am J Occup Ther. 2009;63(3):238-44. [PMID: 19522132] http://dx.doi.org/10.5014/ajot.63.3.238

[72.] Rudnick A, Gover M. Combining supported education with supported employment. Psychiatr Serv. 2009;10(12):1690. [PMID: 19952164] http://dx.doi.org/10.1176/appi.ps.60.12.1690

[73.] Griffin C, Hammis D, Geary T, Sullivan M. Customized employment: Where we are; where we're headed. J Vocation Rehabil. 2008;28(3):135-39. http://dx.doi.org/10.1080/10826080701202403

[74.] Magura S, Blankertz L, Madison EM, Friedman E, Gomez A. An innovative job placement model for unemployed methadone patients: A randomized clincal trial. Subst Use Misuse. 2007;42(5):811-28. [PMID: 17613946]

[75.] Connors SH. Resource facilitation: A consensus of principles and best practices to guide program development and operation in brain injury. McLean (VA): Brain Injury Association of America; 2001.

[76.] Malec J. Vocational rehabilitation. In: High WM, editor. Rehabilitation for traumatic brain injury. New York (NY): Oxford University Press; 2005.

[77.] Curl RM, Chisholm L. Unlocking co-worker potential in competitive employment: Keys to a cooperative approach. J Vocation Rehabil. 1993;3:72-84.

[78.] Curl RM, Fraser RT, Cook RG, Clemmons D. Traumatic brain injury vocational rehabilitation: Preliminary findings for the co-worker as trainer project. J Head Trauma Reha bil. 1996;11:75-85. http://dx.doi.org/10.1097/00001199-199602000-00009

[79.] Thomas DF, Menz FE. Development of a model of community-based employment for persons with traumatic brain injury. In: Thomas DF, Menz FE, McAlees D, editors. Community-based employment following traumatic brain injury. Menomonie (WI): University of Wisconsin-Stout; 1993.

[80.] Corthell DW. Traumatic brain injury and vocational rehabilitation. Menomonie (WI): University of Wisconsin-Stout; 1990.

[81.] Bell MD, Bryson GJ, Greig TC, Fiszdon JM, Wexler BE. Neurocognitive enhancement therapy with work therapy: Productivity outcomes at 6-month and 12-month follow-up. J Rehabil Res Dev. 2005;42:829-38. [PMID: 16680620] http://dx.doi.org/10.1682/JRRD.2005.03.0061

[82.] Bell M, Lysaker PH, Bryson GA. A behavioral intervention to improve work performance in schizophrenia: Work behavior inventory feedback. J Vocation Rehabil. 2003; 18(1):43-51.

[83.] Liberman RP. Recovery from disability: Manual of psychiatric rehabilitation. Washington (DC): American Psychiatric Association Press; 2008.

[84.] Bell MD, Choi J, Lysaker P. Psychological intervention to improve work outcomes for people with psychiatric disabilities. J Norwegian Psychol Ass. 2007;44:606-17.

[85.] Drebing CE, Van Ormer EA, Krebs C, Rosenheck R, Rounsaville B, Herz L, Penk W. The impact of enhanced incentives on vocational rehabilitation outcomes for dually diagnosed veterans. J Appl Behav Anal. 2005;38(3):359-72. [PMID: 16270845] http://dx.doi.org/10.1901/jaba.2005.100-03

[86.] Drebing CE, Van Ormer EA, Mueller L, Hebert M, Penk WE, Petry NM, Rosenheck R, Rounsaville B. Adding a contingency management intervention to vocational rehabilitation: Outcomes for dually diagnosed veterans. J Rehabil Res Dev. 2008;44(6):851-65. [PMID: 18075942] http://dx.doi.org/10.1682/JRRD.2006.09.0123

[87.] Wong CJ, Silverman K. Establishing and maintaining job skills and professional behaviors in chronically unemployed drug abusers. Subst Use Misuse. 2007;42(7):1127-40. [PMID: 17668329] http://dx.doi.org/10.1080/10826080701407952

[88.] Wong CJ, Dillon EM, Sylvest CE, Silverman K. Contingency management of reliable attendance of chronically unemployed substance abusers in a therapeutic workplace. Exp Clin Psychopharmacol. 2004;12(1):39-46. [PMID: 14769098] http://dx.doi.org/10.1037/1064-1297.12.L39

[89.] Silverman K, Wong CJ, Needham M, Diemer KN, Knealing T, Crone-Todd D, Fingerhood M, Nuzzo P, Kolodner K. A randomized trial of employment-based reinforcement of cocaine abstinence in injection drug users. J Appl Behav Anal. 2007;40(3):387-410. [PMID: 17970256] http://dx.doi.org/10.1901/jaba.2007.40-387

[90.] Pinsk, J, Krebs C, Tabol C, Drebing CE. Improving vocational rehabilitation outcomes through supported self employment: An empirical evaluation of a new model of vocational rehabilitation. Proceedings of the VA Conference: Transforming Mental Health Care: Promoting Recovery and Integrated Care; 2008; Arlington, VA.

[91.] Warner R, Mandiberg J. An update on affirmative businesses or social firms for people with mental illness. Psychiatr Serv. 2006;57(10):1488-92. [PMID: 17035570] http://dx.doi.org/10.1176/appi.ps.57.10.1488

[92.] Mueller L, Rose G. Strategies for final success in rehabilitation services. Proceedings of the 117th Annual Meeting of the American Psychological Association; 2008 Aug 14-17; Boston, MA.

[93.] Noone W. Community Based Employment Services (CBES) benefits planning & outreach assessments IMPACT. Boston (MA): Massachusetts Rehabilitation Commission, Statewide Employment Services Department; 2002.

Submitted for publication June 1, 2010. Accepted in revised form April 14, 2011.

This article and any supplementary material should be cited as follows:

Drebing CE, Bell M, Campinell EA, Fraser R, Malec J, Penk W, Pruitt-Stephens L. Vocational services research: Recommendations for next stage of work. J Rehabil Res Dev. 2012;49(1):101-20.

http://dx.doi.org/10.1682/JRRD.2010.06.0105

Charles E. Drebing, PhD; (1) * Morris Bell, PhD; (2) E. Anthony Campinell, PhD; (3) Robert Fraser, PhD; (4) James Malec, PhD; (5) Walter Penk, PhD; (6) Laura Pruitt-Stephens, Med (7)

(1) Psychology Service, Bedford Department of Veterans Affairs Medical Center, Bedford, MA; (2) Department of Psychiatry, Yale University, West Haven, CT; (3) Veterans Health Administration, Office of Mental Health Services, Lowell, MA; (4) Department of Neurology, Neurological Surgery, and Rehabilitation Medicine, University of Washington, Seattle, WA; (5) Indiana University School of Medicine and Rehabilitation Hospital of Indiana, Indianapolis, IN; (6) College of Medicine, Texas A&M University Health Science Center, Bryan, TX; (7) Texas A&M University, College Station, TX

* Address all correspondence to Charles E. Drebing, PhD; Bedford VAMC--Psychology, 200 Springs Rd (116B), Bedford, MA 01730; 781-687-2462; fax: 781-687-2169. Email: charles.drebing@va.gov

doi: 10.1682/JRRD.2010.06.0105

* Drebing CE, Mueller L, Van Ormer EA, Rose GS, Crowder S, Rosenheck R, Drake R, King K, Penk W. Outcome of single-session motivational interviewing to enhance entry and employment in VHA Vocational Services. Psychol Serv. Forthcoming.
Figure.
Gaps in existing measures. DPA = Diversified Placement
Approach, IPS SE = Individual Placement and Support model
of supported employment.

Treatment fidelity: Non-IPS SE.
Treatment fidelity: Non-DPA transitional employment.
Treatment fidelity: Common state/Federal vocational rehabilitation
  services.
Treatment fidelity: Social enterprise and self-employment programs.
Treatment fidelity assessed at individual participant level
 (all models of vocational services).
Degree to which vocational services received was integrated in
  clinical care.
Negative effects of employment.
Family support for return to work.
Clinician support for return to work.
Job search skill.
Job search self efficacy.
Interest in self-employment.
Skills in self-employment.
Participant's perceived link between intervention received and job
  obtained.
Social reciprocity as incentive for work.


Table.
Common variables and measures utilized in vocational studies.

        Variable                      Measure

Work and Work History      Vocational Update Form [1]
Occupational Categories    Standard Occupational
                             Classification 2000 [2]
Motivation and Value       Motivation and Value on
   on Work                   Work Scales [3]
Work Behaviors             Work Behavior Inventory [4]
Work Ability               Dialogue About Ability
                             Related to Work [5]
Work Skills                Assessment of Work
                             Performance [6-7]
Job or VS Satisfaction     Indiana Job Satisfaction
                             Scale [8]
                           Vocational Rehabilitation
                             Consumer Satisfaction
                             Measure [9]
Treatment Fidelity:        Quality of Supported
  IPS SE                     Employment Implementation
                             Scale [10]
Treatment Fidelity:        DPA Fidelity Scale [11]
  DPA
Psychiatric Symptoms       Positive and Negative
                             Syndrome Scale [12]
Substance Use              Alcohol Use Scale and Drug
                             Use Scale [13]
                           Addiction Severity Index [14]
                           Timeline Follow-Back [15]
Substance Use Treatment    Substance Abuse Treatment
                             Scale [13]
Quality of Life            36-Item Short Form Health
                             Survey, 12-Item Short Form
                             Health Survey [16-17]
Hope/Optimism              Herth Hope Index [18]
Empowerment                Boston University Consumer
                             Empowerment Scale [19]
Alliance Between Client    Working Alliance Scale [20]
  and Provider
Access, Supports, and      Pathways Inventory [21-22]
  Barriers to Work
  or VR

[1.] Drake RE, McHugo GJ, Becker DR, Anthony WA, Clark RE. The New
Hampshire study of supported employment for people with severe
mental illness. J Con- sult Clin Psychol. 1996;64(2):391-99.

[2.] Standard occupational classification. Vol 1. Washington (DC):
Bureau of Labor Statistics; 2000.

[3.] Mor-Barak ME, Scharlach AE, Birba L, Sokolov J. Employment,
social networks, and health in the retirement years. Int J Aging
Hum Dev. 1992;35(2):145-59.

[4.] Bell M, Lysaker PH, Bryson GA. A behavioral intervention to
improve work performance in schizophrenia: Work behavior inventory
feedback. J Vocation Rehabil. 2003;18(1):43-51.

[5.] Linddahl I, Norrby E, Bellner AL. Construct validity of the
instrument DOA: A dialogue about ability related to work. Work.
2003;20(3):215-24.

[6.] Sandqvist JL, Bjork MA, Gullberg MT, Henriksson CM, Gerdle BU.
Construct validity of the Assessment of Work Performance (AWP).
Work. 2009;32(2):211-18.

[7.] Sandqvist JL, Tornquist KB, Henriksson CM. Assessment of Work
Performance (AWP)--Development of an instrument. Work.
2006;26(4):379-87.

[8.] Resnick SG, Bond GR. The Indiana Job Satisfaction Scale: Job
satisfaction in vocational rehabilitation for people with severe
mental illness. Psychiatr Rehabil J. 2001;25:12-19.

[9.] Capella ME, Turner RC. Development of an instrument to measure
consumer satisfaction in vocational rehabilitation. Rehabil Counsel
Bull. 2004;47(2):76-85.

[10.] Bond GR, Becker DR, Drake RE, Vogler KM. A fidelity scale for
the individual placement and support model of supported employment.
Rehabil Couns Bull. 1997;40(4):265-84.

[11.] Koop JI, Rollins AL, Bond GR, Salyers MP, Dincin J, Kinley T,
Shimon SM, Marcelle K. Development of the DPA Fidelity Scale: Using
fidelity to define an existing vocational model. Psychiatr Rehabil
J. 2004;28(1):16-24.

[12.] Kay SR, Fiszbein A, Opler LA. The Positive and Negative
Syndrome Scale (PANSS) for schizophrenia. Schizophr Bull.
1987;13(2):261-76.

[13.] Drake RE, Mueser KT, McHugo GJ. Clinician rating scales:
Alcohol Use Scale (AUS), Drug Use Scale (DUS), and Substance Abuse
Treatment Scale (SATS). In: Sederer LI, Dickey B, editors. Outcomes
assessment in clinical practice. Baltimore (MD): Williams &
Wilkins; 1996.

[14.] McLellan AT, Luborsky L, Cacciola J, Griffith J, Evans F,
Barr HL, O'Brien CP. New data from the Addiction Severity Index.
Reliability and validity in three centers. J Nerv Ment Dis.
1985;173(7):412-23.

[15.] Sobel LC, Sobel MB. Timeline follow-back: A technique for
assessing self-reported alcohol consumption. In: Litten RZ, Allen
JP, editors. Measuring alcohol consumption: Psychosocial and
biochemical methods. Totowa (NJ): The Humana Press; 1993.

[16.] Ware JE. SF-36 health survey. In: Maruish ME, editor. The use
of psychological testing in treatment planning and outcomes
assessment. Mahwah (NJ): Lawrence Erlbaum Associates; 1999.

[17.] Ware J Jr, Kosinski M, Keller SD. A 12-item short-form health
survey: Construction of scales and preliminary tests of reliability
and validity. Med Care. 1996;34(3):220-33.

[18.] Herth K. Abbreviated instrument to measure hope: Development
and psychometric evaluation. J Adv Nurs. 1992;17(10):1251-59.

[19.] Rogers ES, Chamberlin J, Ellison ML, Crean T. A consumer
constructed scale to measure empowerment among users of mental
health services. Psychiatr Serv. 1997;48(8):1042-47.

[20.] Kukla M, Bond GR. The working alliance and employment
outcomes for people with severe mental illness enrolled in
vocational programs. Rehabil Psychol. 2009;54(2):157-63.

[21.] Drebing CE, Van Ormer EA, Mueller L, Rosenheck R, Drake R,
King K. Understanding and enhancing entry and completion of
rehabilitation and recovery ser- vices: A pathways-to-care
approach. Proceedings of the VA Conference on Transforming Mental
Health Care: Promoting Recovery and Integrated Care; 2008;
Arlington, VA.

[22.] Drebing C, Movitz R, Lyon P, Harden T, McCarty E, Herz L.
Documenting pathways to care: Relative validity of questionnaire,
interview, and medical record formats. Am J Alzheimers Dis Other
Demen. 2004;19(3):187-97.

DPA = Diversified Placement Approach, IPS SE = Individual Placement
and Support model of supported employment, VR = vocational
rehabilitation, VS = vocational services.
Gale Copyright: Copyright 2012 Gale, Cengage Learning. All rights reserved.