An exploration of the working alliance in mental health case management.
Psychiatric personnel (Services)
Medical case management (Research)
Psychotherapist and patient (Research)
Therapist and patient (Research)
Kondrat, David C.
Early, Theresa J.
|Publication:||Name: Social Work Research Publisher: National Association of Social Workers Audience: Academic; Trade Format: Magazine/Journal Subject: Sociology and social work Copyright: COPYRIGHT 2010 National Association of Social Workers ISSN: 1070-5309|
|Issue:||Date: Dec, 2010 Source Volume: 34 Source Issue: 4|
|Topic:||Event Code: 200 Management dynamics; 360 Services information; 310 Science & research|
|Geographic:||Geographic Scope: United States Geographic Code: 1USA United States|
The working alliance between clients and helpers has been
identified as a common factor of treatment effectiveness, yet very
little research has explored variables associated with working alliance
between mental health case managers and their consumers. This study
explored the potential covariates of working alliance within community
mental health case management. Specifically, the study explored to what
degree the case manager is related to consumer perceptions of working
alliance, to what degree consumers' perceived mental illness stigma
is related to working alliance, and the extent to which the relationship
between perceived stigma and working alliance is different for different
case managers. Cross-sectional data were collected from 160 people
receiving case management services and were analyzed using hierarchical
linear modeling. Case managers accounted for about 11% of the variance
in working alliance scores, which represents a moderate effect.
Perceived stigma approached a statistically significant relationship
with working alliance. The interaction between case managers and stigma
was significantly related to working alliance. Case managers are an
important source of variance in the relationship between stigma and
working alliance. Future attempts to study working alliance should
include case managers and consumers' perceived stigma as
KEY WORDS: case management; mental illness; mental illness stigma; working alliance
Recovery in mental illness represents a real possibility for people with severe mental illness (SMI) such as schizophrenia and bipolar disorder. Recovery can be understood as consumers' movement toward a self-defined, satisfying fife within the community and does not necessarily equate to symptom remission (Anthony, 1993). Recovery in mental illness is a paradigm shift away from past beliefs that people with SMI need help to be maintained in the community toward a belief that people with SMI can thrive in the community (Kruger, 2000). At the national level, the President's New Freedom Commission on Mental Health (2003) has called for the provision of treatment approaches that support consumer movement toward recovery.
Another shift in mental health services is a call for evidence-based practices (EBP),which are interventions that have demonstrated an empirical track record of success with regard to consumer outcomes (Gambrill, 1999). In community mental health, this means ensuring that people living with SMI are provided the most effective treatment and interventions available (Mueser, Torrey, Lynde, Singer, & Drake, 2003). Coupled with recovery, the EBP movement challenges mental health treatment providers and researchers to use and develop treatment approaches that focus on moving consumers toward recovery (Anthony, Rogers, & Farkas, 2003). The working alliance between consumers and helpers is thought to be an important ingredient in effective treatment (Howgego, Yellowlees, Owen, Meldrum, & Dark, 2003). As such, this article explores the working alliance between mental health case managers and consumers, examining factors that might be related to a strong working alliance.
Working Alliance Defined
The working alliance--which also has been called the therapeutic alliance, therapeutic bond, and helping alliance--has been variously defined within the psychotherapy literature (Martin, Garske, & Davis, 2000). In case management research, researchers have generally used Bordin's (1979) pan-theoretical definition of the working alliance (Howgego et al., 2003). Bordin argued that his conception of the working alliance is applicable to any change-oriented activity, such as case management, and that it is the vehicle through which change-oriented activities are successful. He conceptualized the working alliance as consisting of three components: (1) the therapist's and client's agreement on the goals of therapy, (2) the therapist's and client's agreement on the tasks of therapy, and (3) the positive bond between the therapist and the client. Bordin's definition of the working alliance has been operationalized by Horvath and Greenberg (1989) in the Working Alliance Inventory (WAI), which includes Bordin's three components and an overall measure of the working alliance.
In a meta-analysis of the effects of the working alliance in psychotherapy, Horvath (2001) reported an average weighted effect size between working alliance and therapy outcome of 0.21, indicating that working alliance is significantly related to therapy outcome, with perceptions of a stronger working alliance associated with positive therapeutic outcomes. As with studies of the working alliance in psychotherapy, studies in case management have found a significant association between strong working alliances and positive treatment outcomes, such as fewer days hospitalized (Priebe & Gruyters, 1993), greater community living skills (Neale & Rosenheck, 1995), lower reported symptom severity (Neale & Rosenheck, 1995;Tyrrell, Dozier, Teague, & Fallot, 1999), better medication compliance (Solomon, Draine, & Delaney, 1995), fewer days homeless (Chinman, Rosenheck, & Lam, 2002), and greater fife satisfaction (Chinman et al., 2002; Solomon et al., 1995;Tyrrell et al., 1999).Thus, studies of the working alliance have shown this treatment process to be a therapeutic vehicle for consumer recovery.
Factors Affecting Working Alliances between Treatment Providers and Consumers
Although the effect of the working alliance on consumer treatment-related outcomes has been shown to be a catalyst of mental health recovery, only a few studies have explored variables that affect the development of the working alliance between consumers and their community providers. An exception is the work of Draine and Solomon (1996), who studied effects of client characteristics of history of criminal arrests, homelessness, age, number of hospitalizations, and ethnicity on the working alliance, as measured by the Working Alliance Inventory (WAI). They found that being older (over 45 years of age) was associated with client perceptions of stronger working alliance. In addition, they found that having a history of criminal arrests was associated with higher ratings of the bond and tasks subscales. Klinkenberg, Calsyn, and Morse (1998) studied the effects of client and program characteristics--such as number of program contacts and services provided, days housed, income, gender, ethnicity, and education level--on the working alliance. Of these variables, only ethnicity was associated with working alliance: Caucasians had lower working alliance scores than did members of other ethnic groups.
A number of other factors, which have not been studied previously, may be statistically related to the strength of working alliances between case managers and their consumers. Stigma surrounding mental illness presents a significant barrier for individuals suffering from SMI to seeking out needed help and, ultimately, entering into a recovery process (President's New Freedom Commission on Mental Health, 2003). Indeed, individuals with SMI, as well as the general public, hold beliefs that people with SMI are devalued and discriminated against by members of the public, being viewed as unworthy of friends, a .job, or housing and viewed as less than human (Link, 1987; Link, Cullen, Struening, Shrout, & Dohrenwend, 1989). Link (1987) called this phenomena "perceived stigma."
Research on the impact of perceived stigma suggests that people with SMI compared with people without an official psychiatric label are affected by how much perceived stigma they hold, with effects including lower income, greater unemployment, and stronger feelings of demoralization (Link, 1987). Individuals who do not have an official SMI label are unaffected by the degree to which they perceive stigma of individuals with SMI (Link, 1987). In studies of people receiving psychiatric treatment, greater levels of perceived stigma have been associated with lower self-esteem (Markowitz, 1998), greater depressive symptoms (Kanhng & Mowbray, 2005; Link, Struening, Neese-Todd, Asmussen, & Phelan, 2002), less participation in social leisure activities (Perlick et al., 2001), lower feelings of belonging (Prince & Prince, 2002), greater emotional discomfort (Lysaker, Davis, Warman, Strasburger, & Beattie, 2007), and negative perceptions of quality of life (Rosenfield, 1997). In addition, perceived stigma held by people with SMI has been associated with one treatment-related process. Sirey et al. (2001) found that individuals who have higher perceived stigma have lower odds of being medication compliant. A direct effect of perceived stigma on working alliance has yet to be studied. However, the relationship between perceived stigma and medication compliance (Sirey et al., 2001) suggests that perceived stigma can be a barrier to treatment engagement. Therefore, perceived stigma may also be a barrier to the development of a strong working alliance.
Another factor that has been neglected by researchers studying the working alliance is the direct role that treatment providers play in the development of the working alliance. Neither Draine and Solomon (1996) nor Klinkenberg et al. (1998) explored the effects of case managers on consumer perceptions of the working alliance. Ryan, Sherman, and Judd (1994) argued that case managers themselves significantly differ in their implementation of case management services "in ways that are not tied to the particular type of service under study" (p. 965). Ryan et al. argued that provider effects confound the effects of treatment and can significantly affect the effectiveness of treatment for consumers. They explored the effects of case managers on consumers' length of tenure in the case management program (short-term versus long-term tenure). They found that case managers significantly correlated with the length of time consumers were in case management; in fact, case managers accounted for about 15% of the variation in consumer length of time in case management program. As with consumers' length of time in case management, case managers may also be related to consumer perceptions of the working alliance with their treatment provider.
THE CURRENT STUDY
The purpose of this research was to fill the gaps identified in the literature by answering three questions:
1. To what degree are case managers related to consumer perceptions of the working alliance?
2. To what degree does consumers' level of perceived stigma correlate with their perceptions of the working alliance between themselves and their case managers?
3. To what degree does consumers' level of perceived stigma interact with case manager in consumer perceptions of working alliance?
Data for this research project were collected at one community mental health center in Columbus, Ohio. The researcher used a purposive sampling strategy and a cross-sectional research design, recruiting participants in the lobby of the research site and the common area of two group homes run by the mental health center. Although a random sampling strategy would have been preferred, agency policy precluded providing the researcher with a list of clients from which to develop a sampling frame. The researcher and agency staff discussed two additional strategies that were deemed undesirable. The first option considered was for the agency to mail survey instruments to their consumers .Agency personal suggested that this option was undesirable because homeless clients would not be included in the sample and because they believed only a small number of consumers would complete the questionnaire. The second strategy considered was to have case managers hand out the surveys to a predetermined sample of consumers. However, the researcher deemed that case manager involvement could bias results in several ways. Some case managers may have been better able to get consumers to complete the instrument than other case managers. Furthermore, because some of the information collected during the face-to-face interview was on the case manager-consumer working alliance, the researcher was worried that participants would inflate their assessment of the working alliance between themselves and their case managers, taking on the role of the "apprehensive subject" (Pedhazur & Pedhazur-Schmelkin, 1991). Thus, face-to-face recruitment occurred in locations frequented by at least some groups of agency clients, including homeless individuals and others with SMI.
The researcher completed individual interviews with the majority of willing participants in a private location within the mental health agency. A trained assistant similarly conducted individual interviews with the remaining participants in private locations in the group homes. Participants were provided a small snack during the interview. Sample recruitment took place from June 2006 to June 2007. Data on perceived stigma, working alliance, and consumer's case manager were collected in the interviews. Data on control variables were collected from participants' electronic records warehoused at the agency. A nonclinical staff person from the research site and the primary researcher retrieved information on control variables. The Ohio State University's institutional review board (IRB) and the research site's IRB approved the research design, including sample recruitment and face-to-face and control variable data collection.
Research Site and Participants
The mental health center, from which the participants were recruited, provides an array of services to people with SMI and alcohol and drug diagnoses. During fiscal year 2006, the agency provided services to 2,771 people in case management programs. Individuals were eligible to participate in this study if they were enrolled in one of the case management programs (all people in the agency's case management programs had a diagnosed Axis I disorder), were their own guardian, and consented to participate. The researcher and the assistant conducted a total of 160 valid and usable interviews (5.8% of the consumers receiving case management services from the agency). Given the nature of the data collection method used, the researcher was not able to determine the number of clients who refused to consent. However, 13 potential participants (7.5%) began the interview but did not complete it and, thus, were excluded from the analysis.
Descriptive information about the sample is presented in Table 1. About two-thirds of the respondents were male, and more than half were African American. Almost 30% were diagnosed with schizophrenia, and more than half had a comorbid alcohol or other drug disorder. On average, participants were almost 43 years old and had received case management from the agency for close to four years.
Dependent Variable. Working alliance was measured using the 12-item short form of the WAI. The WAI was developed by Horvath and Greenberg (1989) to measure the three aspects of Bordin's conceptualization of the working alliance and includes bond, goals, and tasks subscales. Each of the three subscales has been found to be related to a larger working alliance variable (Tracey & Kokotovic, 1989). The larger working alliance variable average item score was used in the analysis. Because the WAI was initially designed for use in psychotherapy research and to facilitate the face-to-face interview format, the researcher used a strategy suggested by Neale and Rosenheck (1995), replacing the word "therapy" with "case management" and changing the language from the first person declarative to the second person interrogatory. For each item, responses ranged from 1 (never) to 7 (always), with higher scores indicating a stronger working alliance. Internal consistency reliability for the WAI was .95.Working alliance was collected from the client's perspective only. Research by Solomon et al. (1995) indicated that working alliance information collected from consumers was more robustly related to their measured outcomes than was working alliance information collected from case managers.
Independent Variables. Perceived stigma was measured using the 12-item Devaluation and Discrimination Scale (Link, 1987). The Devaluation and Discrimination Scale measures "expectations as to whether most people will reject an individual with a mental illness as a friend, employee, neighbor, or intimate partner, and whether most people will devalue a person with a mental illness as less trustworthy, intelligent, and competent" (Link & Phelan, 2001,p. 373). Responses to each of the items ranged from 1 (strongly disagree) to 4 (strongly agree). In analysis, we used the mean item score for this variable, which also ranged from 1 to 4, with higher scores indicating greater perceptions of perceived stigma. Internal consistency reliability for the Devaluation and Discrimination Scale was .83.
The other independent variable for this study was participants' case manager. Participants' case manager was the actual case manager from whom consumers received services. The identity of participants' case manager was ascertained during the face-to-face interview. Participants identified 47 different case managers. The number of study participants per case manager ranged from two to seven and was 3.4. on average. No other data were collected on case managers.
Control Variables. Several variables related to the clients were included in analysis as control variables. Data for these control variables were collected from the agency's electronic records database and dummy-coded. Gender compared male participants with female participants, with male as the reference category. Race compared white, African American, and other races, with white the reference category. Age was the difference in years between the date of the face-to-face interview and consumers' date of birth. Primary diagnosis was defined as the participant's current DSM-IV (American Psychiatric Association, 1994) or DSM-IV-TR (American Psychiatric Association, 2000) Axis I diagnosis and was coded as schizophrenia, bipolar disorder, major depression, or other diagnosis. Schizophrenia was the reference category. Comorbid alcohol or drug disorder was likewise defined using DSM-IV or DSM-IV-TR criteria. Time in treatment was defined as the number of months between the participant's admission into the case management program and the date of the face-to-face interview. Hours of treatment over the past 90 days was defined as the number of hours participants received services over the 90 days preceding the face-to-face interview.
Theoretical reasons for exploring case manager effects on client perceptions of working alliance were described earlier; statistical reasons exist as well. Because each case manager provided services for several of the consumers in the study, observations of the working alliance did not meet the criteria for independence. Including case managers in the study of working alliance resulted in a nested, or multilevel, data structure, with clients as level-one data and case managers as level-two data. Failure to control for the nested data structure, or case manager effects, is a violation of the independence assumption of ordinary least squares (OLS) analyses (Kenny & Judd, 1986) and can result in issues with autocorrelation. Furthermore, OLS regression pools together level-two and level-one error terms (Luke, 2004), which can result in underestimated standard errors (Hox, 2002). Reduced standard errors can potentiate a chance of making a type I error and can lead to inaccurate interpretation of findings. Hierarchical linear modeling (HLM) was used to account for the nested data structure (Luke, 2004).
To answer the research questions, the researcher used a one-way random effects analysis of covariance (ANCOVA), which is a form of HLM. One-way random effects ANCOVA is similar to OLS ANCOVA, except that one-way random effects ANCOVA separates variances between levels. The one-way random effects ANCOVA uses maximum likelihood (ML) estimation to estimate fixed (level=one effects) and random (level-two effects) parameters. ML estimation is an iterative approach to model building that uses the likelihood function to find the best fitting model (Hox, 2002). To aid in interpretation of level-two variation (Hox, 2002), the independent variables were transformed to be on the same scale through grand mean centering. The researcher used the statistical package HLM (version 6) (Raudenbush, Bryk, Cheong, & Congdon, 2004) to estimate the models.
Although there were fairly complete data for all 160 cases, there were some missing data at the item level. Missing data can bias the results of statistical analysis by serving as a source of measurement error, resulting in distorted relationships between variables (Roth, Switzer, & Switzer, 1999). Some of the items within the working alliance and perceived stigma scales were incomplete, and there were some missing data on race, age, primary diagnosis, comorbid alcohol or drug disorders, and time in treatment in the agency database. The number of missing values on items for the perceived stigma scale and working alliance scale and on control variables ranged from 0% to 10.6%;acceptable upper levels of missingness range from 10% to 40% (FoxWasy1yshyn & El-Masri, 2005). Using techniques suggested by Fox-Wasylyshyn and El-Masri (2005), the researcher tentatively determined that missing data were missing at random. The researcher used a two-step process to account for missing data in the analyses: (1) single random imputation to replace missing items for the stigma and working alliance scales and (2) multiple imputation to replace the remaining missing data.
In single random imputation using the hypothesized relationships between items in each of the scales, the researcher used complete items in the scales to predict missing items in the scales. Past research suggests that this approach is good at reproducing data when items within a scale are missing (Roth et al., 1999). In multiple imputation (MI) using the Markov Chain Monte Carlo algorithm, the researcher replaced the remaining missing data, using all variables included in the final analysis to develop the imputation model (Allison, 2002). As suggested by Allison (2002) and Saunders et al. (2006), data on categorical variables were rounded. Five separate data sets were imputed and analyzed separately. Results from the five separate analyses were pooled together to incorporate the uncertainty about missing data into the standard errors. SAS (version 9.1) proc MI (SAS Institute Inc., 2004) was used to create the separate data sets, and HLM (version 6) was used to analyze and combine the results from each of the data sets. Note that in analysis of multiply imputed data sets, degrees of freedom do not reflect sample size. Rather, degrees of freedom are a function of the within-imputation variances and between-imputation variances. As the between-imputation variance increases relative to the within-imputation variance, degrees of freedom become smaller (Allison, 2002).
The average item score for perceived stigma was 2.7 (SD = 0.4), which was above the midpoint (2.5) of the scale and suggested that participants somewhat agreed that the public devalued and discriminated against people with SMI. The average item score for the WAI was 5.1 (SD = 1.5), which was above the midpoint (4.0) of the scale. In general, participants agreed with statements that supported a strong working alliance with their case managers.
Answering the three research questions required estimation of three HLMs. The first model was an unconditional model, or a one-way random effects analysis of variance. This model was a baseline model and only accounted for the effects of case managers on consumers' perceptions of the working alliance. The second model was the final estimated model and included all independent and control variables. Information on both of these models is presented in Table 2.
The unconditional model, or model 1, provides information necessary for answering the question of whether case managers are related to variance in working alliance. The intraclass correlation (ICC) is the proportion of variance in the dependent variable--working alliance--accounted for by the level-two unit, case managers (Luke, 2004). The ICC is essentially the effect of case managers on working alliance. The ICC for the unconditional model was. 108 (n = 47). Case managers accounted for about 11% of the variance in working alliance, which, according to Davis (1971), is a moderate effect. However, this also means that 89% of the variance in working alliance, which is a very large amount of variance (Davis, 1971), is explained by consumers.
Model 2, which was a test of the independent effects of perceived stigma on working alliance, explained an additional 10% of variance in working alliance, as evidenced by the proportional reduction of prediction error for the level-one statistic (Raudenbush & Bryk, 2002). In this final model, only one variable was significantly associated with working alliance. The number of months a person received treatment at the research site, which was a control variable, was negatively related to working alliance ([beta] = -.003, df = 148, p = .01). The longer a person received treatment, the lower the working alliance score was relative to those who were in treatment for shorter periods of time. Although not significantly related to working alliance, perceived stigma approached a statistically significant negative relationship with working alliance ([beta] = -.568, df = 148,p = .084). Participants who believed that society generally devalued and discriminated against people with SMI had weaker working alliances with their case managers compared with those who demonstrated less perceived stigma.
Model 3, which was a test of an interaction between case managers and perceived stigma on the relationship with working alliance, explained 25% of the variance. Two findings were significant. First, the variance component for the interaction between perceived stigma and case managers was significant (variance component = 1.262, [chi square] = 65.86,p =.03). This finding suggest that some case managers are better able to mitigate the effects of perceived stigma on consumers and, in so doing, have stronger working alliances. The second significant finding relates to months in treatment ([beta] = -.003, df= 148, p = .04). As with the previous model, the longer clients received treatment, the lower their working alliance scores were.
The first research question asked to what degree consumers' case managers actually relate to consumer perceptions of working alliance between themselves and their consumers. Based on the ICC in the unconditional model, case managers accounted for about 11% of ratings of consumers' perceptions of working alliance. Thus, an interpretation is that some case managers were more successful than others in establishing strong working alliances with their clients. Although this finding may be intuitive, the working alliance variable used in this study measures not only the bond that exists between consumers and case managers, but also the degree to which there was cooperation between case managers and consumers on developing the goals and tasks of treatment. A case manager's ability to develop a bond cannot be taught or trained, but case managers can be taught how to collaborate with clients on developing goals and tasks of treatment. The current research findings mirror the work of Ryan et al. (1994), who found that case managers explained about 15% of variation in length of time that consumers were in a case management program. As with Ryan et al., the current study did not test for the effects of between-case manager variation. The lack of demographic and other information about case managers, such as years of experience or embracement of a recovery orientation, did not allow for an understanding about which case manager-related variables are correlated with working alliance. Further research is needed to develop a deeper understanding of what it is about case managers that relates to working alliances as well as case manager effects on other treatment-related variables. Answers from this line of inquiry may help in training case managers and pairing case managers with clients to maximize positive consumer outcomes.
The second question was about to what degree consumers' understanding of perceived stigma relates to their perception of the strength of the working alliance with their case manager. Findings suggest that perceived stigma approached a significant association with working alliance. Potentially, higher perceptions of perceived stigma are associated with lower working alliance scores. As we discuss in the Limitations section, the statistical power for finding a significant effect was low. Past research has explored how perceived stigma is related to other treatment-related processes, including lower rates of medication compliance (Sirey et al., 2001). Although tentative, findings from this study mirror the work of Sirey et al. (2001), which showed that perceived stigma is one factor associated with treatment-related processes. Given the exploratory nature of the study, other unmeasured factors might account for this potential relationship between perceived stigma and working alliance. For example, greater amounts of symptom distress may be related to both higher levels of perceived stigma and weaker working alliance scores. More research is needed to determine if, in fact, perceived stigma is related to the working alliance or if other unmeasured variables better account for this relationship.
The final research question asked whether consumers' level of perceived stigma interacts with case manager in consumer perceptions of working alliance. Although perceived stigma by itself was not a significant predictor of working alliance, the combination of case managers and perceived stigma was significant. Case managers changed the slope of the relationship between perceived stigma and working alliance. Some case managers were able to diminish the effects of perceived stigma on working alliance. Future research is needed to determine why some case managers were able to mitigate the effects of perceived sigma on working alliance. From this line of research, practice principles can be distilled to help case managers develop stronger working alliances with their consumers by lessening the impact of perceived stigma.
The inspiration for this research was a desire to expand understanding of what affects the strength of the working alliance between case managers and their consumers. As discussed, only two previous studies explored what affects working alliance within the context of case management relationships. Draine and Solomon (1996) found that age and past history of arrest were associated with appraisals of working alliance. Klinkenberg et al. (1998) found that ethnicity was associated with working alliance. Case managers, perceived stigma, and the interaction between the two should be considered as potential covariates of working alliance.
Three additional research findings are worth noting. First, participants reported that they somewhat agreed that the public generally devalues and discriminates against people with SMI. Past research suggests that perceived stigma is negatively associated with a number of outcomes, including income (Link, 1987), self-esteem (Markowitz, 1998), and perceptions of quality of life (Rosenfield, 1997). Link et al. (2002) found that interventions engineered to challenge and change consumers' levels of perceived stigma have not been fruitful. They argued for a societal approach that works to debunk and eradicate the negative stereotypes that surround an SMI diagnosis. Such macro-level approaches may be a long-term strategy for decreasing or eliminating negative stereotypes.
Second, length of time in treatment was associated with working alliance scores. The longer consumers had received services from the agency, the lower they appraised their working alliances with their case managers. One might expect that length of time in treatment would be associated with increased working alliance scores, given greater lengths of time to develop this type of relationship with a case manager. However, one potential explanation for this finding, which was not controlled for in this study, is case manager turnover. Case manager turnover can have the effect of creating treatment discontinuity for consumers and might affect their ability to develop strong working alliance relationships (Bond & Pensec, 1991). Informal discussion with staff and consumers of the research site indicated that case manager turnover was a problem at the site. What is particularly striking about this line of argument is that the team approach to case management, which was used by the research site, is supposed to reduce the discontinuity associated with staff turnover (Test, 1992). Future research should explore this inverse relationship between length of time in treatment and working alliance. Finally, controlling for the almost 11% variance explained by case managers in the unconditional model, the final model explained an additional 25% of the variance in working alliance, meaning that 75% of the variance in working alliance at the client level was left unexplained. Future research needs to explore what case manager variables account for the 11% variance explained and determine other potential covariates of working alliance at the client level that explain the remaining 75% of variance.
This study's research design was beset by a number of methodological limitations that temper the interpretations of conclusions drawn from the data. First, the researcher used a purposive sampling method to recruit participants. Purposive sampling methods are associated with a strong possibility that those who volunteered to participate differed appreciably from those who did not (Kazdin, 2003), and they result in an inability to draw statistically valid inferences from the data back to the population (Pedhazur & Pedhazur-Schmelkin, 1991). A related limitation was how case managers (level-two units) were selected. Case managers were included because at least two of their consumers participated in the study. As with data on consumers, the interpretation of case manager effects is limited by the possibility that those case managers appreciably differed from those who were not included. Furthermore, statistically valid inferences about case managers, including their effect on working alliance, cannot be drawn from the sample back to the population of case managers. Not directly sampling case managers also meant that no additional information was available about the case managers, such as type of education, gender, age, or years of experience. Consequently, the nearly 11% variance explained could not be explained by knowledge of case manager variation. This type of information could have helped in developing an understanding about how case managers help or hurt in creating and sustaining strong working relationships with their consumers.
Another limitation relates to statistical power. Prior to sample recruitment, the researcher calculated statistical power using guidelines for OLS regression (Hair, Anderson, Tatham, & Black, 1998) and determined that 150 cases were needed. After sample recruitment, a citation was found that suggested a different way to calculate power was required for multilevel modeling (Hox, 2002). The researcher recalculated the needed sample size for a statistical power of .80 based on a level-two sample size of 50, the probability of making a type I error of .05, a small standardized effect size (d) of 0.2 for each of the parameter estimates, and an interclass correlation of .10. According to Hox (2002), the level-one sample size should have been 312, which was significantly larger than the 160 participants used in this study. The researcher considered reopening sample recruitment. However, a diminishing number of participant volunteers and a change in the administration at the agency, which was the site of subject recruitment, rendered this nearly impossible. In the current study, the lack of statistical power suggests a greater probability of type II error, or the probability of not finding a statistical relationship between variables in a sample when a true relationship exists in the population from which the sample was drawn. This limitation alone represents a rival explanation for the statistically nonsignificant relationship between perceived stigma and working alliance. In addition, reduced statistical power might mean some of the control variables found to be unrelated to the working alliance, such as psychiatric diagnosis, were truly related in the population. Despite the low statistical power, case managers, the interaction term, and length of time in treatment were found to be significantly related to working alliance.
The lack of measurement of other potential confounding variables represents another limitation. In addition to the information about case manager variation described earlier, other unmeasured variables could also provide explanation of what affects working alliance. For example, the number of case managers a person had been assigned over his or her time at the mental health center might be related to working alliance. The addition of other consumer (level-one) variables, such as symptom distress or ability to perform activities of daily living, might have increased variance explained. However, given these limitations, this study sets the groundwork for future research on what affects consumer perceptions of the working alliance between case managers and consumers.
Original manuscript received October 21, 2008
Final revision received July 9, 2009
Accepted July 10, 2009
Allison, P. D. (2002). Missing data. Thousand Oaks, CA: Sage Publications.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.
Anthony, W.A. (1993). Recovery from mental illness: The guiding vision of the mental health service system in the 1990s. Psychosocial Rehabilitation Journal, 16, 11-23.
Anthony, W. A., Rogers, E. S., & Farkas, M. (2003). Research on evidence-based practices: Future directions in an era of recovery. Community Mental Health Journal, 39, 101-114.
Bond, G. R., & Pensec, M. (1991). Intensive case management for frequent users of psychiatric hospitals in a large city: Comparison of team and individual caseloads. Psychosocial Rehabilitation Journal, 15, 90-99.
Bordin, E. S. (1979).The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research, and Practice, 16, 252-260.
Chinman, M.J., Rosenheck, R., & Lam, J. A. (2000). The case management relationship and outcomes of homeless persons with serious mental illness. Psychiatric Services, 51, 1142-1147.
Davis, J. A. (1971). Elementary survey analysis. Englewood Cliffs, NJ: Prentice-Hall.
Draine, J., & Solomon, R (1996). Case management alliance with clients in an older cohort. Community Mental Health Journal, 32, 125-134.
Fox-Wasylyshyn, S. M., & El-Masri, M. M. (2005). Handling missing data in self-report measures. Research in Nursing and Health, 28, 488-495.
Gambrill, E. (1999). Evidence-based practice: An alternative to authority-based practice. Families in Society, 80, 341-350.
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1998). Mulitvariate data analysis (5th ed.). Upper Saddle River, NJ: Prentice Hall.
Horvath, A. O. (2001).The alliance. Psychotherapy, 38, 365-372.
Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the working alliance inventory. Journal of Counseling Psychology, 36, 223-233.
Howgego, I. M., Yellowlees, R, Owen, C., Meldrum, L., & Dark, F. (2003). The therapeutic alliance: The key to effective patient outcome? A descriptive review of the evidence in community mental health case management. Australian and New Zealand Journal of Psychiatry, 37, 169-183.
Hox, J.J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum Associates.
Kanhng, S. K., & Mowbray, C.T. (2005).What affects self-esteem of persons with psychiatric disabilities: The causal attribution of mental illness. Psychiatric Rehabilitation Journal, 28, 354-351.
Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn & Bacon.
Kenny, D.A., & Judd, C. M. (1986). Consequences of violating the independence assumption in analysis of variance. Psychological Bulletin, 99, 422-431.
Klinkenberg,W. D., Calsyn, R.J., & Morse, G.A. (1998). The helping alliance in case management for homeless persons with severe mental illness. Community Mental Health Journal, 34, 569-578.
Kruger, A. (2000). Schizophrenia: Recovery and hope. Psychiatric Rehabilitation Journal, 24, 29-37.
Link, B. G. (1987). Understanding labeling effects in the area of mental disorders: An assessment of the effects of expectation of rejection. American Sociological Review, 52, 96-112.
Link, B. G., Cullen, F.T., Struening, E., Shrout, P. E., & Dohrenwend, B. P. (1989).A modified labeling theory approach to mental disorders: An empirical assessment. American Sociological Review, 54, 400-432.
Link, B. G., & Phelan, J.C. (2001). Conceptualizing stigma. Annual Review of Sociology, 27, 363-385.
Link, B. G., Struening, E. L., Neese-Todd, S., Asmussen, S., & Phelan, J. C. (2002). On describing and seeking to change the experiences of stigma. Psychiatric Rehabilitation Skills, 6, 201-231.
Luke, D. A. (2004). Multilevel modeling. Thousand Oaks, CA: Sage Publications.
Lysaker, P. A., Davis, L. D., Warman, D. M., Strasburger, A., & Beattie, N. (2007). Stigma, social functioning and symptoms in schizophrenia and schizoaffective disorder: Associations across 6 months. Psychiatry Research, 149, 89-95.
Markowitz, F. E. (1998). The effects of stigma on psychological well-being and life satisfaction of persons with mental illness. Journal of Health and Social Behavior, 39, 335-347.
Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Counseling and Clinical Psychology, 68, 432-450.
Mueser, K.T., Torrey, W. C., Lynde D., Singer, R, & Drake, R. E. (2003). Implementing evidence-based practices for people with severe mental illness. Behavior Modification, 27, 381-411.
Neale, M. S., & Rosenheck, R.A. (1995).Therapeutic alliance and outcome in a VA intensive case management program. Psychiatric Services, 46, 719-721.
Pedhazur, E. J., & Pedhazur-Schmelkin, L. (1991). Measurement, design and analysis: An integrated approach. Hilldale, NJ: Lawrence Erlbaum.
Perlick, D. A., Rosenheck, R.A., Clarkin, J. F., Sirey, J. A., Salahi, J., Stuening, E. L., & Link, B. G. (2001). Stigma as a barrier to recovery: Adverse effects of perceived stigma on social adaptation of persons diagnosed with bipolar affective disorder. Psychiatric Services, 52, 1627-1632.
President's New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America (DHHS Publication. No. SMA-033832). Rockville, MD: U.S. Department of Health and Human Services.
Priebe, S., & Gruyters, T. (1993).The role of the helping alliance in psychiatric community care: A prospective study. Journal of Nervous and Mental Disease, 181, 552-557.
Prince, P. N., & Prince, C. R. (2002). Perceived stigma and community integration among clients of assertive community treatment. Psychiatric Rehabilitation Journal, 25, 323-331.
Raudenbush, S.W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage Publications.
Raudenbush, S. W., Bryk, A. S., Cheong, Y. K., & Congdon, R.T., Jr. (2004). HLM 6: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International.
Rosenfield, S. (1997). Labeling mental illness: The effects of received services and perceived stigma on life satisfaction. American Sociological Review, 62, 660-672.
Roth, P. L., Switzer, E S., & Switzer, D. M. (1999). Missing data in multiple item scales: A Monte Carlo analysis of missing data techniques. Organizational Research Methods, 2, 211-232.
Ryan, C. S., Sherman, P. S., & Judd, C. M. (1994).Accounting for case manager effects in the evaluation of mental health services. Journal of Consulting and Clinical Psychology, 62, 965-974.
SAS Institute Inc. (2004). SAS/STAT 9.1 user's guide. Cary, NC:Author.
Saunders, J. A., Morrow-Howell, N., Spitznagel, E., Dore, P., Proctor, E. K., & Pescarino, R. (2006). Imputing missing data: A comparison of methods for social work researchers. Social Work Research, 30, 19-31.
Sirey, J. A., Bruce, M. L., Alexopoulos, G. S., Perlick, D.A., Friedman, S.J., & Meyers, B. S. (2001). Perceived stigma and patient-rated severity of illness as predictors of antidepressant drug adherence. Psychiatric Services, 52, 1615-1620.
Solomon, R, Draine, J., & Delaney, M.A. (1995).The working alliance and consumer case management. Journal of Mental Health Administration, 22, 126-134.
Test, M.A. (1992).Training in community living. In R. R Liberman (Ed), Handbook of psychiatric rehabilitation (pp. 153-170). New York: Macmillan.
Tracey, T.J., & Kokotovic, A. M. (1989). Factor structure of the Working Alliance Inventory. Psychological Assessment, 1, 207-210.
Tyrrell, C. L., Dozier, M., Teague, G. B., & Fallot, R. D. (1999). Effective treatment relationships for persons with serious psychiatric disorders: The importance of attachment states of mind. Journal of Counseling and Clinical Psychology, 67, 725-733.
David C. Kondrat, PhD, MSW, is assistant professor, Utah State University, Department of Sociology, Social Work, and Anthropology, 0730 Old Main, Logan, UT 84322-0730; e-mail: email@example.com. Theresa J. Early, PhD, MSW, is associate professor, College of Social Work, Ohio State University, Columbus. This article is from the dissertation of the first author under the direction of the second author. The research was supported by a grant from the Ohio Department of Mental Health, but the views expressed herein are those of the authors. The authors thank the agency and the consumers who participated in the study.
Table 1: Client Demographics, Treatment, Working Alliance, and Perceived Stigma Descriptive Information (N = 160) Variable n M (SD) Gcllder 160 Female 53 Male 107 Race 144 White 55 African American 82 Other 7 Age (years) 159 42.59 (8.48) Primary diagnosis 147 Schizophrenia 47 Bipolar 39 Depression 25 Other 36 Missing 13 Comorbid alcohol/other drug 147 Yes 76 No 71 Time in treatment (months) 159 45.43 (57.59) Hours of treatment over past 90 days 20.61 (24.73) Working alliance (a) 160 5.12 (1.49) Perceived stigma (a) 160 2.71 (.40) Variable Frequency (%) Gcllder Female 33.1 Male 66.9 Race White 34.4 African American 51.2 Other 4.4 Age (years) Primary diagnosis Schizophrenia 29.4 Bipolar 24.0 Depression 15.0 Other 22.5 Missing 8.1 Comorbid alcohol/other drug Yes 47.5 No 44.1 Time in treatment (months) Hours of treatment over past 90 days Working alliance (a) Perceived stigma (a) (a) Estimates calculated using single random imputation of missing/ unknown items within each scale. Table 2: HLM Analysis of the Interaction of Perceived Stigma and Case Managers on Relationship with Client Perceptions of the Working Alliance (N = 160 clients, N = 48 case managers) Model 7 Fixed Effect Coefficient (SE) Intercept (df = 147) 5.146 ** (0.134) Months in treatment (df = 147) Hours of treatment past 90 days (df = 147) Age (years) (df = 147) Gender: Female (df = 147) Race: Black (df = 45) Race: Other (df = 147) Diagnosis: Bipolar (df = 104) Diagnosis: Depression (df = 82) Diagnosis: Other (df = 147) Secondary alcohol/drug diagnosis (df = 147) Perceived stigma (df = 147) -0.591 Variance Random Effect Component (SD) 1xvd 1 (df = 47) 1.977 (1.40) Level 2 (df = 47) .244 * (.49) Perceived stigma slope (df = 47) Model Summary Intraclass correlation 0.11 Percentage of reduction in error within client Model 2 Fixed Effect Coefficient (SE) Intercept (df = 147) 5.146 ** (0.129) Months in treatment (df = 147) -.004 ** (.001) Hours of treatment past 90 days (df = 147) <.001 (.003) Age (years) (df = 147) -.010 (.015) Gender: Female (df = 147) .157 (.246) Race: Black (df = 45) .390 (.260) Race: Other (df = 147) -.043 (.813) Diagnosis: Bipolar (df = 104) .353 (.306) Diagnosis: Depression (df = 82) .544 (.332) Diagnosis: Other (df = 147) .431 (.323) Secondary alcohol/drug diagnosis (df = 147) -.180 (.312) Perceived stigma (df = 147) -.568 (.327) Variance Random Effect Component (SD) 1xvd 1 (df = 47) -1.781 (1.336) Level 2 (df = 47) .482 * (.48) Perceived stigma slope (df = 47) Model Summary Intraclass correlation Percentage of reduction in error within client 10% Model 3 Fixed Effect Coefficient (SE) Intercept (df = 147) 5.097 * (0.132) Months in treatment (df = 147) -.003 * (.002) Hours of treatment past 90 days (df = 147) <.001 (.003) Age (years) (df = 147) -.015 (.013) Gender: Female (df = 147) .232 (.254) Race: Black (df = 45) .389 (.237) Race: Other (df = 147) .089 (.762) Diagnosis: Bipolar (df = 104) .511 (.316) Diagnosis: Depression (df = 82) .348 (.305) Diagnosis: Other (df = 147) .515 (.318) Secondary alcohol/drug diagnosis (df = 147) -.127 (.293) Perceived stigma (df = 147) -.578 (.301) Variance Random Effect Component (SD) 1xvd 1 (df = 47) -1.477 (1.215) Level 2 (df = 47) .365 * (.604) Perceived stigma slope (df = 47) 1.263 * (1.12) Model Summary Intraclass correlation Percentage of reduction in error within client 25% Note: HLM = hierarchical linear modeling. * p < .05. ** p < .01.
|Gale Copyright:||Copyright 2010 Gale, Cengage Learning. All rights reserved.|