The effectiveness of occupational performance outcome measures within mental health practice.
Introduction: The routine use of outcome measures is essential to
the maintained delivery of quality care and the continued commissioning
of mental health occupational therapy services. Occupational therapists
are required to demonstrate that intervention is successful in an
evidence-based, valid and reliable way. Therefore, this critical review
aims to address the issue of choosing an appropriate occupational
performance outcome measure for use within mental health services.
Method: Evidence was critically appraised for the effectiveness of the Assessment of Communication and Interaction Skills (ACIS), Occupational Therapy Task Observation Scale (OTTOS), Canadian Occupational Performance Measure (COPM) and Goal Attainment Scaling (GAS), all recommended for use by occupational therapists within mental health practice.
Findings and discussion: The review identifies that there are a limited number of clinically based studies evidencing the validity and reliability of occupational performance outcome measures. It also identifies a paucity of literature concerning service user experience of outcome measures, bringing into question how client centred and meaningful these tools are.
Outcome measure, occupational performance, mental health.
Treatment outcome (Research)
Psychiatric services (Management)
Mental health (Practice)
|Publication:||Name: British Journal of Occupational Therapy Publisher: College of Occupational Therapists Ltd. Audience: Academic Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2011 College of Occupational Therapists Ltd. ISSN: 0308-0226|
|Issue:||Date: August, 2011 Source Volume: 74 Source Issue: 8|
|Topic:||Event Code: 200 Management dynamics; 310 Science & research Computer Subject: Company business management|
|Product:||Product Code: 8000186 Mental Health Care; 9105250 Mental Health Programs NAICS Code: 62142 Outpatient Mental Health and Substance Abuse Centers; 92312 Administration of Public Health Programs|
|Geographic:||Geographic Scope: United Kingdom Geographic Code: 4EUUK United Kingdom|
In the 1990s, key objectives from the outcome measures movement (Epstein 1990) were included in health care related government policy, emphasising the use of outcome measures to evidence delivery of care (Department of Health 1991, 1998). Specific to the clinical context of mental health services, the National Service Framework for Mental Health requested routine measurement of patient-related outcomes (Secretary of State for Health 1999). It also called for interventions to be evaluated from the perspective of both clinicians and service users. This was further supported by the National Institute for Mental Health in England (NIMHE 2005), which stressed the importance of outcome measurement in improving service quality and accountability. Therefore, the routine collection and use of outcome measure data are vital to meet the trend in mental health policy (Slade 2006).
In the past decade, the evolution of the concept of evidence-based practice has shaped health system expectations. A recent high profile report by Lord Darzi stressed the importance of outcome measures to increase quality of care: we can only be sure to improve what we can actually measure' (Department of Health 2008, p49). In response to the recommendations outlined in this report, the National Health Service Chief Executive set out his implementation plans through the quality and productivity challenge, specifically highlighting the importance of evidence-based practice to embed change in practice (Nicholson 2009). The use of outcomes to evidence the effectiveness of clinical intervention will have obvious financial implications in the future. Evidence of outcomes will be used when services like occupational therapy are commissioned.
Skelton (2010), an occupational therapist, noted: 'if we don't make savings they will be done to us and they may not be done favourably' (p28). This suggests that evidence of clinical outcomes will be key in securing future funding. However, in the field of mental health, defining outcomes to measure is a significant challenge owing to the complex and subjective experience of each individual case (Schofield 2006). Measurement is complicated further by the dynamic and multifaceted nature of occupational performance (Laver-Fawcett 2007).
With the future of the occupational therapy profession in mind, it seems essential to consider how occupational therapists will evidence the benefit of therapeutic intervention to the occupational performance ability of the service users they work with. This critical review of the literature critiques the effectiveness and use of four outcome measures of occupational performance available within mental health practice. This should assist occupational therapists in making an informed decision regarding which measure to use within their service.
In order to convert this clinical practice issue into a focus for evidence-based research, an answerable question must be formulated (Flemming 1998). The search strategy was constructed by developing search terms from the key words using the patient-intervention-comparison-outcome (PICO) framework (Table 1). This approach has been shown to save searching time (Flemming 1998).
Careful consideration was exercised when formulating search terms. The results were limited to make them specific to the question. This was also supported by the use of a controlled thesaurus (for example, Medical Subject Headings [MeSH] using the Medline database). Mapping tools within databases were used to cross-reference terms and synonyms to ensure that all terminology for each PICO component had been considered. To capture all references, 'wildcard characters' were used to search for multiple words that share the same root (for example, Therap* will return, Therapy and Therapist) (Polit and Beck 2009).
The next stage was to use these terms to search for evidence in electronic databases, as summarised in Table 2. The databases were chosen because they collate studies relevant to allied health professionals in the area of mental health (Melnyk and Fineout-Overholt 2005). Initial searches resulted in a huge amount of varied, and largely irrelevant, literature, including generic outcome measures and those designed purely for research purposes.
The great number of outcome measures available poses a significant challenge to informed decision making in clinical practice. The large search results are supported by one key text that lists over 100 outcome measures available to occupational therapists (Law et al 2005). Therefore, the decision was taken to focus the search on recommended outcome measures most appropriate to clinical practice.
Key texts within occupational therapy and mental health were consulted to identify recommended outcome measures (Table 2). These recommendations were then cross-referenced against evidence of outcome measures currently used (College of Occupational Therapists 2010) and survey data from an unpublished thesis to ensure relevance to clinical practice. The thesis describes choices made by occupational therapists practising in mental health using outcome measures (Duggan 1999). The final four outcome measures (Table 3) were included by recommendation of key texts, evidence of use in practice (Duggan 1999, Duncan et al 2003), and being standardised and individualised outcome measures appropriate to the question (Donnelly and Carswell 2002).
The search was then re-run by including the name of each individual outcome within the PICO framework. A range of literature relevant to clinical practice was retrieved, from experimental quantitative studies through to qualitative studies, unpublished theses and opinion pieces. However, this diversity makes comparison between studies very difficult. Therefore, to capture the clinically meaningful evidence required to answer the question, only evidence specific to the validity and reliability of the outcome measure is included in order to critique its effectiveness in clinical practice.
The literature search identified large numbers of occupational performance outcome measures. These may exist due to the challenge of measuring the many elements of occupational performance. This is supported by the work of Creek (2003), who defined and discussed the complexities of occupational therapy by analysing the individual elements of the process. To inform clinical decision making, four individualised and standardised outcome measures were identified that have clinical utility for occupational therapists within the field of mental health. Table 4 summarises key details from the articles that met the inclusion criteria and enabled review of each study based on guidelines for critiquing research reports (Polit and Beck 2009). A critical appraisal of the literature surrounding the effectiveness and use of these outcome measures in practice follows.
The Assessment of Communication and Interaction Skills (ACIS) and the Occupational Therapy Task Observation Scale (OTTOS)
The ACIS sits within the strong theoretical framework of the Model of Human Occupation (Fisher and Kielhofner 1995). The development of any new scale must be carefully considered and justified, because superfluous scales can add to confusion in the literature (Streiner and Norman 2003). However, there was a clear purpose for the development of the ACIS because it was a concept from the Model of Human Occupation yet to be measured. The ACIS development study included a range of clients from both learning disability and psychiatric clinical settings. It may have been more scientifically rigorous to sample only one clinical group because it is important to ensure that a sample is representative of a specific population (Polit and Beck 2009). However, this method enabled the analysis of the ability of the ACIS to discriminate between clients of different functional levels (Forsyth et al 1999).
Validity is the degree to which the tool (or outcome measure) measures what it is intended to measure (Polit and Beck 2009). Rasch analysis was an appropriate statistical method to use for validating the ACIS because it allows rating-scale observations to be converted into a linear measure, which can then be checked to ensure that sensitivity is in the correct order (Laver-Fawcett 2007). This analysis confirmed the validity of the ACIS by calibrating the test items on a clinically logical linear scale (Forsyth et al 1999). It also identified items that were too easy to score and, therefore, did not provide discriminative value. The authors' justification for the exclusion of these items was transparent, made clinical sense and was endorsed by occupational therapists, therefore adding to the credibility of the article.
Another approach to ensuring validity is to test the relationship between the outcome measure and an established tool (Polit and Beck 2009). This can be achieved by comparing scores through correlation coefficient analysis, whereby a mathematical formula correlates scores on the established tool with those on the new outcome measure (Polit and Beck 2009). The OTTOS task behaviour section showed high correlation with two standardised and validated scales: the Bay Area Functional Performance Evaluation cognitive subscale (r = 0.726) and the Comprehensive Occupational Therapy Evaluation Scale (r = 0.880) (Margolis et al 1996). The closer these results are to 1 the better, with results above r = 0.7 being most desirable (Polit and Beck 2009). The high r, or coefficient value, suggests that the OTTOS was succeeding in rating function during task, and the section on behaviour was rooted in fundamental cognitive skills (Margolis et al 1996). As with the ACIS, development of the OTTOS was clearly justified by being created in response to the clinical need for a simple, quantitative and rapid method of evaluating task performance (Margolis et al 1996). In contrast to the ACIS, the specific sample of psychiatric service users was entirely appropriate because the OTTOS was designed specifically for psychiatric services to answer the needs of practitioners (Margolis et al 1996).
Interrater reliability is the degree of consistency between two independent assessors in scoring the attribute being measured (Polit and Beck 2009). The ACIS has good interrater reliability, because 90% of responses fitted within the Rasch measurement model. Lack of experience was cited as the reason for the small number of clinicians that did not fit within the model (Forsyth et al 1999). This lack of experience could highlight a training need to ensure understanding of the use of this tool; it is worth noting that no training is currently available in the United Kingdom (UK). A similar issue regarding experience was found in the OTTOS.
This outcome measure also had good interrater reliability (r = 0.92 total score, where scores closest to 1 show highest reliability) but only in experienced therapists (Margolis et al 1996).
The Canadian Occupational Performance Measure (COPM) and Goal Attainment Scaling (GAS)
The issue of experience is also evident in the articles relating to the COPM and GAS. Both rely on accurate scoring from practitioners; if this is not achieved then reliability is questionable. In these individualised outcome measures, goals are designed in collaboration between patient and practitioner. They are based on sound clinical reasoning, but this is inevitably affected by experience. However, in reference to GAS this issue must be viewed with a certain amount of caution because the results themselves are uncertain. Lannin (2003) showed that GAS was capable of detecting change through an intervention in pre-test/ post-test scores. However, there was no comment on whether this change was clinically significant. This was compounded by the study failing to use any statistical analysis to make inference from the data.
Howell (1986) set out to see if the process of goal setting was a useful therapeutic intervention in itself. Using the Student's t-test, a comparison was made between results of an experimental group who completed GAS and a control group who received weekly reviews. This experimental comparison of two groups on a dependent variable makes the t-test an appropriate statistical test to use in order to make inferences about the validity of the hypothesis (Polit and Beck 2009). However, there was no comment on whether the data were normally distributed; thus, it was unclear if this test was appropriate because normal distribution is required for parametric analysis such as the t-test (Polit and Beck 2009). The sample size was small (n = 24) and so statistical power could not be ensured. Further to this, the paper concluded that goal setting should be used as a separate treatment technique, although a non-significant result was identified. The credibility of this paper is questionable and, at over 20 years old, it is recommended that this article should not be used to justify the clinical application of GAS in current evidence-based practice.
The COPM has been shown to demonstrate change of client self-perception regarding occupational performance in an individual recovering from a depressive episode (Waters 1995). The significant limitation of this study is the inability to be representative of the wider population due to its single case study design. Although this study is clinically meaningful and provides guidance on clinical practice, it is recognised that single case designs are regarded as one of the least powerful forms of evidence available (Polit and Beck 2009). However, studies with much larger sample sizes, and therefore more powerful forms of evidence, have also detected change. In a large-scale pilot study (n = 268), Law et al (1994) identified clinically significant change, although this conclusion was not justified and, therefore, must only be accepted tentatively. It was intended that the mixed population sampled within the pilot study would allow generalisation of the results to a wider clinical population; however, the recommendation remained for further research into specific clinical groups. Chesworth et al (2002) built on the pilot work and specifically recruited from a sample of adult psychiatric patients. The Pearson correlation coefficient was used as an appropriate statistical test to summarise the direction and relationship between two variables (Polit and Beck 2009). In this case, the two variables were task performance and satisfaction, with analysis confirming a statistically significant result (p< 0.0001). Therefore, they also concluded COPM to be sensitive to change (Chesworth et al 2002).
The criticisms of the literature appraised within this systematic review illustrate the challenge faced by occupational therapists in evidencing their choice of outcome measure in mental health practice. However, the systematic nature of this review has ensured that this is the best evidence currently available with which to make these decisions. The next step within the evidence-based process is to consider how to apply this to practice.
This critical review has sourced and explored the literature regarding four occupational performance outcome measures currently in use by occupational therapists across the UK. The effectiveness and use of these measures has been discussed to assist occupational therapists practising in mental health in making informed decisions, based on the best evidence available, for which measure, or combination, to use with the service users under their care. It is quite clear that outcome measurement is a complex process. If it were easy then all occupational therapists would be completing it routinely as recommended in national policy (Secretary of State for Health 1999, NIMHE 2005). The challenge in practice is likely to be compounded by the complexity of occupational therapy as an intervention (Creek 2003). This, coupled with the individual nature of service users' own occupational performance ability and the opportunity to use an endless variety of tasks for assessment, results in a huge array of potential measures.
The complexity of measuring occupational performance leads to the contradictory 'lack of suitable measures' and 'too many to choose from', as discussed by Duggan (1999). At a UK Occupational Therapy Research Foundation event, the need for more research into the use, validity, reliability, sensitivity and specificity of outcome measures was recognised, with a focus on working with what we have rather than creating new outcome measures (Sainty 2010). Adding to the huge range of outcome measures already in existence is discouraged (Streiner and Norman 2003) and would only lead to less clarity in which outcome measure to choose, placing further pressure on professionals already stretched for time.
Often the quickest way to access current research is by using the internet. Although it provides access to a wealth of fast expanding health care knowledge (Melnyk and Fineout-Overholt 2005), the amount of information available can be superfluous to that required for individual clinical questions. Therefore, a clear strategy for the inclusion and exclusion of evidence is required to answer the question in the most rigorous way (Polit and Beck 2009). With this in mind, the decision was taken to focus the literature review on four outcome measures currently used in practice. Further to this, the outcome measures chosen represent a range of clinical requirements: the ACIS and OTTOS use observation of task function, whereas GAS and the COPM use individualised goal setting and are well suited to higher functioning clients. It is acknowledged that there are other worthy outcome measures cited within the literature; for example, the Assessment of Motor and Process Skills (AMPS) is widely used and was recommended as an appropriate outcome measure (Pan and Fisher 1994). It has not been included in this review due to the high training costs (Chard 2000), which may be difficult to fund in the current financial climate. Further to not being able to review all outcome measures currently in use, a limitation of this review was the inaccessibility of some articles. However, the systematic nature of this review ensured that, where possible, all literature was identified and retrieved.
In order for this critical review to be clinically relevant, outcome measures had to be used in practice. Critical appraisal of the literature supporting the development, validity and reliability testing of the outcome measures was completed. In summary, although there is some evidence for the validity and reliability of these outcome measures, it is often sparse and can only be found in the initial user manuals for the tools. This has to be questioned because the authors of the tools will make money out of their success and, therefore, may be considered to have a vested interest. The lack of continued studies over time into the clinical use of the outcome measures brings into question their clinical application and highlights the need for further studies regarding clinical effectiveness. Although beyond the scope of this article, it is acknowledged that GAS and COPM do have further evidence of validity and reliability; however, caution must be exercised in applying evidence from different clinical settings. What is clear is that there is limited literature specific to mental health; thus reducing the number of studies eligible for inclusion. This is a reality with occupational therapy literature across the board, not helped by the fact that the profession is a relative newcomer to the UK research scene (College of Occupational Therapists 2007).
Busy clinicians are striving to practise in an evidence-based way, but tracking down this evidence and critically appraising it for individual service users and services is a time-consuming process. However, it is of note that the use of standardised outcome measures facilitates evidence-based practice (Unsworth 2011). Therefore, it is hoped that this review will assist occupational therapists to make informed decisions about what outcome measures may be appropriate to their area of practice. This decision-making process will include considering what the tool needs to measure, how it measures this and who it will be measuring. It must also consider the clinical utility regarding ease of use, administration, scoring, costs and training requirements. All these questions require evidence to support the validity, reliability and specificity of the tool within the specific clinical area of work.
The search revealed a paucity of literature regarding service user experience of outcome measures. With the clinical focus on quality client-centred evidenced-based practice, this is clearly an area for concern. There are obvious advantages within the realms of client-centred practice to use individualised outcome measures. Clinicians should ensure that they are using these for the benefit of the patient and not just to meet service aims, asking themselves: how meaningful is the task of completing an outcome measure to each individual service user? This should assist in the choice of measure used: if service users are high functioning then setting their own goals may be an intervention in itself; for others, it may be taking part in the task that matters, enabling the therapist to observe function. Outcome measures provide different levels of information to different people, especially in the context of the complex intervention of occupational therapy.
The outcome measurement of occupational performance in mental health services is a complex process. However, it is required in order to ensure that evidence-based, quality interventions are being provided. Choosing the right outcome measure to demonstrate effective practice involves sound clinical reasoning and consideration of the resulting use of the information collected. The literature available to evidence these decisions is limited; further research should aim to support clinicians in choosing and using quality outcome measures from those already in existence.
This review has identified a limited number of clinically relevant studies concerning the validity and reliability of four recommended occupational performance outcome measures. There is a clear need to
recognise the potential benefits and challenges from the perspective of everyone affected by outcome measures to justify fully those chosen in practice.
* There are limited clinically-based studies evidencing the validity and reliability of occupational performance outcome measures.
There is a lack of literature concerning service user experience to develop meaningful outcome measures.
What the study has added
The study provides clinically relevant critical appraisal of literature supporting the use and effectiveness of four occupational performance outcome measures, promoting evidence-based decision making and inquiry within mental health practice.
This critical review was completed in part fulfilment of a National Institute of Health Research funded Master's degree in Clinical Research (MRes). I am sincerely grateful for the support of Anita Bowser and the Occupational Therapy team at Ravenswood House for the opportunity to complete the MRes. I would also like to thank Dr Christopher Bailey of the University of Southampton for his advice regarding this review. Conflict of interest: None declared.
Chard G (2000) An investigation into the use of the Assessment of Motor and process skills (AMPS) in clinical practice. British Journal of Occuaptional Therapy, 63(10), 481-88.
Chesworth C, Duffy R, Hodnett J, Knight A (2002) Measuring clinical effectiveness in mental health: is the Canadian Occupational Performance an appropriate Measure? British Journal of Occuaptional Therapy, 65(1), 30-34.
College of Occupational Therapists (2007) Building the evidence for occupational therapy. London: COT.
College of Occupational Therapists (2010) Snomedsubsets and OT assessment tools. Available at: http://www.cot.co.uk/Homepage/Research_and_UKOTRF/ eHealth_%28information_management%29/SNOMED_subsets_and_OT_ assessment_tools/ Accessed 14.06.10.
Creek J (2003) Occupational therapy defined as a complex intervention. London: College of Occupational Therapists.
Creek J, Lougher L (2008) Occupational therapy and mental health. 4th ed. Philadelphia: Churchill Livingstone Elsevier.
Crepeau EB, Cohn ES, Boyt Schell BA (2003) Willard and Spackman's occupational therapy. 10th ed. Philadelphia: Lippincott Williams and Wilkins.
Department of Health (1991) The health of the nation: a strategy for England. London: HMSO.
Department of Health (1998) Modernising mental health services: safe, sound and supportive. London: HMSO.
Department of Health (2008) High quality care for all: NHS next stage review final report. The Darzi Report. London: TSO, 49.
Donnelly C, Carswell A (2002) Individualized outcome measures: a review of the literature. Canadian Journal of Occupational Therapy, 69(2), 84-94.
Duggan M (1999) An investigation of the use of outcome measures by occupational therapists practising in mental health. Unpublished MSc thesis. Exeter: University of Exeter.
Duncan EAS, Munro K, Nicol MM (2003) Research priorities in forensic occupational therapy. British Journal of Occuaptional Therapy, 66(2), 55-64.
Epstein AM (1990) The outcomes movement--will it get us where we want to go? New England Journal of Medicine, 323(4), 266-70.
Fisher AG, Kielhofner G (1995) Skill in occupational performance. In: G Kielhofner, ed. A Model of Human Occupation: theory and application. Baltimore, MD: Williams and Wilkins.
Flemming K (1998) Asking answerable questions. Evidence Based Nursing, 1(2), 36-37.
Forsyth K, Lai J, Kielhofner G (1999) The Assessment of Communication and interaction skills (ACIS): measurement properties. British Journal of Occupational Therapy, 62(2), 69-74.
Howell C (1986) A controlled trial of goal setting for long-term community psychiatric patients. British Journal of Occupational Therapy, 49(8), 264-68.
Lannin N (2003) Goal attainment scaling allows program evaluation of a home-based occupational therapy program. Occupational Therapy in Health Care, 17(1), 43-54.
Laver-Fawcett A (2007) Principles of assessment and outcome measurement for occupational therapists and physiotherapists. Theory, skills and application. Chichester: John Wiley.
Law M, Polatajko H, Pollock N, McColl MA, Carswell A, Baptiste S (1994) Pilot testing of the Canadian Occupational Performance Measure: clinical measurement issues. Canadian Journal of Occupational Therapy, 61(4), 191-97.
Law M, Baum C, Dunn W (2005) Measuringoccupationalperformance: supporting best practice in occupational therapy. 2nd ed. Thorofare, NJ: Slack.
Margolis RI, Harrison SA, Robinson HJ, Jayaram G (1996) Occupational Therapy Task Observation Scale (OTTOS): a rapid method for rating task group function of psychiatric patients. American Journal of Occupational Therapy, 50(5), 380-85.
McKay EA, Craik C, Lim KH, Richards G (2008) Advancing occupational therapy in mental health practice. Oxford: Blackwell Publishing.
Melnyk BM, Fineout-Overholt E (2005) Evidence-based practice in nursing and healthcare: a guide to best practice. Philadelphia: Lippincott Williams and Wilkins.
National Institute for Mental Health in England (2005) Outcome measures implementation best practice guidelines. Leeds: NIMHE.
Nicholson D (2009) Letter: Implementing the next stage review visions: the quality and productivity challenge. London: Department of Health.
Pan A, Fisher AG (1994) The Assessment of Motor and Process Skills in persons with psychiatric disorders. American Journal of Occupational Therapy, 48(9), 775-80.
Polit DF, Beck T (2009) Essentials of nursing research: appraising evidence for nursing practice. Philadelphia: Lippincott Williams and Wilkins.
Sainty M (2010) Outcome measurement and effective practice. Occupational Therapy News, 18(5), 50.
Schofield P (2006) Measuring outcome in psychiatric rehabilitation. British Journal of Occuaptional Therapy, 69(10), 481-83.
Secretary of State for Health (1999) National service framework--mental health. London: HMSO.
Skelton J (2010) Quoted in: A McGauran. Can you demonstrate quality and productivity? Occupational Therapy News, 18(1), 28.
Slade M (2006) Commissioning outcome-focused mental health services. Mental Health Review, 11(3), 31-36.
Smith A (1994) Introduction and overview. In: TJ Kiresuk, A Smith, JE Cardillo, eds. Goal attainment scaling: applications, theory and measurement. Mahwah, NJ: Lawrence Erlbaum Associates.
Streiner Dl, Norman GR (2003) Health measurement scales: a practical guide to their development and use. 3rd ed. Oxford: Oxford University Press.
Unsworth CA (2011) Evidence-based practice depends on the routine use of outcome measures. (Editorial.) British Journal of Occupational Therapy, 74(5), 209.
Waters D (1995) Recovering from a depressive episode using the Canadian Occupational Performance Measure. Canadian Journal of Occupational Therapy, 62(4), 278-82.
Correspondence to: Kate Fuller, Occupational Therapy Practitioner, Southern Health NHS Foundation Trust, Ravenswood House Medium Secure Unit, Knowle, Fareham, Hampshire PO17 5NA. Email: firstname.lastname@example.org
Table 1. PICO framework Framework Definition Key words P Patient Adult mental health I Intervention Occupational performance outcome measure C Comparison Effectiveness of outcome measure O Outcome Effectiveness and use in practice Table 2. Evidence search strategy Online English language Key textbooks searched databases searched (February 2010) (February 2010) CINAHL: Cumulative Index to Occupational therapy and Nursing and Allied Health mental health (Creek and Literature (1982-2010) Lougher 2008) Medline (1966-2010) PsycINFO (1887-2010) Measuring occupational Cochrane Database of Systematic performance: supporting Reviews best practice in occupational therapy(Law et al 2005) Unpublished thesis An investigation of the use of Advancing occupational outcome measures by occupational therapy in mental health therapists practising in mental practice (McKay et al 2008) health (Duggan 1999) Willard and Spackman's occupational therapy (Crepeau etal 2003) Table 3. Occupational performance outcome measure inclusion criteria Outcome measure Inclusion/exclusion Outcome measure inclusion criteria criteria of evidence Assessment of Measures Specific to the Communication and occupational outcome measure in Interaction Skills performance Uses question (ACIS) standardised instructions and The study population methods was adult mental health only Occupational Individualised TherapyTask outcome measures-- The article referred Observation Scale compares the person to the development (OTTOS) against himself or of the outcome herself over time measure and/or (Laver-Fawcett 2007) validity/ reliability data Goal Attainment Specific to adult regarding the Scaling (GAS) mental health effectiveness of its use in clinical Canadian Demonstrates practice Occupational clinical utility Performance Measure (COPM) Not designed purely for research Table 4. Summary of studies in critical review Description Articles retrieved Assessment of The ACIS is a structured (Forsyth Communication observational rating scale et al 1999) and Interaction designed to capture in detail Skills (ACIS) the person's social interactional ability whilst he or she is participating in a meaningful social context. Occupational A 10-item 'task behaviour' (Margolis Therapy Task and 5-item 'general behaviour' et al 1996) Observation observational rating scale, Scale (OTTOS) designed to facilitate evaluation and documentation of patient performance during occupational therapy task groups and to improve the communication between occupational therapists and other treatment team members. Goal Attainment Scores if individual patient (Howell Scaling (GAS) goals have been met by an 1986) intervention. Developed for use within community mental health settings (Smith 1994). Through defining the goals, service user and clinician produce an individualised (Lannin outcome measure. 2003) Canadian Service users are directly (Law et al Occupational involved in determining the 1994) Performance initial content of the Measure measure. Individual problems (COPM) related to self-care, productivity and leisure are rated by importance resulting in up to five goals, which are (Waters subsequently rated by 1995) satisfaction and performance. (Chesworth et al 2002) Articles Study method Data analysis retrieved (Forsyth Repeated use ofACIS by Rasch analysis was et al 1999) occupational therapists used to determine (n = 52) in various reliability of item psychiatric and learning statistics, construct disability settings. validity and interrater reliability. (Margolis Pilot study of Reliability testing et al 1996) occupational therapists using correlation (n = 12) using the tool coefficient analysis daily over a one-year on data from 25 period in an American patients. psychiatric inpatient rehabilitation unit. Criterion validity tested through Pearson's product-moment correlations against accepted valid and reliable measures. (Howell Psychiatric day and Student's t-test 1986) inpatient district service between centre based in the UK. experimental (n = 13) and control group scores (n = 11). (Lannin Measured individual and Pre-test/post-test 2003) programme outcomes design to show in Australian community average difference psychiatric patients. in scores before and after intervention. (Law et al Pilot study of 268 Pearson correlation 1994) clients across four coefficient to countries including the detect change in UK. Data include adult performance and psychiatric populations satisfaction scores. from a range of services. (Waters Single case design. Reports changes in 1995) Clinical practice report performance and of acute care satisfaction scores. intervention for a lady No statistical analysis diagnosed with major performed. depression. (Chesworth Sixty adults in contact Pearson correlation et al 2002) with a rural UK mental coefficient to health service. COPM detect change in used to detect change performance and post-intervention. satisfaction scores. Articles Clinical usability retrieved (Forsyth Takes around 20 minutes to et al 1999) complete and is said to be easy to incorporate into practice. Two-day training was implemented prior to data collection for research purposes. Currently no training offered in the United Kingdom (UK). (Margolis Created by occupational therapists et al 1996) practising clinically. The completion time is 1-2 minutes and a free copy is attached to the article. It aims to improve the occupational therapist's ability to communicate findings with other members of staff. It aims to identify subtle changes in client presentation. (Howell Individualised scale. Embraces the 1986) client-centred approach. Detects meaningful change. Can be administered in a time-effective manner. Can be used for programme evaluation. (Lannin 2003) (Law et al An individualised scale measure 1994) of client's self-perception of occupational performance in self-care, leisure and productivity. It has been shown to take less than 30 minutes to complete. (Waters 1995) (Chesworth et al 2002)
|Gale Copyright:||Copyright 2011 Gale, Cengage Learning. All rights reserved.|