Blending the clinical and the cultural: a framework for conducting formal psychological assessment in bicultural settings.
Abstract: Evidence-based, culturally relevant assessment constitutes a major function of applied psychology in New Zealand's bicultural society and wherever psychologists work. Yet what is termed assessment in psychology takes many forms, and it intersects with everyday life and culture in a number of ways. This position paper considers some of these issues, stressing the importance of unpacking the assessment process in the light of culture and lived experience, and then advocates a framework that promotes the blending clinical and cultural knowledge within the process of psychological assessment. The framework is offered as an example of how culture may intersect with psychological assessment where practice or research involves Maori individuals and families, but can be generalised to other cultural settings and situations.
Subject: Family
Authors: Macfarlane, Angus H.
Blampied, Neville M.
Macfarlane, Sonja H.
Pub Date: 04/01/2011
Publication: Name: New Zealand Journal of Psychology Publisher: New Zealand Psychological Society Audience: Academic Format: Magazine/Journal Subject: Psychology and mental health Copyright: COPYRIGHT 2011 New Zealand Psychological Society ISSN: 0112-109X
Issue: Date: April, 2011 Source Volume: 40 Source Issue: 2
Accession Number: 313012092
Full Text: Kaua e rangiruatia te ha o te hoe; e kore to tatou waka e u ki uta

Do not lift the paddle out of unison or our canoe will never reach the shore

This proverb serves to emphasise the importance of working collaboratively.

The people of Aotearoa/New Zealand live in a group of islands located in the remote south-west Pacific. The indigenous Maori migrated from Polynesia and settled about 1000AD. European contact began in 1642, with a rapid increase in migration in the early years of the 19th C. In 1840 a treaty, Te Tiriti o Waitangi (the Treaty of Waitangi) was signed between Maori and the British government, establishing Aotearoa/New Zealand as a British colony. It is now an independent country with a democratic government and a capitalist economy. Maori constitute a substantial minority population, while Pakeha (the term now used for descendants of European migrants) are the majority. There are also other migrant groups, including substantial numbers from Pacific islands, Asia, and India (Williams & Cleland, 2007; Kingi-'Ulu'ave, Falefa, & Brown, 2007). English and Maori are the principal languages of the country, with English being the lingua franca.

The Treaty guaranteed certain rights to Maori but, despite this, Maori suffered grievously from the process and impact of colonization. Adversities included disease, warfare, alienation and confiscation of their land, loss of their language and disruption of their culture. Among the many legacies of this today are relative poverty, educational underachievement, and physical and mental health problems (see Herbert, 2002; Herbert & Morrison, 2007; Nairn, 2007). From the middle of the 19th C to the middle of the 20th C the Treaty was extensively dishonoured, but since 1975 legislation has increasingly enshrined the Treaty in modern national law, established a Tribunal to adjudicate on claims by Maori against the government for Treaty breaches, and marked the ushering in of a "bicultural" perspective that now infuses all national life, although to varying degrees depending on place and context. Although still argued about, two central principles of this bicultural perspective are those of "partnership" (between the government and Maori) and "rangatiratanga" (authority over one's own things, self-determination (Love & Whittaker, 1997, p 125). It is from this sense of there being two equal Treaty partners that the "bi" in "bicultural" comes. Herbert (2010) comments that the Treaty "has enabled two cultures--Maori and Pakeha--with distinctive histories the opportunity to embrace mutual understanding and power sharing, and to provide a functional framework for multicultural practice" (p 108). Thus, psychologists in Aotearoa/New Zealand are legally and ethically required to have regard to the principles of the Treaty, and to be bicultural (Herbert, 2002; Seymour, 2007). They must also recognize the multicultural nature of the population as well.

This history and these principles have direct implications for the practice of psychological assessment and have played important roles in shaping the assessment policies and procedures that are used within the discipline of psychology in contemporary Aotearoa/New Zealand. The practice of formal psychological assessment is critical because it allows psychologists and other professionals to make relevant decisions in the context of formulating diagnoses, giving advice, and devising intervention programmes that aim to achieve better outcomes, for individuals experiencing difficulties in their lives and those who live and interact with them. It is, therefore, a major function in applied professional services.

Psychology, in its formal, disciplinary representation as a science, emerged from Western paradigms (Herbert & Morrison, 2007; Palmer, 2005). While there has been some indigenisation of the discipline in the last 15 years in Aotearoa/New Zealand,. the issues, problems and processes associated with transporting and using psychological assessment protocols in cultures beyond those for which they were developed (Hambleton, 2001; van de Vijver & Hambleton, 1996) continue to be discussed and specific concerns continue to be raised about the "import and test" approach to psychological assessment in general, (Cheung, 2004) and the use of tests specifically in the New Zealand bicultural context (Eatwell & Wilson, 2007). Further, questions are continually being raised that challenge us to reflect on the extent to which the adoption of international evidence about psychological assessment and therapy leads to better outcomes for all (young) people receiving psychological and social services in New Zealand (Blampied, 1999; 2008; Evans, 2008; Evans, Fitzgerald, Harvey, & Herbert, 2008; Herbert, 2002; Macfarlane, 1998; Macfarlane, 2009; Macfarlane, Glynn, Grace, Penetito & Bateman, 2008; Meyer, 2003; Nikora, 2007). The need to rethink and reposition psychological assessment approaches through attention to culture, therefore, is an important challenge.

Cultural differences are often hard for psychologists to handle and accept, in whichever setting (e.g., psychotherapy, schools, health facilities, organizations), they work, because they are so value-laden, and it is the dominant culture that provides the majority of professionals, even though the minority culture may provide large numbers of the individuals receiving the services. When undertaking psychological assessment and programme planning across cultures, it is important that practitioners consider the adequacy of the methodology used, the relevance of the ecological context, and compatibility of the protocols they deploy.

If, to draw on the bicultural context of Aotearoa/New Zealand, we assume that a Maori person who has been referred for psychological assessment has a whakapapa (genealogy; please see the glossary for definitions of Maori words) that ties him or her to a web of cultural patterns, it is useful to pause and consider the historical trajectory that has given rise to these patterns, rather than to assume that the patterns are the product of isolated, singular events that pertain solely to the here and now. This is not to say that psychological assessment and programme planning should focus or dwell on the past or on Maori (or other cultures') socialisation, but neither should the past--or things pertaining to Maori (or other) socialisation--be swept aside "so that we can get on with the real business of the day" (adapted from Mead, 1997). A fundamental consideration must be that professionals "will fail to understand fully enough the problems that people experience in their day-to-day relations with others if they don't take into account the shaping forces that give structure and substance to the backdrops against which individuals speak and act" (Robinson-Zanartu, 2007, p. 55).

If we accept that authentic assessment will allow professionals to make relevant decisions that will lead to better outcomes for clients, then the acknowledgement that there are ethnically linked ways of thinking, feeling and acting that are acquired through socialisation (see Phinney & Rotheram, 1987) is a necessary one. Such an acknowledgement clearly indicates that culture must be acknowledged and taken seriously: It is an absolute that must be heeded or listened to.

The growing literature on bicultural aspects of psychological assessment (e.g., Manna, 2002; Palmer, 2005; Pitama, Robertson, Cram, Gilles, Huria, & Dallas-Katoa, 2007), while emphasising the centrality of cultural knowledge, has not, in general, provided much guidance as to the degree to which cultural ritual and processes should precede or accompany assessment, perhaps leading the naive reader to infer that such rituals are a necessary accompaniment, to the fullest degree, of all assessment, even of the most minor kind; nor has it considered the implications of stereotype threat (discussed below). This article first considers these various matters by drawing upon ideas that locates assessment in a very broad, not just professional psychological, context (cf Palmer, 2005).

The Ubiquity of Assessment

Throughout our lives, each of us assesses ourselves and others in multiple ways, in situations that are sometimes referred to as "socially diagnostic" (e.g., Beck & Clark, 2009). This makes it possible to see assessment as part of our life experiences, meaning that psychological assessment can be considered in a multi-faceted rather than a categorical (as diagnosed/labelled or otherwise) way--as a specific, technical and perhaps an unusual part of an individual or a group's experience, but not something that is outside their range of common experience. Thinking in this more complex way may help us understand how cultural considerations intersect with the process of assessment. This, in turn, may help us understand that while all psychological assessment requires consideration of organizational context and culture (Gregory & Lee, 1986; Hays, 2001; Okazaki & Sue, 1995; Yamada & Brekke, 2008), not every instance of such assessment needs to be proceeded by some distinctive cultural event or ritual (e.g., a mihi whakatau; Manna, 2002). Such understanding should help psychologists discriminate those occasions where cultural knowledge and formal cultural rituals are a necessary, or at least a desirable, part of the process from those occasions where they are not. We also, in this article, endeavour to show that ability to assess appropriately rests on an ability to blend various streams of knowledge in order to bring disciplinary knowledge, culture, and assessment contexts into productive alignment, and we suggest some practical strategies for achieving this. These include the use of Kaitakwaenga (cultural consultants) and Kaumatua (elders) to assist the psychologist in the task of engaging with culture.

As noted above, assessment, at least in the form of personal and group appraisal, is a universal aspect of human life. It is likely that in the last weeks of her pregnancy your mother was asked to monitor your movements as a way of watching out for adverse events in your development. Within minutes of your birth, a member of the obstetric team would have given you an APGAR score; if it was low, you may have needed urgent intervention. At the other end of life, you will probably be monitored for a time by your loved ones or medical staff to detect your transition from life to death. In between these two events, personal and interpersonal appraisal will be a continuous aspect of everyone's life experience.

This day-by-day mutual appraisal is a major part of what has come, within contemporary mainstream psychology, to be called "social cognition"--"the processes by which people make sense of themselves, others, social interactions, and relationships; in other words how people perceive and think about themselves and other people" (Westin, Burton, & Kowalski, 2006, p. 699). Social psychologists continue to document the extensive ways in which we constantly appraise each person we meet in our daily lives, often using the most subtle cues in ways that are largely automatic, non-conscious, and amazingly rapid (Ambardy & Skowronski, 2008; Beck & Clark, 2009). We also appraise ourselves, both deliberately, as when we take that final check in the mirror before venturing forth, and continuously and largely automatically in our social interactions. Indeed, socialization and achievement are closely linked to a developing capacity for self-appraisal and self-assessment. These are capacities that lie at the heart of self-control as well as social control (Beck & Clark, 2009).

Appraisal, monitoring and assessment are specific and acknowledged parts of many individuals' social roles. Parents monitor their children; adult partners, siblings and friends monitor one another; teachers assess their students' learning; sports coaches assess their athletes' fitness; supervisors assess employees. Physicians measure our health; the tax department assesses our tax liabilities; social welfare determines our eligibility for benefits; police monitor our driving and other public behaviours. Assessment is simply a multifaceted and ubiquitous aspect of everybody's life, all the time. We can conclude that the process of social cognition and the mutual and personal assessment and evaluation it gives rise to are likely to be as universal among humankind as is language cognition, but the aspects of persons and social behaviours that are being evaluated will be strongly determined by culture, just as language is, so that different ethnic, linguistic, cultural and national groups will nuance social cognition in many different ways, but no group is likely to be without social cognitive appraisal and assessment as a constant and intrinsic part of its social, personal and economic relationships.

Everyday Assessment Relative to Psychological Assessment

Although social assessment is thus continuous and ubiquitous in all our experiences, most people never experience formal psychological assessment, by which we mean an episode of interaction between a psychologist and one or more clients in which the psychologist uses her/his training in psychological assessment with respect to the client(s), and has the client's informed consent (or other legal warrant) for the process. Thus the individual or group (clients) waiting at the door of the psychologist's office for their first experience of psychological assessment may be uncertain and anxious about the process they will encounter. Nevertheless they, along with the psychologist waiting within, are not, in fact, about to transition from a world of no assessment to a world of assessment because, as already stressed above, interpersonal assessment is an ongoing part of life. Psychological assessment may differ in form, purpose and technical detail from other more commonly experienced kinds of assessment, but it is not different in principle from those numerous and continuous forms of assessment experienced every day.

Because of the ubiquity of social cognition, clients also need to remember that they have been both the doer and the done-to in assessments of many kinds before. They will therefore have experiences, knowledge, personal and cultural resources, strengths and coping skills that will be available to them in the new and unfamiliar formal assessment situation. In short, they will have individual, and in the group case, collective efficacy (Bandura, 2000; Capara, Regalia, Scabini, Barbarenelli, & Bandura, 2004). It is essential good practice for psychologists to do what they can to activate and engage these resources and strengths, and certainly to strive to ensure that they are not weakened or denied during and following formal psychological assessment (Yamada & Brekke, 2008).

Those who are waiting, and the psychologist they are waiting for, may differ in many ways--age, gender, ethnicity, culture, first language, education, social class status, economic advantage, social and/or legal power--however, they still will share many aspects of common humanity, including lifelong experience of being both assessor and assessed. As such, it may be helpful for the clients and psychologist to identify some aspects of social identity held in common, a process that will see the client(s) assessing the psychologist while the psychologist assesses the client; reciprocity (and collaboration) is thus inescapable (Hays, 2001, p. 78). Such a practice is consistent with the recommendation by Duckworth (2009) that "cultural similarities should be addressed before addressing cultural differences" (p 71). Such an approach is also consistent with the ethical principles under which psychologist work (Nairn, 2007) and the requirements for culturally competent practice (New Zealand Psychologists Board, 2006).

Stereotype Threat and its Implications

There is an important caution to mention at this point. While appropriate use of language, gesture and ritual in the assessment situation may prime strengths and resources in the client(s) waiting for assessment (this is a hypothesis yet to be tested), some forms of interaction may activate stereotype threat, to the detriment of the client(s) (Osborne, 2007). Stereotype threat is a subtle, social-psychological cognitive process that arises when one is in a situation or doing something for which a negative stereotype about one's group applies, and which leads to impaired performance in ways consistent with the stereotype, for example, when women underachieve on assessments of quantitative ability (Jordan & Lovett, 2007; Steele, 1997; see also Sackett, Hardison, & Cullen, 2004). When activated in an assessment situation (see Brown & Day, 2006, for an example) stereotype threat distracts individuals from the required performance, consumes cognitive resources, and may engender a prevention focus in which the preoccupation is with not confirming the negative stereotype rather than a focus on the task (Schmader, 2010; Walton & Spencer, 2009). Schmader (2010) summarises our knowledge of stereotype threat thus:

"Individuals live, work, and learn in increasingly diverse environments--environments that might be laden with subtle reminders of societal stereotypes. As we have reviewed, the mere knowledge of stereotypes that impugn your group's abilities can set in motion psychological processes aimed at disproving these beliefs. Ironically, those processes themselves divert resources from effective performance and can exacerbate the appearance of group differences in ability." (p 17).

It is important to note that stereotype threat is not confined to situations where members of minority groups are being assessed by members of, or using procedures designed by, the dominant culture. The fact that women, who are numerically in the majority in Western cultures still show the effects of stereotype threat in particular contexts demonstrates that. It may be, however, that experience of negative stereotyping and/or discrimination against the group is the critical prerequisite for stereotype threat to affect assessment.

Given this, Jordan and Lovett (2007) note that research on stereotype threat suggests a paradox wherein active promotion of social or ethnic identity in an assessment situation can enhance rather than diminish stereotype threat because such reminders of identity may simultaneously (but unintentionally) make existing negative stereotypes more salient. Actions that reduce rather than enhance the salience of social identity may thus be helpful (Schrader, 2010). More empirical investigation of this issue across a range of social and group identities and assessment situations is clearly needed, noting that no such research seems to have been done in Aotearoa/New Zealand. Such research, by enhancing psychologists' disciplinary and cultural knowledge (Love & Waitoki, 2007) may thus be critical to their clients having positive rather than negative experiences in the assessment context. Jordan and Lovett (2007) refer to this approach as one that encourages "identity safety" (cf cultural safety; Ramsden & Spoonley, 1994) and they discuss ways in which psychologists operating in school settings can facilitate this form of safety for those participating in assessment. Ethically, it is clear that awareness of issues pertaining to identity safety/cultural safety are primarily the responsibility of the psychologist, not the client(s)--they are the ones whose identity and culture must be safeguarded, and they must be protected from the threats and risks of the assessment situation to the greatest degree possible, although judgement of how well the psychologist has met the cultural safety responsibilities rests primarily with the client(s), given that it is the consumers who primarily define culturally safe practice (Herbert, 2010; New Zealand Psychologists Board, 2006; Ramsden & Spoonley, 1994).

Approaches to Eliminating Stereotype Threat and other Negative Outcomes

Encouragingly, recent research by social psychologists into interactions between different individuals and groups within society suggests ingenious ways in which contact across different cultural and ethnic groups may be facilitated, anxiety about such contact reduced, stereotype threat eliminated, and collaboration enhanced (see Walton & Spencer, 2009, Table 1 for a list of possible interventions against stereotype threat).

Crisp and Turner (2009), for example, offer a procedure that is remarkable in its simplicity. It involves a rehearsal of the interaction involving imagined inter-individual/intergroup contact (Crisp & Turner, 2009). All that seems to be required is, first, a direct instruction to imagine an interaction as a brief, scripted event. Thus, a non-Maori psychologist about to undertake an assessment with a Maori client might be asked by his or her supervisor or some other facilitator to "take a minute to imagine you are meeting a Maori client and their whanau for the first time." Second, it is important that the imagined interaction is positive in tone: "Imagine that the interaction is positive, relaxed, and comfortable for everyone" (Crisp & Turner, 2009, p. 234). The client(s) can be asked similarly to imagine having a positive interaction with the psychologist. Despite the extreme simplicity of this procedure, positive effects on attitudes and behaviours in intergroup settings have been reported. In particular, Abrams, Crisp, Marques, Fagg, Bedford, and Provias (2008) found that the imagined rehearsal reduced stereotype threat in a situation in which older people interacted with younger people and where the performance of the older persons on a cognitive task was generally supposed to be inferior to that of the younger participants (see also Crisp & Abrams, 2008). The generality and durability of the effects of this imagined contact procedure remains to be established, but its relevance to psychological practice in Aotearoa/ New Zealand is clear to see, and local research is to be encouraged. Again, we emphasise that, given the subtle and perhaps counter-intuitive nature of stereotype threat, it is the responsibility of the psychologist to be active in preventing or minimising its impact on assessment, not a client responsibility. This is an epistemic as well as an ethical responsibility (Blampied, 2008; O'Donohue & Henderson, 1999).

As a further but different (and compatible) example of how interaction between assessor and assessed may be positively facilitated, Sanders and Lawton (1993) describe what they term a Guided Participation Model for psychologists to use when giving families feedback about family assessments. The aim of this process is to have the family and the therapist take time to reach general agreement about the presenting problem, about an explanation of the problem (where the explanation blends family understandings and psychological knowledge about the causes of such problems), and about what family members might constructively do about the problem (Sanders & Lawton, 1993). This general model of assessment feedback might well warrant wider generalization across assessment settings and modalities (Hays, 2001, Chapter 6). The psychologist's cultural knowledge may be especially important at this stage, as his or her understanding of the clients' world views and cultural contexts may permit a blending of knowledge (see below) that facilitates constructive, collaborative and mutually respectful engagement in the therapy process. Kaitakwaenga and kaumatua (cultural consultant, respected elder respectively; see below) can readily be incorporated in this assessment and feedback process.

Clinical and Cultural Intersections

Before we present a possible representation of the multi-faceted aspects of clinical assessment and how culture intersects with these, we present the following scenario as a way of making the issues more explicit and concrete. Suppose a psychologist, responding to a teacher's or a parent's concerns, asks "How often has Hemi hit Tau this week?" Although cultures might differ in the degree to which they would necessarily want to quantify an answer to the question (Porter, 1995), to answer the question by counting the frequency of hitting across different situations does not seem to be an assessment fraught with much cultural baggage, nor is it outside of the common experience of assessment in normal life--we often rate people in terms of how much or how frequently they do specific things. If the question is addressed using some more formal means assessment such as behavioural functional assessment (Evans & Paewai, 1999; Yoman, 2008), we may discover that Hemi hits Tau a lot during maths classes and not at all during kapahaka practice. The psychologist has thereby learned something about the possible intersections of persons, situation, culture and the problem, but again, without that being particularly problematic either to the psychologist or to Hemi or to Tau and to those who share their lives and culture. Nor has the psychologist gone much beyond assessment as everyday experience, even while doing something quite psychologically sophisticated.

Other forms of psychological assessment, particularly those that lead to categorization or diagnosis, pose more challenging questions. One of these is the "Why do you want to know?" question. Is it to address Hemi's aggression, or is it to identity him as a potential warrior? Note that there is a deliberate double entendre in that question. In past times, Hemi's martial skills might have been the focus; today, a rugby league coach might be interested in his potential (the Warriors being New Zealand's only professional rugby league team). That raises directly the issue of who the psychologist is serving in gathering the information--a "Who wants to know" question. If the information is sought by the individuals directly concerned--Hemi himself, his whanau (family) and/or teachers, because they want to solve some problem, then the situation is fairly simple. Complications arise when the psychologist is serving some other agency or person, such as a school, a hospital, a department of the justice system, an iwi authority, and so forth.


The extreme end of the difficulty continuum is reached when the "What do you want to know?" question has connotations that are much more complex, such as: "Is Hemi's IQ < 100?" or "Does Hemi still meet diagnostic criteria for conduct disorder?", or "Is Hemi a psychopath?" Such questions ask for information that is bound into a particular worldview, makes sense only within that worldview, and ties into theories and assessment practices that make sense in that particular milieu and can only be interpreted within it. It is also important to remember that "assessment is not an end in itself but part of the whole clinical process and part of an overall treatment plan including intervention and follow-up" (Gregory & Lee, 1986, p. 637). The intersection of culture with evidence-based intervention is also an important matter to consider, with useful discussions by Whaley and Davis (2009), Kumpfer, Alvarado, Smith, and Bellamy (2002), Morales and Norcross (2010), and Castro, Barrera, and Martinez (2004; see also Yamada & Brekke, 2008).

What we attempt to show in Figure 1 is that as assessment or, more correctly, the decisions based on assessment, become more complex, more life-course determining, and have more implications for the individual and others linked to them, there is more necessity to consider the cultural context and the relevance of cultural knowledge to the assessment process. We have taken the meeting point (zero) of the dimensions in the figure as a "least point", with the dimensions increasing in consequence or complexity outwards; however, we make no attempt to quantify or enumerate these dimensions. Furthermore, the examples given of points on each dimension are not necessarily in exactly the correct relative position, and changes could be made without affecting the argument being illustrated, so long as it is accepted, for example, that making a status determination (e.g., "X is intellectually handicapped") is a "bigger" decision (in terms of process, conceptual complexity, cultural implications, and consequences) than basing advice on a particular bit of information, or that a frequency of event measure is a less complex assessment than determining an IQ score or a psychiatric diagnosis. In the figure, as the size of the triangle grows bigger that indicates the growth of and relative importance of culture in the assessment process.

To consider a further example in order to clarify how the figure can be used, suppose a sibling wants to know how fast his younger sister ran the 100m last week in order to give her advice about entering the school athletics competition. Considered within the framework of the figure, where the space delineated by the triangle indicates cultural concern or significance, the assessment lies within the domain of everyday life; formal cultural knowledge or protocol hardly intrudes. However, if a psychologist is asked to write a report that may be used to decide if a child is to be taken into care, or an accused person is to be judged unfit to stand trial because he or she is psychotic, then culture, as evident from the size of the resultant triangle, becomes extremely important. Any cultural disjunction between assessed and assessor can thus be very problematic for both parties as they move out along these arrows; however (we argue below) this problem can be suitably addressed by the practitioners bringing a blending of clinical and cultural knowledge streams to their assessment and programme planning. This blending, which we now consider in detail, and show diagrammatically in Figure 2, represents the culturally inclusive end of the assessment domains depicted in Figure 1.

Blending the Clinical and the Cultural: Implications for Theory and Practice

Assessment and programme planning, regardless of when, where, and for whom, should be practical and efficient. The practicality and efficiency are determined by having clarity of purpose, accuracy of procedural content, flexibility of structure (insofar that it can be modified if necessary), and integrity in response to things cultural. While all this may appear relatively straightforward, Figure 1 and our discussion of it above serves to illustrate how the conceptualisation of assessment requires more unpacking, especially as it intersects with culture. This process, we believe, requires us to address two gaps in our knowledge.

First, many standard psychological (conventional) assessment constructs tend to be anchored in scientific, clinically-oriented paradigms. The significance of these paradigms should not be understated. For answering some kinds of questions, scientific knowledge is the best kind of knowledge there is (Popper, 1991), however, as Popper warns, given that all scientific knowledge is held tentatively and may be subject to disconfirmation, humility and caution must be adopted in terms of its use. These scientific constructs are often based on etic concepts and applied to the indigenous culture under the assumption that they are universal concepts that are context-independent and cross-culturally relevant. Whether these Western-orientated etic (1) constructs are universally applicable is an empirical question that needs to be investigated.

Second, relevant indigenous constructs, based on emic (2) concepts, may be missing from these standard psychological measures (Cheung & Leung, 1998), raising an important question that asks "To what extent are indigenous worldviews and cultural philosophies included in any standard assessment?" These two gaps highlight the limitations of using only conventional approaches to assessment. Several commentators appear to favour a cross-cultural approach that proposes the adoption of the combined emic-etic approach--a blending of streams--in which the clinical constructs complement the cultural constructs in providing a comprehensive understanding of individuals referred for psychological assessment form non-Western cultures (Durie, 1999; Evans et al., 2008; Macfarlane & Macfarlane, 2008; O'Connor & Macfarlane, 2002; van de Vijver & Leung, 1997). Note that similar blendings are also possible when therapy is undertaken following assessment (Bennett, Flett, & Babbage, 2008; Cunningham, Foster, & Warner, 2010).

In the light of this, and our argument above, we endorse approaches to psychological assessment that conceptualize assessment as a shared enterprise between the client(s) and the professional(s) (Cunningham, et al., 2010; Sanders & Lawton, 1993). In terms of Figure 1, this collaborative stance can become more difficult to achieve as the agencies being served by the assessment get further and further away from the client individual/group and as the cultural gap between the assessor and the assessed grows.

While progress has been evident relative to Maori, a lingering lack of understanding of a Maori worldview on the part of many professionals seems to be a factor that requires urgent attention. Responding to this state of cultural lag is not easy, given the reality that many professionals often have limited understanding of cultures other than their own (Love & Waitoki, 2007). This reality notwithstanding, in order to meet the requirements of culturally safe and culturally competent practice, professionals must critically determine the level of cultural congruence that exists between themselves and those who are referred for professional services (Patton, 1998). While the professional does not need to be of the same culture as those being referred in order to be effective in their work with them, it is imperative that he or she be able to connect with the culture of the other. This might require a cultural self-assessment (see Bromley, 1998; Macfarlane, 1998) wherein responses to the following key questions are sought:

1. What is my perception of cultural diversity?

2. What is my perception of human development and psychology from within a Maori/indigenous worldview?

3. What steps do I need to take in order to articulate these perceptions so as to develop better awareness and understanding of a Maori/indigenous worldview?

4. What steps do I need to take in order to make my assessment and programme planning more responsive to the needs of Maori/indigenous individuals referred for professional support?

5. What kinds of information, skills, and resources do I need to acquire to more effectively carry out the assessment and programme planning?

6. In what ways do I collaborate with other professionals, whanau members and community groups to address deterrents of a culturally responsive process?

7. What frameworks or exemplars do I need to access in order to enhance my awareness, understanding and application of a culturally inclusive, evidence-based intervention process?

Macfarlane and Macfarlane (2008) encourage psychologists engaged in assessment to build on this self-appraisal. They also offer a culturally inclusive framework, To Tatou Waka (3) (see Figure 2) as an answer to question seven above. This framework, used in concert with understandings gained through reference to Figure 1, allows them to inform their practice by further considering and recognising the significance of the cultural dimension in assessment.

A Culturally Inclusive Framework

The authors' frame of reference relative to Figure 2 aligns with our contention that standard psychological knowledge and cultural knowledge must be blended during psychological assessment. The authors also bring into their framework their claim that the blending must involve minimal complexity and maximum integrity. While Figure 2 focuses on a Maori worldview, it could be readily adapted for use with other cultural groups.

In terms of actualising To Tatou Waka (which must be read from the ground up), the bottom rung of the figure requires psychologists to explore and learn more about the Maori worldview so that they can better understand, acknowledge as important and infuse into their practice particular social and behavioural nuances specific to Maori. This initial interaction, referred to as culturally reasoned epistemology, both challenges and enriches thinking processes because it requires the practitioners to acknowledge and pursue understanding of particular notions that may be important rather than ignore them.


The need to pay attention to individual socialisation patterns, as well as whanau, hapu (sub-tribe) and iwi (tribe) values and preferences signals the importance of digging deeper for greater understanding and explanation. This approach may sound simple, but its relevance lies in consideration of how psychologists can implement this step. How are psychologists who are not Maori expected to "think, feel, and act" Maori if they have not been socialised as Maori? Hence, the call (see the second rung of the figure) for reference to socio-cultural expertise, support and guidance. Socio-cultural expertise might reside and emerge from within the whanau, hapu or iwi, in the form of a kaumatua (respected senior member of the Maori community). At another level, it might be accessed from within the professional services, by way of a kaitakawaenga (professional consultant).

Kaumatua take responsibility for ensuring that the tikanga (processes and protocols) are properly carried out and preserved for contemporary and future generations. Kaumatua tend to live active lives, physically, socially and tribally. They tend to be the antithesis to "disengagement theories" (Cumming & Henry, 1961) and their lives are marked by a lifestyle that is culturally enriching and physically challenging. They have competency in te reo (Maori language) and, with a secure cultural identity, are comfortable with their roles as leaders. Because of their greater age and accumulated wisdom, they are accorded a respect that is well deserved. Often, the regard in which they are held means that they become busy and much sought after in the helping professions, such as psychology and counselling. They are able to challenge as well as assist non-Maori professionals working in a bicultural environment and the ethical and practical dimensions of these challenges need to be carefully considered and respected by such psychologists.

Kaitakawaenga means mediator or go-between, and these people are qualified professionals, often with copious experience in their respective fields. Such credentials, while often different from those of their colleagues, ensure that the roles they play are at parity with as well as complementary to those of the clinicians and others within a multidisciplinary team. Their responsibilities are based on best-practice standards that comply with the key performance indicators that are outlined by the organisation that employ them. Kaitakawaenga bring with them a high level of cultural knowledge and expertise, and are therefore critical to the co-working partnership in terms of building strong relationships with Maori youngsters, adults and whanau, accessing and making meaning of key information, and contributing to intervention assessment, analysis and programme planning.

These sources of knowledge and influence (of kaumatua and kaitakawaenga) have considerable mana, or integrity, and are sources that could augment the expertise that psychologists and other professionals have accumulated over years of training and practice. From what is implicit in these culturally sourced forms of knowledge there emerges an empirically based process that places value on observation and experiment as well as on scientific theory, elements that are critical given that professionals draw on many sources of information to guide and shape practice. Indeed, science transcends culture but it should also serve culture; therefore, it would be fair to conclude that culture is important whenever science is applied, again because context is so vital. In other words, content cannot be considered without regard for context, as context provides the ecology wherein people exercise their individuality within a set of social relations and responsibilities. Thus, throughout the "blended" process depicted in Figure 2, knowledge in various forms--oral and written, indigenous-grounded and science-grounded--informs and guides the professional practice. When these contributing factors manifest, a synergetic momentum is likely to occur, a momentum that drives the process forward, like a waka (canoe) upon the water. The practice then becomes evidence-based from the perspective of a holistic view of evidence.


This paper offered responses to the questions psychologists must grapple with in terms of how to generate a professional relationship based on cultural connectedness while undertaking valid psychological assessments in a way that has cultural, ethical, professional, and scientific integrity, and is safe for both clients and professionals. The particularities and examples we considered have come from our own particular national, ethnic, and historical context, but we believe that they have generality whenever psychologists of one culture practice across cultural boundaries. We propose that more is required than just an intention to undertake professional activities armed with an open-minded benevolence. If there is to be a turning within the process, then that turn must be a "turn to meaning", premised on recognition that culture is probably the most important determiner of meaning in a person's life and should therefore not be overlooked in clinical settings (Hall, 1994).

An obligation to seek understanding and responses to the intricacies of human behaviour underscores the work of psychologists (Blampied, 1999; Prochnow & Macfarlane, 2008). However, when cultural diversity enters the mix, the demand for a further and deeper dimension of understanding becomes more urgent. In Aotearoa/ New Zealand, the challenge for applied psychologists conducting assessments in bicultural contexts is to find ways of doing science that embodies knowledge in forms and styles of practice that maximise the benefit that those who are seeking, needing and asking for help, receive. To do this, psychologists must "listen to culture" as well as to disciplinary knowledge. To Tatou Waka (Macfarlane & Macfarlane, 2008) is a culturally inclusive framework that provides a pathway to achieving this meaning, or understanding. It is premised on an epistemological foundation that is culturally reasoned and strong. This foundation clearly supports what follows in terms of practice and outcomes, given that it is the springboard from which practice emanates.

To summarise, evidence-based assessment practice requires psychologists to be responsive to four key factors that are both interweaving and interdependent, yet are clearly defined in terms of progression. These are:

1. The awareness, knowledge and perspectives that one brings to a particular context or setting;

2. The processes that are utilised (i.e., the ways of engagement, interaction, and communication with the individual and his or her whanau);

3. The data available and how these are analysed: Making meaning, drawing inferences, interpreting what is happening from cultural and clinical viewpoints; and

4. The responses, programmes and interactions that ensue.

To Tatou Waka also allows us to explore the structural and symbolic forces at work in psychology--to acknowledge the relationship between scientific and indigenous knowledge.. The framework proposes that culture is materially, socially and ideologically constructed and embedded in the lives we live. To Tatou Waka is not a stand-alone framework that can simply be applied regardless of context. It is a framework that is cognisant of particular trials and tribulations, is insistent of the specificity of community ties, is considerate of the availability or otherwise of resources, and is intent on enabling rather than subverting the potential of a blending of streams of knowledge in psychological practices.


We are grateful for the excellent advice and guidance provided to us by Paula Wagemaker, a freelance editor based in Christchurch, whose work connects her to staff of the College of Education at the University of Canterbury.

The assistance of Emma Marshall in the preparation of this manuscript is also acknowledged. Emma was supported by a Summer Scholarship 2009-2010 jointly funded by the Tertiary Education Commission of New Zealand and the University of Canterbury. Jonathan Wiltshire is thanked for his help in devising Figure 1.


Aotearoa--original (indigenous) name for New Zealand

hapu--sub-tribe--made up of a group of families, or whanau

iwi--tribe--made up of a group of sub-tribes, or hapu

kaitakawaenga--Maori cultural specialist, professional consultant, liaison person

kapahaka--Maori cultural performing group

kaumatua--a respected senior member of the Maori community

mana--prestige, divine right, influence, status, integrity, charisma, authority

Maori--indigenous people of New Zealand

mihi whakatau--formal welcome process, speech of greeting, official welcome speech--speech acknowledging those present at a gathering.

te reo--language

te reo Maori--Maori language

tikanga--cultural processes, protocols, customs, obligations and conditions

to tatou--our waka--canoe

whakapapa--genealogy, cultural identity, family tree

whanau--family, group of people closely connected in some way


Abrams, D., Crisp, R. J., Marques, S., Fagg, E., Bedford, L., & Provias, D. (2008). Threat inoculation: Experienced and imagined intergenerational contact prevents stereotype threat effects on older people's math performance. Psychology & Aging, 23, 934-939.

Ambardy, N., & Skowronski, J. J. (2008). (Eds.). First impressions. New York, NY: Guilford.

Bandura, A. (2000). Exercise of human agency through collective efficacy. Current Directions in Psychological Science, 9, 75-78.

Beck, L.A., & Clark, M.S. (2009). Choosing to enter or avoid diagnostic social situations. Psychological Science, 20, 1175-1181.

Bennett, S.T., Flett, R.A., & Babbage, D.R. (2008). The adaptation of cognitive behavioural therapy for adult maori clients with depression: A pilot study. In M. Levy, L.W. Nikora, B Masters-Awatere, M. Rua, and W. Waitoki (Eds.). Claiming spaces: Proceedings of the 2007 national Maori and Pacific psychologies symposium 23rd-24th Novewmber 2007. Hamilton, NZ: The Maori & Psychology Research Unit.

Blampied, N. M. (1999). Issues in teaching health psychology in Aotearoa/New Zealand. Bulletin of the New Zealand Psychological Society, 97, 4-15.

Blampied, N. M. (2008). Journeying under Matariki and the Southern Cross: What are our guiding stars? A comment and reflection on Evans (2008). New Zealand Journal of Psychology, 37, 1-22.

Bromley, K. D. (1998). Language art: Exploring connections. Needham Heights, MA: Allyn & Bacon

Brown, R.P., & Day, E.A. (2006). The difference isn't black and white: Stereotype threat and the race gap on Raven's Advanced Progressive Matricies. Journal of Applied Psychology, 91, 979-985.

Capara, G. V., Regalia, C., Scabini, E., Barbarenelli, C., & Bandura, A. (2004). Assessment of filial, parental, marital, and collective efficacy beliefs. European Journal of Psychological Assessment, 20, 247-261.

Castro, F.G., Barrera, M., & Martinez, C.R. (2004). The cultural adaptation of prevention interventions: Resolving the tensions between fidelity and fit. Prevention Science, 5, 41-45.

Cheung, F.M. (2004). Use of Western and indigenously developed personality tests in Asia. Applied Psychology: An International Review, 53, 173-191.

Cheung, F. M., & Leung, K. (1998). Indigenous personality measures: Chinese examples. Journal of Cross-Cultural Psychology, 29, 233-248.

Crisp, R. J., & Abrams, D. (2008). Improving intergroup attitudes and reducing stereotype threat: An integrated contact model. In W. Stroebe & M Hewstone (Eds.), European Review of Social Psychology (Vol. 19, pp. 242-284). Hove, UK: Psychology Press.

Crisp, R. J., & Turner, R. N. (2009). Can imagined interactions produce positive perceptions? Reducing prejudice through simulated contact. American Psychologist, 64, 231-240.

Cuningham, P.B., Foster, S.L., & Warner, S.E. (2010). Culturally relevant family-based treatment for adolescent delinquency and substance abuse: Understanding with-in session processes. Journal of Clinical Psychology in Session, 66, 830-846.

Cumming, E., & Henry, W. (1961). Growing old. New York: Basic Books.

Duckworth, M.P. (2009). Cultural awareness and culturally competent practice. In W. O'Donohue & J.E. Fisher (Eds.). General principles and empirically supported techniques of cognitive behavior therapy (pp 63-76). Hoboken, NJ: Wiley.

Durie, M. (1999). Mental health and Maori development. Australian and New Zealand Journal of Psychiatry, 33, 5-12.

Eatwell, J., & Wilson, I. (2007). The effective use of psychometric assessments in decision making. In I.M. Evans, J.J. Rucklidge, & M. O'Driscoll (Eds.). Professional Practice of Psychology in Aotearoa New Zealand. (pp 317-333). Wellington, NZ: New Zealand Psychological Society.

Evans, I.M. (2008). Steering by Matariki and the Southern Cross: Piloting clinical psychology's course in New Zealand. New Zealand Journal of Psychology, 37, 5-14.

Evans, I., Fitzgerald, J., Harvey, S., & Herbert, A. (2008). Cultural competencies for complex systems (family, school, and community): Perspectives on training clinical child psychologists in Aotearoa New Zealand. In B. Anton (Chair), International perspectives on professional training for clinical child and adolescent psychologists: A symposium presented at the International Congress of Psychology, Berlin, Germany, July 2008.

Evans, I.M., & Paewai, M.K. (1999). Functional analysis in a bicultural context. Behaviour Change, 16, 20-36.

Gregory, S., & Lee, S. (1986). Psychoeducational assessment of racial and ethnic minority groups: Professional implications. Journal of Counselling & Development, 64, 635-637.

Hall, R. (1994). Partnership accountability. Dulwich Centre Journal, 2/3, 6-20.

Hambleton, R.K. (2001). The next generation of the ITC test translation and adoption guidelines. European Journal of Psychological Assessment, 17, 164-172.

Hays, P. A. (2001). Addressing cultural complexities in practice. Washington, DC: APA.

Herbert, A. M. L. (2002). Bicultural partnerships in clinical training and practice in Aotearoa/New Zealand. New Zealand Journal of Psychology, 31, 110-116.

Herbert, A.M.L. (2010). Treaty framework for a sustainable future for psychology: C ultural co mpetenc ies in co ntext. Psychology Aotearoa, 2, 106-111.

Herbert, A.M.L., & Morrison, L.E. (2007). Practice of psychology in Aotearoa New Zealand: A Maori perspective. In I.M. Evans, J.J. Rucklidge, & M. O'Driscoll (Eds.). Professional Practice of Psychology in Aotearoa New Zealand. (pp 35-47). Wellington, NZ: New Zealand Psychological Society.

Jordan, A.H., & Lovett, B.J. (2007). Stereotype threat and test performance: A primer for school psychologists. Journal of School Psychology, 45, 45-59.

Kingi-'Ulu'ave, D., Falefa, M., & Brown, T. (2007). A Pasifika perspective of psychology in Aotearoa. In I.M. Evans, J.J. Rucklidge, & M. O'Driscoll (Eds.). Professional Practice of Psychology in Aotearoa New Zealand. (pp 67-83). Wellington, NZ: New Zealand Psychological Society.

Kumpfer, K.L., Alvarado, R., Smith, P., & Bellamy, N. (2002). Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science, 3, 241-246.

Love, C., & Waitoki, W. (2007). Multicultural competence in bicultural Aotearoa. In I.M. Evans, J.J. Rucklidge, & M. O'Driscoll (Eds.). Professional Practice of Psychology in Aotearoa New Zealand. (pp 265- 280). Wellington, NZ: New Zealand Psychological Society.

Love, H., & Whittaker, W. (1997). (Eds.), Practice Issues for Clinical & Applied Psychologists in New Zealand. Wellington, NZ: New Zealand Psychological Society.

Macfarlane, A. (1998, July). Piki Ake Te Tikanga: Culture counts in special education. Paper presented at the 28th annual conference of the Australian Teacher Education Association, Melbourne, Australia.

Macfarlane, A., & Bateman, S. (2007, September). A synthesis of culturally inclusive practice: From research to classrooms. Paper presented at the annual conference of the Australian Association of Special Education (AASE), Crown Plaza Hotel, Sydney.

Macfarlane, A., Glynn, T., Grace, W., Penetito, W., & Bateman, S. (2008). Indigenous epistemology in a national curriculum framework? Ethnicities, 8(1), 102-127.

Macfarlane, A., & Macfarlane, S. (2008). A synthesis of culturally inclusive approaches in special education: From research to practice. A presentation at the Education Plus Professional Development Forum, College of Education, University of Canterbury, Christchurch, October 2008.

Macfarlane, S. (2009). Te Pikinga ki Runga: Raising Possibilities. SET: Research information for teacher, 2 (2009). Wellington, New Zealand: New Zealand Council for Educational Research.

Manna, L. (2002). Biculturalism in practice, 'Te Pounamu': Integration of a Maori model with traditional clinical assessment processes. Proceedings of the National Maori Graduates of Psychology Symposium 2002, 37-44. (Downloaded on 06/11/08 from wfass-trinity.fass.

Mead, H. (1997). A new vision of a better world. In: New Zealand Education Administration Society Conference (NZEAS) book of proceedings (pp. 1- 10). Auckland: NZEAS.

Morales, E., & Norcorss, J.C. (2010). Evidence-based practices with ethnic minorities: Strange bedfellows no more. Journal of Clinical Psychology in Session, 66, 821-829.

Meyer, L. H. (2003). Wanted: Internationally appropriate best practices. Research and Practice for Persons with Severe Disabilities, 28, 33-36.

Nairn, R. (2007). Ethical principles and cultural justice in psychological practice. In I.M. Evans, J.J. Rucklidge, & M. O'Driscoll (Eds.). Professional Practice of Psychology in Aotearoa New Zealand. (pp 19-33). Wellington, NZ: New Zealand Psychological Society.

New Zealand Psychologists Board. (2006). Standards of cultural competence. Wellington, NZ: Author.

Nikora, L. (2007). Maori social identities in New Zealand and Hawai'i. Unpublished doctoral thesis, University of Waikato, Hamilton, New Zealand.

O'Connor, M., & Macfarlane, A. H. (2002). New Zealand Maori stories and symbols: Family value lessons for western counsellors. International Journal for the Advancement of Counselling, 24, 223-237, Kluwer Academic Publishers.

O'Donohue, W., & Henderson, D. (1999). Epistemic and ethical duties in clinical decision-making. Behaviour Change, 16, 10- 19.

Okazaki, S., & Sue, S. (1995). Methodological issues in assessment research with ethnic minorities. Psychological Assessment, 7, 367-375.

Osborne, J. W. (2007). Linking stereotype threat and anxiety. Educational Psychology, 27, 135-154.

Palmer, S. (2005). Psychometrics: An ancient construct for Maori. New Zealand Journal of Psychology, 34, 44- 51.

Patton, M. (1998). The disproportionate representation of African-Americans in special education: Looking behind the curtain for understanding and solutions. Journal of Special Education, 32, 2531.

Phinney J., & Rotheram, M. (1987). Children's ethnic socialization: Pluralism and development. Newbury Park, CA: Sage Publications.

Pitama, S., Robertson, P., Cram, F., Gilles, M., Huria, T., & Dallas-Katoa, W. (2007). Meihana model: Clinical assessment framework. New Zealand Journal of Psychology, 36, 118-125.

Porter, T. (1995). Trust in numbers: The pursuit of objectivity in science and public life. Princeton, NJ: Princeton University Press.

Popper, K.R. (1991). Knowledge and the shaping of reality: The search for a better world. New Zealand Science Review, 48, 56-66.

Prochnow, J., & Macfarlane, A. (2008, August). Managing classroom behaviour: Assertiveness and warmth. Paper presented at the 1st Educational Psychology Forum, Faculty of Education, University of Auckland, New Zealand.

Ramsden, I., & Spoonley, P. (1994). The cultural safety debate in nursing education in Aotearoa. New Zealand Annual Review of Education, 3, 161-174.

Robinson-Zanartu, C. (2007). A response to a short history of colonization and decolonization. In G. Monk, J. Winslade, J., & S. Sinclair (Eds.), New horizons in multicultural counseling (pp. 78-80). Thousand Oaks, CA: Sage Publications.

Sackett, P. R., Hardison, C. M., & Cullen, M. J. (2004). On interpreting stereotype threat as accounting for African AmericanWhite differences on cognitive tests. American Psychologist, 59, 7-13.

Sanders, M. R., & Lawton, J. M. (1993). Discussing assessment findings with families: A guided participation model of information transfer. Child & Family Behavior Therapy, 15, 5-35.

Schmader, T. (2010). Stereotype threat deconstructed. Current Directions in Psychological Science, 19, 14-18.

Seymour, F.W. (2007). Ethics: The foundation for practice. In I.M. Evans, J.J. Rucklidge, & M. O'Driscoll (Eds.). Professional Practice of Psychology in Aotearoa New Zealand. (pp 5-15). Wellington, NZ: New Zealand Psychological Society.

Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist, 52, 613-629.

van de Vijver, F., & Hambleton, R. K. (1996). Translating tests: Some practical guidelines. European Psychologist, 1, 89-99.

van de Vijver, F., & Leung, K. (1997). Method and data analysis for cross-cultural research. Thousand Oaks, CA: Sage Publications.

Walton, G.M., & Spencer, S.J. (2009). Latent ability: Grades and test scores systematically underestimate the intellectual ability of negatively stereotyped students. Psychological Science, 20, 1132-1139.

Westen, D., Burton, L., & Kowalski, R. (2006). Psychology: Australian & New Zealand edition. Milton, QLD: Wiley.

Whaley, A.L., & Davis, K.E. (2009). Cultural competence and evidence-based practice in mental health services. American Psychologist, 62, 563-574.

Williams, M.W., & Cleland, A.M.M.M.T. (2007). Asian peoples in New Zealand: Implications for psychological practice. In I.M. Evans, J.J. Rucklidge, & M. O'Driscoll (Eds.). Professional Practice of Psychology in Aotearoa New Zealand. (pp 85-102). Wellington, NZ: New Zealand Psychological Society.

Yamada, A-M., & Brekke, J. S. (2008). Addressing mental health disparities through clinical competence not just cultural compe tence : The n eed for assessment of sociocultural issues in the delivery of evidence-based psychosocial rehabilitation services. Clinical Psychology Review, 28, 1387-1399.

Yoman. J. (2008). A primer on functional analysis. Cognitive & Behavioral Practice, 15, 325-340.

Corresponding Author:

Angus Macfarlane

College of Education,

University of Canterbury

Christchurch 8140


Angus H. Macfarlane, University of Canterbury

Neville M Blampied, University of Canterbury

Sonja H. Macfarlane, Ministry of Education


(1.) Etic: Relating to features or items analysed without considering their role as a structural unit in a system.

(2.) Emic: Relating to features or items analysed with respect to their role as a structural unit in a system.

(3.) Pronounced tor-tar-toe-wok-a; literally meaning "our canoe"; metaphorically meaning "to paddle in unison, to work collaboratively".
Gale Copyright: Copyright 2011 Gale, Cengage Learning. All rights reserved.