A not so strange silence: why qualitative researchers should respond critically to the qualitative data archive.
A driving force behind the establishment of a qualitative data
archive in the United Kingdom has been the oral historian, Paul
Thompson. He has complained that there is a 'strange silence'
among qualitative sociologists on re-analysis, and that many have been
reluctant to deposit data. The first part of the paper suggests that the
common ethical and practical objections can be overcome in establishing
an archive in Australia. However, there is a more serious underlying
ideological objection: that archiving promotes and institutionalises a
narrow empiricist version of qualitative research. The rest of the paper
makes this case by examining teaching materials on a British website, by
reviewing Thompson's arguments, and by considering some examples of
re-analysis by sociologists. It is argued that qualitative researchers
should respond critically, but that it is possible to address and
overcome these problems when developing an Australian archive.
Keywords: qualitative research, archive, re-analysis
Data libraries (Analysis)
Historians (Beliefs, opinions and attitudes)
|Publication:||Name: Australian Journal of Social Issues Publisher: Australian Council of Social Service Audience: Academic Format: Magazine/Journal Subject: Sociology and social work Copyright: COPYRIGHT 2009 Australian Council of Social Service ISSN: 0157-6321|
|Issue:||Date: Spring, 2009 Source Volume: 44 Source Issue: 3|
|Topic:||Event Code: 290 Public affairs Advertising Code: 91 Ethics|
|Product:||Product Code: 8525301 Historians NAICS Code: 54172 Research and Development in the Social Sciences and Humanities|
|Persons:||Named Person: Thompson, Paul (British oral historian)|
|Geographic:||Geographic Scope: United Kingdom; Australia Geographic Code: 4EUUK United Kingdom; 8AUST Australia|
Sociologists conducting qualitative research in Britain have not responded with enthusiasm to the opportunities afforded by the qualitative data archive (i.e. Qualidata) established in the early 1990s (Corti et al. 2000). A considerable amount of data has been archived, but this covers only a tiny proportion of qualitative research projects pursued each year, and a limited range of topics. Researchers have been asked to submit data as a condition of obtaining funding from the Economic and Social Research Council (ESRC). Few do so, partly because depositing data is not yet a condition for obtaining further grants and it is possible to decline politely on ethical grounds. Although they have put on a brave face, those enthusiasts who have argued that the archive will improve the scientific standing of qualitative research no doubt feel slightly disappointed about the lukewarm response. Paul Thompson (2000), the oral historian who has done most to establish and promote the archive, has criticised sociologists for having little interest in preserving and re-analysing data. He describes the lack of discussion in qualitative methods texts as a 'strange silence'.
This paper responds to these criticisms in the context of proposals to establish a national qualitative archive in Australia. It will start by considering the ethical and practical arguments that are often given for not participating. It will be suggested that, although these 'issues' should not be discounted as foot-dragging, they could be overcome by more generous levels of funding to support the bureaucratic work of obtaining consent and depositing data. The second part of the paper will discuss what may be a more serious underlying objection that puts off, and even alienates, many qualitative researchers. This is that the archive, along with the current system of funding social science, promotes a narrow, empiricist understanding of collecting and analysing data, even though it pays lip-service to the diversity of qualitative research. The next section will argue that, despite the undoubted value of archiving and re-analysis for some purposes, Paul Thompson as an historian does not appreciate how data is used in sociology. It will be suggested that sociological research involves applying different theoretical frameworks to data, so there is something problematic about the search for objective or cumulative knowledge. The final section will consider these general arguments through examining part of a data set and surrounding documents deposited in the British archive. The overall argument will be that an Australian archive has the potential to be useful for sociological purposes, but the problems may outweigh the benefits.
Ethical and Practical Objections
Most objections to the qualitative data archive in Britain have been on ethical grounds. A common argument is that data has been given to a particular researcher, based on a personal relationship, for analysis and publication by that researcher. When deposited in a public archive, it would be possible for another researcher to use the same data for different purposes, even to the extent of presenting the original interviewee in a negative light. Another view would be that the risks are exaggerated in the first place: there is little danger of harming anyone in publishing anonymised qualitative data whether in a journal article or in a data archive (Travers 2005, Dingwall 2006). To put this another way, the benefits of doing research, and maintaining an open society where people publish and read empirical studies, outweighs the low level of risk in most projects. Whether or not one agrees with ethics regulation, social scientists asked to deposit data in the archive commonly refuse on ethical grounds (Corti et al. 2000). Inevitably, those working in archives have proposed bureaucratic solutions: more consent forms for interviewees to sign and procedures whereby the original researchers can vet secondary analysis.
Focus groups with Australian qualitative researchers about archiving have shown that ethical, and to some extent practical issues, are central concerns (Broom et al. 2009). There may, however, be other issues that are more difficult to articulate but perhaps more fundamental in understanding why many researchers have been reluctant to deposit data. These have to do with a perception that academic freedom has been gradually eroded as funding has been removed from universities and given to the Research Councils. It should be remembered that, in Britain, the Social Science Research Council was replaced by the ESRC, for political reasons (Halsey 1989). The purpose of the new agency was to fund 'useful', apolitical research, using methods that managers in government agencies could understand, resulting in concrete findings and recommendations. Sociological research in Australia also has to be packaged in this way, even though many of those applying for grants understand the purpose of producing knowledge differently.
There are researchers in Australia with strong political views, feminists and queer theorists for example, who have stopped applying to the ARC because they know that research on these topics will not get funded. They are well aware that the Excellence in Research for Australia initiative (ERA) will concentrate funding in universities that obtain ARC grants. More often, researchers participate in the heavily bureaucratic process without enthusiasm, because they have to translate what they really wish to accomplish into the common denominator, empiricist, policy-relevant projects favoured by panels. Consider, for example, how a researcher with constructionist epistemological assumptions feels when asked to present academic work as producing a list of 'key findings' that lead to 'impacts' and 'outputs'. Constructionism is a defensible and useful methodological position that should be distinguished from epistemic or moral relativism (for discussion, see Hughes and Sharrock 1996). It would, however, be foolhardy to submit an application to either the ESRC or ARC that makes a strong constructionist or relativist case about some area of government policy. Fortunately, any analytic approach can be watered-down and re-packaged so that it appears to employ straightforward methods, resulting in clear-cut findings that can inform policy recommendations. Many researchers accept these constraints, and 'play the game'. Others become alienated, which means that they may respond to requests to deposit data without enthusiasm, especially if the archive seems to promote or endorse empiricism.
Those associated with the British site would probably want to see it as theoretically neutral, and some efforts have been made to acknowledge the diversity of social research. To give one example, the archive has produced a teaching guide which discusses, and provides examples of, seven types of interviewing. Students are told that many types of interviewing are informed by constructionism:
In this view, all participants in the interview are agents and meanings are subjectively 'constructed', not objectively 'found'. The purpose is to explore co-constructed identities and social worlds, not to ascertain facts. This constructionist viewpoint informs many of the interview types described here ... (ESDS 2009).
In addition, Qualidata holds workshops on 'challenging' data. This probably means re-analysing data sets using different theoretical frameworks. Moreover, one can find contextual statements for projects that are influenced by critical theory, and cite theorists such as Pierre Bourdieu or Michel Foucault who have been critical towards empiricism. How can one possibly come away with the impression that Qualidata, like the ESRC in general, institutionalises an empiricist or positivist understanding of qualitative research?
The answer is, unfortunately, all too easily. The teaching guide offers what many university-based researchers would consider a limited and misleading introduction to qualitative research. There are mixed messages, so even though we are told in the following paragraph that qualitative researchers produce different types of knowledge, the last sentence implies that all interviewers employ sampling techniques associated with the positivist tradition:
Although this passage acknowledges a variety of interview 'styles' or frameworks, it seems significant that most emphasis is placed on the procedures employed in oral history interviews. There has been little consultation with sociologists in traditions that produce different forms of knowledge when designing these exercises. This makes it possible to describe theoretical frameworks reductively in terms of interviewing style or technique as in the following student exercise:
Students should work individually or in pairs. Give each student a transcript.
1. What interviewing style/s can you identify in this transcript?
2. If more than one, what would you say was the most dominant technique?
3. Provide justification and evidence for your answers (ESDS 2009).
There is nothing wrong, in itself, about asking students to describe how questions are phrased in say, a feminist and life history interview. This does not, however, do justice to qualitative research as an academic field. It may suggest to students or teachers that theoretical differences can be exaggerated, or do not matter, since styles can be combined in the same transcript. This is unfortunate, since this archive claims to promote an understanding, and appreciation of, qualitative research.
Whereas the teaching guide at least mentions constructionism, a more powerful message is conveyed by the cumulative effect of contextual statements that present qualitative data as if it represents objective findings, leading to 'impacts' and 'outputs'. As is common in grant proposals, discussions of methodology are almost completely absent from these statements. Some do cite critical theories, but they are not critical in spirit (for an example of a genuinely critical response to empiricism, see Bourdieu et al. 2000). Many theoretical traditions, including interpretive approaches such as ethnomethodology and conversation analysis, but also critical traditions in ethnography and discourse analysis, are not acknowledged in the teaching guide or represented in the archive.
The usual tactic of those building new paradigms in social science is to incorporate or ignore alternatives. This is arguably how to understand institutions that fund research such as the ESRC and ARC. They promote one way of conducting and understanding research, while claiming to respect intellectual diversity. This tacit or explicit support for empiricism may not be the main reason why qualitative researchers have been reluctant to deposit data in the British archive. However, some kind of ideological response, which may simply amount to resentment at being regulated, may underpin and strengthen ethical and practical objections.
Re-Analysing Data in Sociology: Conceptual Considerations
The editors of this special issue have suggested that debates about archiving enable us to examine our assumptions as qualitative researchers. This seems a useful way of approaching these issues, especially since the British archive has arisen, in part, due to the efforts of an oral historian who is convinced about the value of archiving and re-analysis in his own discipline, and cannot understand why there has been little interest among sociologists (Thompson 2000).
Thompson is justly proud of having conducted an oral history study with colleagues at the University of Essex. He preserved the material, and made it available for analysis by different cohorts of graduate students. He points to the benefits that flow from re-analysing data: from subjecting the same data set to a different interpretation or theoretical perspective. He also suggests that, when researching any topic, it would be useful to be able to consult data collected by other researchers. Thompson has used this purely intellectual or scientific argument in lobbying for the formation of a data archive in Britain (Thompson and Corti 1998). Although there is some common ground, it can be distinguished from the naively empiricist view advanced in some parts of the Qualidata website that systematically reviewing the evidence from previous studies will lead to an improvement in government services. Thompson raises some interesting and difficult issues and deserves a response.
This section will, firstly, review Thompson's argument, drawing on his (2000) paper published in Forum Qualitative Social Research. It will then attempt to show differences between Thompson's assumptions about the cumulative character of social enquiry and the value placed by sociologists in theoretical argument. The section will also consider the case of conversation analysis to show that a data archive already exists for at least one sociological tradition, and there is hardly a 'strange silence' about re-analysing data.
Thompson has made a significant personal contribution in establishing a qualitative archive in Britain. As an oral historian, he is genuinely puzzled that sociologists are reluctant to share their data:
As qualitative researchers, we love to meet and exchange ideas. But we are considerably less at ease about sharing our material. Quite often through our work at Qualidata we encounter sociologists who seem almost bonded to their own ethnographic fieldwork notebooks or interviews, feeling that nobody but themselves could interpret them sufficiently well or share the intimate understandings that they have of their informants ... So what is the root of this reluctance to draw on material created by other researchers? Is it simply an unspoken inhibition, a feeling that we may be cheating when not creating our own data? Or are there likely to be such difficulties in using material created by other people that it is scarcely worth the time to try looking at it? (Thompson 2000: 2).
These are good questions, and will be considered later in this paper. For now, however, it is worth noting that Thompson finds other aspects of sociological inquiry equally strange. He cannot, understand, for example, why qualitative and quantitative researchers cannot work together:
... it is difficult to see any convincing intellectual or methodological justification for not attaching, as a regular practice, sub-samples of qualitative interviews to [national longitudinal surveys[. My experience has suggested that it will need a shift of attitudes on both sides, and in particular a greater mutual respect and consideration for different research traditions. But the potential would be enormous. It would allow quantitative sociology to become surer of its sample base and its interpretations of informants' behaviour, and also far richer in its power of illustration. It would give qualitative sociologists the chance to make controlled comparisons outside their own group of interviewees and to test their hypotheses on convincing samples (Thompson 2000: 12).
Many sociologists, including prominent theorists, have similar views, and would like the theoretical and methodological differences inherited from the past to disappear, so that the discipline finally becomes a normal science (11). The difficulty with such views is that there really are incommensurable paradigms. Sociology is a diverse discipline, founded on philosophical debates about the best way to produce knowledge. This is widely recognised, both in introductory texts, including the Qualidata teaching guide, and in the fact there are long-standing debates between different theoretical traditions, and distinctive research literatures. Moreover, the fractious nature of the discipline is only embarrassing if one expects it to become like natural science (the Durkheimian, positivist project). An alternative view is that the debates and divisions generate insights, including methodological advances. They constitute sociology as an academic discipline.
Qualitative sociologists and cumulative inquiry
Once one accepts that sociology is a multi-perspectival discipline, it becomes possible to understand why many qualitative researchers have been reluctant to contribute to the British archive. This is because it is presented by Thompson as a means of testing hypotheses and arriving at objective knowledge. This is meant to happen, firstly, by researchers checking their findings and interpretations against the data obtained in previous studies and, secondly, because any study will be subject to critical examination by future researchers. It seems significant that Thompson presents the research process as a collective enterprise in which researchers are most likely to 'multiply' rather than challenge or subvert the original findings:
There are, in short, many very important gains from re-analysis. At the start of a research project, it can be invaluable in providing a sense of the topics which can be successfully covered in interviewing, and therefore make the pilot stage of the new project both more effective and also much swifter. At a later stage a comparable interview set may also provide a crucial wider sample base for testing the interpretations which are emerging. Finally, by making your research data available to re-analysis by others, you may strikingly multiply the outcomes from your research through the publications of others from the same material which you have created (Thompson 2000: 11).
It is possible to conduct historical research that is informed by a variety of epistemological positions including positivism, interpretivism and postmodernism (see, for example, Jenkins 1991). However, most historians see themselves as producing objective knowledge, through critical engagement with previous studies. Certainly, those who have used the oral history data-base established by Thompson share a common set of theoretical and methodological assumptions. The studies described in his paper are all concerned with developing and refining hypotheses, or developing new questions that do not challenge or disrupt the original project. This can also happen in sociology, but there is considerably more scope for theoretical disagreement, both in collecting and interpreting data.
This could be demonstrated in many ways, but a ready to hand example can be found in a recent study about quality assurance in the British public sector (Travers 2007). This was based largely on original data collected through interviewing those conducting inspections for government agencies, and quality managers in organisations such as schools and hospitals. There is, however, one chapter, discussing responses to bureaucracy and 'red tape', that drew on interviews with professionals, but also on data published in two other studies (Harrison and Dowswell 2002; Sommerlad 1999).
In re-using this data, the researcher wrote an introduction to the source study (what might be called, using the language of the archive, a re-contextualisation statement), before developing his own argument. The data from Harrison and Dowswell (2002) was an interview with a General Practitioner (GP) complaining about evidence-based medicine as creating a bureaucratic burden. These authors would probably agree with the analysis, although the theme was not central to their own study of changes in GP practices (iii). It seems likely, however, that Sommerlad (1999) would take issue with the interpretation of her study about the response of legal aid lawyers to franchising. She had used the interviewee's comments about facing 'overwhelming' burdens as part of a politically-driven argument against state regulation. The re-analysis of this and other materials tries to show how some professionals have ideological objections to attempts to measure their output without siding with this viewpoint. In fact, the chapter suggests that one should respond to claims about 'overwhelming burdens' with some scepticism. Most professionals experience frustration and irritation in coping with 'red tape', but the burdens are not overwhelming (iv).
In writing this chapter, and the study as a whole, I was not seeking as a sociologist to produce objective or cumulative knowledge. It should be apparent, for example, that quality assurers and many professionals affected by their activities have opposing views of quality assurance. As described in the study, quality assurers see these procedures as vital to raise the performance of organisations. By contrast, the professionals who deliver services often view them as burdensome or ritualistic (Power 1997). Because of this one might conclude that it is not possible to arrive at an objective account: that any data deposited in an archive will be interpreted differently by proponents and critics.
Interestingly, while the data sets from several studies about public sector work and organisations have been deposited on the British archive, most of those sampled do not contain complaints about 'red tape' or poor management. This illustrates the difficulties involved in using an archive to make cumulative findings. Perhaps the underlying problem is that all knowledge is contextual and locally subject to interpretation, whereas empiricists assume one can easily make objective findings about social groups and organisations.
The case of conversation analysis
Another response to Thompson might be that, as an historian, he does not appreciate the diversity of research methods employed by sociologists or how different traditions produce cumulative findings working within a particular set of theoretical and methodological assumptions. It would be possible to make this case through contrasting different varieties of ethnography and discourse analysis (see, for example, Travers 2001 or Wooffitt 2005). For the purposes of this paper, conversation analysis is worth considering, since it illustrates that re-analysis does take place in one tradition, and there is considerable methodological discussion (and so no 'strange silence'). It also illustrates a weakness of the archive in not acknowledging the purposes of different analytic traditions.
Conversation analysis has become a major field at the intersection of sociology and linguistics, which has developed a distinctive methodology for studying everyday and institutional interaction (see, for example, Ten Have 1999). The early work was concerned with building up a cumulative understanding of mechanisms in everyday conversation, based on carefully transcribing what could be heard on audio-tapes. There was, initially, no need for an archive since any data could equally well provide findings. However, certain pieces of conversation, especially those recorded and transcribed in great detail by Gail Jefferson are still used in workshops teaching this research method. One can, moreover, make new findings in any piece of data: it depends on developing an analytic sensitivity to what one hears on an audio-recording.
As conversation analysis has developed, there has been more emphasis on analysing interaction in institutional settings. This data is also examined and re-analysed at workshops. However, one can exaggerate the extent to which conversation analysis is a closed pursuit only available to those working inside the tradition. In common with natural science, there is a culture of publishing the raw data used to make findings. It is, therefore, possible to develop an argument through re-analysing the transcripts published in journals. There is, arguably, no need for an archive to store this material, although there would be no harm in putting a selection of audio-tapes with transcripts, and discussion of the methodological objectives of conversation analysis, on a website as a teaching aid or resource for further research.
Conversation analysis is only one among many research traditions in the social sciences, and like any tradition can be criticised on principled grounds. What seems unfortunate about the qualitative archive, as it has developed in Britain, is that one would not know that these debates took place, or even that conversation analysis existed, from the website. The Qualidata site does not explore the possibilities of archiving audio- or video-recorded data, or acknowledge that there are different understandings of transcription. Perhaps inevitably, Thompson as an oral historian equates data with the transcripts of interviews and focus groups, transcribed for the purposes of analysis in his own discipline. This is unfortunate since any archive should represent and acknowledge the diversity of qualitative research, including conversation analysis (v).
A Visit To The British Archive
To make this discussion more concrete, it seems worth visiting the British archive, and considering the possible uses of a particular set of data. To do so, you will need to join the archive, which requires completing an on-line registration form, and also sending a fax giving signed consent to the conditions of use. Once you have successfully registered, you can search the catalogue, and obtain access to the data in many open-access projects. For others, you are asked to contact the researchers to explain your purposes. One topic that interests me is bureaucracy or 'red tape', but these terms do not appear within the key words supplied by projects (vi). However, when browsing the abstracts of projects deposited in the last few years, I found a few about organisational change in health care that seemed relevant. The most useful or relevant project was data set SN5591, 'What Changes When Incentives Change in Primary Medical Care, 2005-2006', which was deposited by Bruce Guthrie of the Tayside Centre for General Practice, University of Dundee (vii).
The first document I looked at was the contextual statement. You can get a sense of the methodological assumptions informing the project, and the bureaucratic procedures involved in preparing data for the archive, from the contents page:
Although this project did not set out to test a hypothesis, it has many features in common with research by natural scientists. There is a concern with sampling, so the interview data is representative, and it becomes possible to make generalisations from the data. There is also a concern with maintaining data quality, to the extent that investigators are asked to sign a legal declaration on the accuracy of transcriptions:
During the course of the study the main researcher proofread all focus group and interview transcripts in order to ensure that typographic and grammatical errors were corrected. Corrections were made using the digital recording that was stored on the main researcher's computer with the researcher listening to appropriate sections of the recording in order to ensure that the transcription was corrected. Prior to depositing the transcripts in Qualidata, the main researcher carried out a further read-through of all the transcripts in order to ensure that errors were removed (Contextual statement SN5591: 11).
Although depositors are required to sign this statement for bureaucratic and legal reasons, the content also commits them to a particular understanding of social science research (viii). Although users are invited to challenge the findings of investigators using a different analytic framework elsewhere on the website, the procedures and language used in these contextual statements convey a different message. This is more powerful, precisely because anyone submitting data is required to complete these forms in a standard way. In completing these and other applications, the researcher is required to subscribe to the positivist paradigm, although it should be possible to address and overcome this problem when designing the Australian archive.
A visit to this data set also helps in making one think about the interpretive difficulties that arise in re-using data. Thompson (2002: 2) asks whether qualitative sociologists have 'an unspoken inhibition' against sharing data, or whether 'there are likely to be such difficulties in using material created by other people that it is scarcely worth the time to try looking at it?' How one answers the first question will probably depend on the type of data collected. Sociologists address all kinds of sensitive topics that depend on building a relationship of trust. Many researchers argue that the ethics system, in which participants are asked to sign consent forms, prevents them from researching these topics. It is, at least in theory, possible for ethics committees to waive the requirement of written consent when conducting research (ix). No one seems to have considered this issue in the case of archiving.
The second question, as to whether it is 'worth the time' to try looking at other people's data is more interesting. Although I did not look at all the data deposited from this project, I did find a focus group in which there were different views of 'red tape'. My reading of the transcript is that most participants (mainly Scottish GPs) expressed some scepticism about the administrative reforms examined in the project. One reports, for example, that:
There is certainly more recognition that you have got to record things, you have got to be much more organised. A lot of it is delegated to nurses and chronic disease management is largely run by nurses, with GP input as you go along. It has got to be much more organised. It is a lot of what you have been doing already, you are just recording it. You are basically just sitting there and ticking boxes. The disadvantage is that it is becoming 'tick-box' medicine (Interview data SN5591: 3).
There was also a quality manager for a Scottish health board who seems rather quiet for much of the focus group. However, right at the end, he or she says:
I think that the best thing for me is that the contract has provided a data set and really allowed us to celebrate the quality of general practice which we haven't really been able to do before. The evidence base is there and now we can shout from the rooftops. And that is fantastic for Boards (Interview data SN5591: 24).
This data could certainly be used to document the existence of concerns about "red tape" among Scottish GPs, and a difference of view between GPs and the quality manager. The problem facing anyone doing a re-analysis is that the record in the focus group is at some remove from the practical issues faced by these managers and professionals in their everyday working lives. Martyn Hammersley (1997: 138-9) makes this point when discussing the strengths and weaknesses of the data archive:
Exactly this problem arises here. What sense can we make of these comments, without knowing a great deal more about GP practices in Scotland, and how they have been affected by the reforms? There is little indication in the contextual statement that administrative burdens were a great problem. This is perfectly understandable, since this was not a central research question in the original study. It does, however, limit the value of the interviews, observations and focus groups as re-analysable data. One cannot get round this problem by saying that we can conduct more research, since the original context is no longer available.
Conclusion The purpose of this special issue is to consider whether a Qualitative Archive should be established in Australia. My objective in advancing some criticisms about the British archive, which seems to be a model favoured by proponents here, has not been entirely negative. In contrast to many qualitative researchers consulted in focus groups (Broom et al 2009), I have no ethical objections.
Even though many would like to abolish the whole system of ethics review internationally, I expect that many research participants would agree to sign forms allowing researchers to archive interviews. In fact, I would support the formation of an archive provided that lessons are learnt from the British system which has not generated great enthusiasm among researchers. In the first place, we should be paid for our time in preparing data. In the second place, the ideological objections reviewed in this paper can be addressed and overcome if the archive, and any publicity materials, contained a statement acknowledging the theoretical diversity of sociological research, and distinguishing this from the empiricist research conducted for government agencies. This could also be incorporated into all deposit forms and contextual statements. Another suggestion would be to establish the archive with an advisory group, with representatives from different traditions.
There have always been criticisms of intellectual bias, and lack of vision, on the part of state-supported funding bodies in social science (x). For a recent example, it is worth reading the recent reports published by the American National Science Foundation that seek to improve the quality of applications (Ragin et al. 2004; Lamont and White 2009), and the reply by Howard Becker (2009). While acknowledging that there are a variety of views in the collections, and that the report is pitched at an high intellectual level, Becker sees this initiative as promoting and seeking to institutionalise a narrow view of science:
Many...of the papers...repeat the message delivered by Lamont and White in the 15 page executive summary and short introduction, which might be summarised as 'Quit whining and learn to do real science by stating theoretically derived, testable hypotheses, with methods of data gathering and analysis specified before entering the field. Then you'll get NSF grants like us'. Less contentiously, you could say that the report recommends an unnuanced and incomplete version of the King, Keohane and Verba Designing Social Inquiry (1994) message: start out with clear, theoretically anchored hypotheses, pick a sample that will let you test those ideas, and use a pre-specified method of systematic analysis to see if they are right (Becker 2009: 2).
Ideally, it would be helpful to have a critical discussion of these debates in a teaching guide. In this case, empiricist research designed to produce objective, cumulative knowledge would be presented as one among a number of traditions, rather than as the only legitimate way of conducting scientifically rigorous qualitative research. Logically, one might also want to overhaul the ARC mission statement so that it serves those doing both academic and policy research, since the data archive will be part of this system of funding. We will not know for many years whether storing virtual mountains of raw data with their contextual statements for posterity in cyberspace will make an important contribution to sociological knowledge. It does, however, seem worth investing some effort and thought in developing an archive, even on an experimental basis, in Australia.
Abbott, A. (2001). Chaos of Disciplines, Chicago, University of Chicago Press.
Becker, H. (2009) 'How to find out how to do qualitative research', International Journal of Communication, 3,545-53, Available online: http://ijoc.org/ojs/index.php/ijoc/article/viewFile/550/329, Accessed: May 2009.
Bourdieu, P. et al (2000) The Weight of the World: Social Suffering in Contemporary Society, Cambridge, Polity Press.
Broom, A., Cheshire, L. & Emmison, M. (2008) 'Ethics, relationship and ownership: dilemmas in qualitative data sharing and the development of the Australian Qualitative Archive (AQuA)', Paper delivered at the Australian Sociological Association (TASA), University of Melbourne, December.
Clifford, J. & Marcus, G. (eds.) (1986) Writing Culture: The Politics and Poetics of Ethnography, Berkeley, University of California Press.
Corti, L., Day, A. & Blackhorse, G. (2000) 'Confidentiality and informed consent: Issues for consideration in the preservation of and provision of access to qualitative data archives', Forum Qualitative Sozialforschung, 1(3), Available online: http://qualitative-research.net/ fqs-texte/3-00/3-00cortietal-e.htm, Accessed: June 2009.
Dingwall, R. (2006) 'Confronting the anti-democrats: The unethical nature of ethical regulation in social science', Medical Sociology Online, 1(1), 51-8.
ESDS (2009) Qualidata teaching resource: exploring diverse interview types, Economic and Social Data Service, Available online: http://www/esds.ac.uk/ qualidata/support/interviews/, Accessed: June 2009.
Halsey, A. (1989) 'A Turning of the tide? The prospects for sociology in Britain', British Journal of Sociology, 40(3), 369-80.
Hammersley, M. (1997) 'Qualitative Data Archiving: Some Reflections on its Prospects and Problems', Sociology, 31: 131-42.
Savage, M. (2005) 'Revisiting classic qualitative studies', Forum Qualitative Sozialforschung, 1(3). Available online: http://www.qualitative-research.net/ index.php/fqs/article/view/502. Accessed: June 2009.
Harrison, S. & Dowswell, G. (2002) 'Autonomy and bureaucratic accountability in primary care: What English general practitioners say', Sociology of Health and Illness, 24(2), 208-26.
Hughes, J. & Sharrock, W. (1996) The Philosophy of Social Research, London, Longman.
Jenkins, K. (1991) Rethinking History, London, Routledge.
King, G., Keohane, R. & Verba, S. (1994) Designing Social Inquiry, Princeton, Princeton University Press.
Lamont, M. & White, P. (2009) Workshop on Interdisciplinary Standards for Systematic Qualitative Reseach, Washington, National Science Foundation, Available online: http:www.nsf.gov.sbe/ses/soc/ISSQR_workshop_erpt.pdf, Accessed: June 2009.
National Statement on Ethical Conduct in Human Research (2007) Canberra, Commonwealth Government.
Power, M. (1997) The Audit Society: Rituals of Verification, Oxford, Oxford University Press.
Ragin, C., Nagel, J. & White, P. (Eds.) (2004) Workshop on Scientific Foundations of Qualitative Research, Washington, National Science Foundation, Available online: http://www.nsf.gov/sbe/ses/soc/ISSQR_ workshop_rpt.pdf, Accessed: June 2009.
Sommerlad, H. (1999) 'The implementation of quality initiatives and the new public management in the legal aid sector in England and Wales: Bureaucratisation, stratification and surveillance', International Journal of the Legal Profession, 6(1), 311-40.
Ten Have, P. (1999) Doing Conversation Analysis: A Practical Guide, London, Sage.
Thompson, P. & Corti, L. (1998) 'Are you sitting on your qualitative data? Qualidata's mission', Social Research Methodology: Theory and Practice, 1(1), 85-90.
Thompson, P. (2000) 'Re-using qualitative research data: A personal account', Forum Qualitative Sozialforschung, 1(3), Available online: http://www. qualitative-research.net/index.php/fqs/issue/view/27, Accessed: June 2009.
Travers, M. (2001) Qualitative Research Through Case Studies, London, Sage.
Travers, M. (2005) 'A question of ethics', Network: Newsletter of the British Sociological Association, 90: 20.
Travers, M. (2007) The New Bureaucracy: Quality Assurance and its Critics, Bristol, The Policy Press.
Watson, R. (2000) '"Intepretive" sociology in Great Britain: The state of the art', Swiss Journal of Sociology, 26(3), 507-29.
Wooffitt, R. (2005) Conversation Analysis and Discourse Analysis, London, Sage.
i The more general issue is that researchers face institutional pressures to publish and to apply for the next grant. There would be little incentive to spend time preparing a data set, even if a budget was allocated for this purpose.
ii For a sophisticated argument, along these lines, see Abbott (2004).
iii It would be interesting to know if these researchers had unused material relevant to my research questions.
iv For those concerned about research ethics, this demonstrates, incidentally, that qualitativeresearchers do not always accept what their interviewees say at face value. I would not, however, regard this as raising an ethical problem, since the person who supplied the data originally, even if he or she believed that it would be used for particular purposes, does not suffer what could be considered harm.
v Thompson does not simply prefer certain types of data, but has what sounds like a principled objection to traditions founded on different epistemological assumptions. He notes that 'the failing of much qualitative work in the postmodern or narrative modes has been to make the interactive research process the centre of study in itself, and forget what can be learnt from the stories that can be re-told' (Thompson 2000: 12).
vi The principal investigators in an ESRC-funded project were Bruce Guthrie, Guro Huby and Huw Davies, with help from Suzanne Grant and Francis Watkins.
vii One reviewer suggested that all qualitative researchers 'take sampling seriously, not because we are aping the rigor of quantitative analysis, but because we take the rigor of our analysis and the quality of our data seriously'. You will not, however, find much emphasis on sampling or rigour in ethnographies by postmodernists. In some cases the findings are presented as poems (see Travers 2001, Chapter 7).
viii For some interesting observations on how the audit model influences data archives, see Hammersley (1997: 136). For a different way of understanding the objectivity of texts, see Clifford and Marcus (1986).
ix In Australia, consent can be given in different ways under the (2007) National Statement depending on the level of risk (see sections 2.2.4 and 2.2.5). Most ethics committees do, however, require written consent even when interviewing professionals or politicians.
x For a critique of the ESRC in Britain by an interpretivist, see Watson (2000). For discussion of the institutionalisation of empiricism in American social science after the second world war, see Mills (1956).
There are multiple typologies for qualitative interviews but very little consensus among those typologies ... Some interviews aim to gather descriptive data, more typically with many structured or semi-structured interviews, whereas other interviews seek to generate data which probe deeper into the lives of the interviewees. It is usually possible to identify an interview's form as structured, semi-structured or open-ended by looking at a transcript. However, other typologies are derived from methodological perspectives and it is not possible unambiguously to classify an interview as, for example, life history, oral history, or narrative, as these approaches can depend on the analytical framework applied to the transcript. Each style of interview creates different types of data and different forms of knowledge, each requiring a different kind of analysis. In addition to the different interview types, there are also different types of sampling procedures, such as random sampling, purposive/quota, intergenerational, snowball etc., all of which have implications for the types of analysis and interpretation which are possible from the interviews (ESDS 2009).
CONTENTS Section 1: Background 1.1 Policy Background 1.2 Aims of Study Section 2: Data Collection Methods 2.1 Focus Groups 2.1.1 Sampling 2.1.2 Data Collection and Analysis 2.2 Ethnographic Fieldwork Practices 2.2.1 Sampling 2.2.2 Practice Description: "Summerhill Practice" 2.2.3 Practice Description: "Maitland Medical Centre" 2.2.4 Data Collection and Analysis 2.3 Health Board Level Interviews 2.3.1 Sampling 2.3.2 Data Collection and Analysis 2.4 Confidentiality Section 3: Data Management 3.1 Proofreading and Quality Control of Transcripts 3.2 Anonymisation of Data Section 4: Structure and Organisation of Dataset 4.1 Full data listing 4.2 Structure of the Dataset Section 5: Overview of Contextual Information Appendix A: Focus Group Introductory Letter Appendix B: Focus Group Information Sheet Appendix C: Focus Group Topic Guide Appendix D: Practice Introductory Letter Appendix E: Practice Information Sheet Appendix F: Health Board Level Interviews Introductory Letter Appendix G: Health Board Level Information Sheet Appendix H: Health Board Interviews Topic Guide Appendix I: Consent Form Appendix J: Data Archiving Information Sheet Appendix K: Data Archiving Consent Form (Contextual statement, SN5591, Economic and Social Data Service)
... there is a difference between how ethnographers read the fieldnotes they have produced themselves, and how someone else will read them. The fieldworker interprets them against the background of all that he or she tacitly knows about the setting as a result of first-hand experience, a background that may not be available to those without that experience. And much the same problem arises with other sorts of data, even with listening to audio-tapes and watching video-recordings that someone else has produced.
|Gale Copyright:||Copyright 2009 Gale, Cengage Learning. All rights reserved.|