Sample
Paper: Karkou V. & Sanderson P. (2000), Dance Movement Therapy in UK Education
The Royal Charter for the Arts and Humanities Research Council (undated) states that it is responsible for promoting “basic, strategic and applied research” The definition given can be summarised in terms of defining research questions, issues or problems to be addressed. Research must also state why it is important that the specified issues and/or problems are addressed and, particularly “it should also explain the rationale for the chosen research methods and why they provide the most appropriate means by which to answer the research questions, issues or problems” (AHRC 2009, p. 1). This leads to a clarification of what a methodology is, and what a method is. There are numerous explanations, one summary one being that “method refers to the particular method involved” whereas “methodology refers to the general study of methods and involves arguments about what kinds of methods are appropriate” (Wagenen 1991, p. 66). With these points in mind, we can consider the paper in question.
Fundamentally, Karkou and Sanderson (2000) write an interesting and informative article concerning Dance Movement Therapy (DMT) in the UK, and inasmuch as they explore the educational issues within and around this relatively new subject area, their efforts should be applauded. Unfortunately, however, there are weaknesses in the methods employed and this essay will not only consider them, but will also suggest some alternative approaches.
The Paper
One starting point is with the aims and objectives. Apart from an overall aim to describe DMT as “fully as possible,” these were to “gain information relating to theory, methodology and assessment/evaluation of the practice of DMT, as well as details of the practitioners themselves and their client groups” (Karkou and Sanderson 2000, p. 73). The authors go on to outline a ‘research methodology’ (Karkou and Sanderson 2000, p. 74), which seems more like the particular approach which they are going to take to conduct their research, which should rather be described as ‘methods.’
This leads us to the actual methods employed. These were in three stages, first “preliminary research, consisting of semi-structured interviews with leading practitioners in the area, who acted as key informants.” Indeed, a total of five were interviewed and these are named as ‘key informants.’ However, when consideration is given to the general justifications for semi-structured interviews using key informants, it seems to be a rather unusual method for constructing an interview questionnaire. Smith and Osborn (2008, p. 53) make the point, for example, that the “aim of interpretative phenomenological analysis (IPA) is to explore in detail how participants are making sense of their personal and social world,” in other words it uses a normative, human-value and post-positivist approach to try to elicit deep and meaningful events in people’s lives. In sum, the researcher is trying to “get close to the participant’s personal world” (Smith and Osborn 2008, p. 53). Furthermore, in terms of such a qualitative approach, semi-structured interviews can be described as the “exemplar” approach to interpretative phenomenological analysis because they enable the researcher to adjust the questions so that any areas of interest that come up can be probed more thoroughly (Smith and Osborn 2008, p. 57).
Boyce and Neale (2006, p. 3) confirm the points made about in-depth interviews and go on to note that the selection of key informants is an important part of the process. Indeed, whilst one part of the sampling must, by definition, be purposive (in this case, only potential participants with a good understanding of DMT), there should be some explanation of which criteria were used and which groups or individuals were approached. It is unsatisfactory and confusing, furthermore, to state that semi-structured interviews “were conducted with leading practitioners in the area,” who acted as key informants, and then in the next sentence state that “the interviews were completed with dance movement therapists” (Karkou and Sanderson 2000, p. 74). In a general sense, furthermore, the way that the interviews were planned and conducted should be explained in detail, as should how they were recorded (written or audio-recorded), and how and when informed consent was obtained.
It is stated (Karkou and Sanderson 2000, p. 74) that, as a result of the semi-structured interviews, a questionnaire was raised, then piloted and distributed to practising and registered arts therapists. The reasons for the low response (41 out of 155 returned) are listed, but it should have been noted that this method represented convenience sampling, and that although perhaps unavoidable, it is generally considered to be ‘weak’ (Lunenburg and Irby 2008, p. 174). The method employed for test piloting the questionnaire would also have been useful (for example how many respondents were used in the pilot, how they were selected etc.).
We then come to the identification of a group, which was those 16 of the 41 respondents who had identified themselves in terms of education as being their “main working environment,” as opposed to the rest of the 41, of whom at least six more were “working in education” (Karkou and Sanderson 2000, p. 74). Perhaps some clarification is required here, because the issue is rather confusingly presented. Fundamentally, the researchers received 41 responses, and were thus in possession of 41 questionnaires, all of which had been completed by people who were “practising arts therapists registered with an arts therapies association” (Karkou and Sanderson 2000, p. 74). The researchers then decided to focus on the smaller group and used the other 25 responses as a group with which to compare the 16 who had stated that education was their main working environment (Karkou and Sanderson 2000, p. 74).
This leads to a question of what the purpose of trying to establish differences between the two groups was, and Karkou and Sanderson (2000, p. 74) state that it was to show “descriptive results from dance movement therapists responding to the questionnaire and working mainly in education” and “significant differences between this group of practitioners and DMT therapists who were not working in education are revealed and discussed.” However, more than half of those working in the whole group were “working in one way or another in education” (Karkou and Sanderson 2000, p. 74), so the delineation was based on how long in a given time period each was working in education, but this is not stated. If we presume that on the basis of an answer to a general question asking whether they are ‘mainly’ working in education or not, the groups were separated, then it is based on a subjective and non-defined response by those being questioned. In other words, the answer will have been dependent upon the interpretation of the question. This required much more detail and explanation before it could be considered to be a satisfactory method or approach.
Chi-square analysis is utilised in an attempt to better understand the differences which may exist between the expected proportions and the observed proportions within categories (Lunenburg and Irby 2008, p. 78). In these terms, it may be considered an appropriate approach (notwithstanding the previously noted issues concerning sampling). The key determination is whether observed differences are due to either sampling error or coincidence, and this is represented in the significance value (p). However, it is important to understand that any level of significance means that there is some chance that the result is a random event, but that the normal ‘convention’ is, effectively, 5 per cent. It could therefore be argued that if 20 tests were run, we could randomly expect that one would produce such a level of ‘significance.’ In Table 2 (Karkou and Sanderson 2000, p. 76), there are a total of 17 tests and two suggest significance. A further potentially confounding issue comes from a footnote reference at the bottom of Table 2, which states that “more than one answer was accepted.” This means, if it is interpreted as stated, that each respondent was free to state that any number of the theoretical principles proposed were important to them. Thus, one respondent could answer that one was and another that all 17 were. Therefore, within such a relatively small sample, the results could be influenced by the subjective interpretation of the answers between respondents and a large difference in this interpretation between one or a few respondents could produce a flawed result. This leads to a brief mention of sampling size. It is acknowledged that this factor is a matter of opinion, for example McDonald (2009) draws our attention to the numbers in each category, with small numbers likely to “decrease the power of your test to detect a significant difference.” Unfortunately, we don’t know the number of respondents who stated a belief that ‘object relations theory’ and ‘play therapy’ were important theoretical approaches, thus we can question the validity of the p values.
Table 3 and Table 4 (Karkou and Sanderson 2000, pp 77 - 78) list 28 variables, and the authors found ‘significance’ in 6 of the comparisons. This time, it would appear that each respondent was asked to categorise their answers by 1 = agree, 2 = uncertain and 3 = disagree. The mean values of their answers were used to establish significant differences. If we take the largest difference in the answers, which was 0.52 (also, naturally, the most ‘significant’), we can calculate that if 10 of the smaller sample (16) had answered 1 (agree) and 6 answered 3 (disagree), the average would have been 1.75 (28/16). If 19 of the larger group (25) had answered 1 (agree) to the same question and 6 had answered 2 (uncertain), then the average would have been 1.24 (31/25). Thus, we can have an (almost) equally significant result when the majority of respondents give the same answer and 6 in each case are differentiated by the subjective words ‘uncertain’ and ‘disagree.’
Alternative Methods
Research in education can embrace numerous methodologies and the precise methods to be adopted require careful and considered contemplation. One part of the thought process must be what we are trying to find out or, perhaps more relevantly, what are the research objectives or questions? In this case, these were, as noted above, an overall aim to describe the field of DMT, and more specifically to “gain information relating to theory, methodology and assessment/evaluation of the practice of DMT, as well as details of the practitioners themselves and their client groups” (Karkou and Sanderson 2000, p. 73). Surely the words ‘describe,’ theory’ and ‘details’ are suggestive of normative, human values, and therefore of a post-positivist approach. Hatch (2002 pp 131-4) discusses some of these issues and notes that, for example, a focus group approach can enable the information taken from such a method to be ‘triangulated’ with other sources. This type of approach surely goes further towards encompassing the hugely complex but entirely relevant issue of human values and human judgements.
Almost any research method must be a compromise in terms of available respondents, time and money. Therefore, while a focus group approach may have been a worthwhile suggestion, the semi-structured interviews have been shown to have been feasible. Therefore, the alternative approach is a predominantly qualitative study which focuses first on semi-structured interviews with carefully and rationally chosen key informants. As a secondary issue, the quantitative data would be utilised in a support capacity but only using average values and with appropriate discussion on their severe limitations. Fundamentally, it would be used to produce some graphical support for the qualitative study where appropriate. As a supplement, if possible, it would also be appropriate to address one further area of the research objectives, namely the clients, which would be conducted in the form of focus group interviews if appropriate permission and informed consent could be obtained. In sum, a mixed methods approach.
Such an approach would utilise the lost opportunities from the original paper, the approach of which may have been influenced by some tangible or intangible pressure to conduct a quantitative study, and strengthen the findings with deeper understandings of values emerging from semi-structured interviews which utilised open-ended questions.
Ethical Issues
In the original paper (as opposed to the revised suggestions, which would present some more challenging ethical issues), there are a number of points that should be made, starting with the most obvious. Informed consent would have been required from the participants in the semi-structured interviews, and added consent if audio recordings were made. Furthermore, informed consent would have been required from the questionnaire respondents. In an overall sense, it should have been kept in mind that the researcher and the teachers may have different ethical professional requirements (Roberts et al 2005), and that in a further general sense, the research is about humans and is aimed at human benefit. Therefore, apart from informed consent, confidentiality must be protected and relevant educational board permissions obtained where and if applicable (Roberts et al 2005).
References
Arts and Humanities Research Council Charter (undated), www.ahrc.ac.uk/.../Documents/AHRC%20Royal%20Charter.pdfស្រដៀង
Arts and Humanities Research Council (2009), Definition of Research, www.ahrc.ac.uk/About/.../Definition%20of%20Research2.pdfស្រដៀង
Boyce C. & Neale P. (2006), Conducting in-Depth Interviews: A Guide for Designing and Conducting in-Depth Interviews for Evaluation Input, Pathfinder International
Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed
methods approaches. London: Sage.
Flick, U. (2009). Managing Quality in Qualitative Reasearch. London: Sage
Greener, I. (2011). Designing social research: A guide for the bewildered. London:
Sage.
Hatch J. A. (2002), Conducting Qualitative research in Educational Settings, State University of New York Press
Hesse- Biber. Sh. N. (2010). Mixed Methods Research. Merging theory with
Practice. London: The Guilford Press.
Karkou V. & Sanderson P. (2000), Dance Movement Therapy in UK Education, Research in Dance Education, 1:1, pp 69-86
Kvale, S. (2007). Doing Interviews. London: Sage
Lunenburg, F. C. & Irby, B. J. (2008), Writing a Successful Thesis or Dissertation,
Corwin Press, USA, 2008
Newby, P. (2010). Research methods for education. Harlow: Pearson
Pring, R. (2010). Philosophy of educational research. London: Continuum.
Roberts W. L., Geppert C., Coverdale J., Edenharder K. (2005), Ethical and Regulatory Considerations in Educational Research, Academic Psychiatry Vol. 29, No. 1
Rowntree, D. (2000) Statistics without tears: An introduction for non-mathematicians.
London: Penguin.
Smith A. J. & Osborn M. (2008), Interpretative Phenomenological Analysis in Smith A. J. (ed.) (2008), Qualitative Psychology, A Practical Guide to Research Methods, 2nd ed., Sage Publications, London
Thomas, G. (2009). How to do your research project. London: Sage.
Van Wagenen R. K. (1991), Writing a Thesis, Substance and Style, Prentice Hall Inc.