Second Language Acquisition - Theory and Pedagogy: Proceedings of the 6th Annual JALT Pan-SIG Conference.
May. 12 - 13, 2007. Sendai, Japan: Tohoku Bunka Gakuen University. (pp. 1 - 6)

Evidence-based education: Benefits and challenges

by John Wiltshier (Miyagi University)


Abstract

This article explains the origin of the term evidence-based practice before considering the benefits, theoretical opposition, and appropriateness in the field of education. The author believes that researchers and teachers have different orientations and practices and so the type of evidence each finds useful often differs. Following from this belief, the idea of different levels of evidence (LOE) is proposed and two specific levels are outlined: a policy making level and a classroom decision-making level. The article continues by expanding on the usefulness, qualities and variety of classroom decision-making evidence.


Keywords:

evidence-based practice, levels of evidence, narrative-based practice, educational theory


The relatively consequence-free environments in which many teachers work can be breeding grounds for apathy and low job satisfaction. In order to preserve a teacher's self esteem, increase job satisfaction and generally re-energize such situations, a form of evidence-based education can be adopted. Evidence-based education refers to policy and practice that can be justified in terms of sound evidence about their likely effects (Coe, 1999). After briefly explaining the origin, benefits and some theoretical arguments against evidence-based education, this article will propose classifying evidence into two different levels and will outline appropriate use for each level. In this way acknowledging that the role and orientation of researchers and teachers are different. In order to benefit from evidence-based education, each requires a level of evidence sufficient to act as a base for their professional practice.

[ p. 1 ]

Origin

The phrase "evidence-based" was first used in evidence-based medicine, in which doctors refer to scientific evidence to evaluate the likely risks and benefits of a particular treatment. In such practice, the scientific evidence guides the doctor in deciding the most suitable treatment or is used to justify no treatment, where the risks outweigh the likely benefits.
The term evidence-based education was first used by Hargreaves (cited in Brusling, 2005) in 1996 in a lecture given at a teacher training agency. Later, Hargreaves (1997) wrote:
Practicing doctors and teachers are applied professionals.... Doctors and teachers are similar in that they make decisions involving complex judgements. Many doctors draw upon research about the effects of their practice to inform and improve their decisions; most teachers do not, and this is a difference. (p. 407)

A move to evidence-based education requires: (1) utilizing existing evidence and (2) establishing new evidence where current evidence is insufficient or non-existent (Davies 1999). It is no surprise that such a movement started in the UK at a time when the education system was adjusting to the huge reforms which had taken place during the Thatcher era. It was perhaps a response to some government reforms which seemed ill-thought out and over hastily introduced. From my experience as a newly qualified teacher at that time, I can say the education system was in a state of flux and numerous revisions to curriculum attainment targets and the key-stage tests left many teachers confused and stressed. Also such frequent revisions fuelled criticisms that the government needed more research and piloting before introducing sweeping changes.

Avowed benefits

The benefits of adopting an evidence-based approach is that the teacher or policy makers can gain more knowledge about actions and consequences that will lead to more rational decision making. In theory, teachers should be able to more accurately predict the outcomes of their teaching and policy makers should be able to more accurately predict the effect of introducing new curriculum or endorsing a particular methodology. This may seem like common sense, but underlying such a rationale is the assumption that educational practice is similar in character to medical practice.

[ p. 2 ]


Theoretical objections

Hargreaves earlier made a close comparison between the practice of doctors and teachers, but Biesta (2007) argues that education is more than the application of strategies to bring about pre-determined goals. Kennedy (1999), states that case study reports are valued over statistical data in education because "educational events are governed not by universal laws of cause and effect but, instead, by human interactions and by multiple concurrent and interacting influences" (p. 514). Any evidence collected from such interactions only tells us what worked at that particular time and not necessarily what is still working or will work in the future.
Biesta continues that education is not a treatment. It does not involve a simple intervention by the teacher to cause a desired effect. Students need to interpret and make sense of what they are being taught; education is not a simple cause and effect relationship. The process and products are internally related and to assume educational ends are given is a mistake. Evidence-based movements work best when cause and effect are separated and a desired effect is pre-determined. When such is largely the case, as perhaps it can be claimed in the field of medicine, the question is how best to achieve such an effect.
This said, evidence-based medicine alone may not be the best overall approach to providing patient care. A complimentary approach which stresses conversation between doctor and patient, where the doctor is curious about the patient, the context and the history of the illness can compliment an evidence-based approach. Such an approach in the medical field is called narrative-based medicine.
In education a narrative-based approach would similarly place emphasis on the communication between teacher and student. It would be an interactive, creative, humanistic approach which prizes students, ensuring student-centered classrooms. The teacher would empathize with, and be curious about each student's learning style and learning history. The practices involved in narrative-based education are, I imagine, familiar even though the term itself may not be.

[ p. 3 ]

Continuing with theoretical opposition to evidence-based education Biesta (2007) makes the point that evidence-based education does not imply that something is desirable educationally. Data evidence only gives us information about what is happening. Individual teachers need to make their own decisions about whether such actions and consequences are desirable.
Hammersley (2001) states that advocates of evidence-based education "tend to treat practice as the application of research-based knowledge; neglecting the extent to which it necessarily involves uncertain judgment" (p.12). He also states that "critics managed to counter this [claims that their opposition to evidence-based practice was irrational] by denying that practice can be based on evidence; in the sense that research evidence can provide its exclusive foundation" (Hammersley, 2001, p.1).
Such criticisms were perhaps a reaction against the privileging of research evidence over professional experience rather than a direct criticism of using more research to guide educational practice. In response weaker formulations of the evidence-based idea and new terms followed: evidence-informed teaching (Hargreaves 1999), evidence-aware teaching and evidence-influenced education (Davies et al 2000).

". . . what counts as evidence should depend on the type of decision to be taken . . ."


These terms share the underlying idea that research evidence should play a larger part in decisions about education. However, of course, there are many types of decisions related to education. I contend that, what counts as evidence should depend on the type of decision to be taken and copying the medical practice's Levels of Evidence idea would be useful in the field of education.

Levels of evidence

The Oxford Centre for Evidence-Based Medicine (2001) lists five levels of evidence upon which practice can be based. All levels are positively worded. This idea of different positively-worded levels of evidence is, I believe, useful and directly transferable to education.

[ p. 4 ]


Evidence for educational practice should, I suggest, be grouped into two levels: (1) policy making evidence and (2) classroom decision making evidence. Although some of the characteristics and the collection methods of these two levels may be similar, the uses that will be made of the evidence differ.
Policy making evidence is designed for education policy makers to use to make decisions about curriculums and policies that are generalizable across many schools. The evidence needs to be standardized, reliable and valid. This level of evidence must come from extensive studies and may include; randomized testing, studies using control and experimental groups and pilot studies. In general, policy making evidence needs to be drawn from a large database over a long period of time. The other level of evidence I will call classroom decision-making evidence.

Classroom decision-making evidence

Classroom decision making evidence is non-standardized and non-generalizable. Its purpose is for internal use only. It is designed to help a particular teacher and their students. It is not designed for use by policy makers. Having said this, the evidence needs to be as reliable as possible and needs to be valid (measuring what it claims to be measuring).
The benefit of the evidence to the teacher and students must out-weigh the time it takes to collect and analyze the evidence. In addition evidence should be easy to interpret and easy to disseminate to students.
What counts as evidence is debatable (Maxwell, 2004; Morrison, 2001). But evidence that satisfies the criteria listed above for classroom decision making evidence may include copies of written work, audio or video recordings, teacher observations, student questionnaires, tests, progress charts, student feedback, third party observations, peer demonstrations and peer assessments.
Ultimately classroom decision making evidence must be useful. It should help the teacher to more confidently state what is being learnt in class. This in turn helps the teacher to evaluate such learning in line with class or course goals. Furthermore, it provides evidence about current practice that makes future planning easier. In these ways, I believe, teacher professionalism can be enhanced, self esteem can be preserved and job satisfaction can be increased.
Collected evidence can be disseminated to students as teachers see fit. This may help students to feel good about their progress or make them aware they need to do more either alone or with the teacher's help to progress. Providing this type of evidence-based feedback may enhance the admiration and respect students have for their teacher.

[ p. 5 ]


Conclusion
If evidence-based medicine is going to be used as a model for evidence-based education the similarities and differences between medicine and education need to be considered. As teachers we should proceed cautiously. Even in the medical field there is an on-going debate about the limits of evidence-based medicine. Writers such as Hunter (1996), Greenhalgh (1999) and Ban (2003) emphasize the need for a balance between an evidence-based and a narrative-based approach.

". . . the evidence needed for policy making and the evidence needed for everyday teaching is different."

Originally, evidence-based education targeted policy makers. I argue that the evidence needed for policy making and the evidence needed for everyday teaching is different. Teachers do not need to be academic research design specialists. Teachers should only collect evidence that they believe will be helpful for them to teach there students. Suggesting teachers become mini-researchers maybe detrimental because, if compared to full-time researchers, teachers may feel they lack the skills and time required to carry out high quality academic research leading to defensive attitudes and inaction.
Hopefully by thinking about evidence in terms of two levels teachers can separate themselves from full-time education researchers and find easy ways to collect evidence that is useful to them, while at the same time not choking a teacher's creativity and individual flair. The collected evidence can be a starting point to re-designing classes so that they have a higher chance of delivering stated aims and in so doing, create a positive wave running through teacher and students alike.
I hope evidence-based education, in this teacher-friendly form, will be something that is supportive when, as from time to time, we may find ourselves in teaching situations which for one reason or another are not going as we planned.

References

Ban, N. (2003). Continuing care of chronic illness: Evidence-based medicine and narrative-based medicine as competencies for patient-centered care. Asia Pacific Family Medicine, 2, 74-76.

Biesta, G. (2007). Why "what works" won't work: Evidence-based practice and the democratic deficit in educational research. Educational Theory, 57, 1-22. Retrieved May 2, 2007, from http://www.blackwell-synergy.com/toc/edth/57/1

Brusling, C. (2005, November). Evidence-based practice in teaching and teacher education. Paper presented at the conference of Professional Development of Teachers in a Lifelong Perspective: Teacher Education, Knowledge Production and Institutional Reform. Centre for Higher Education Greater Copenhagen in Collaboration with OECD, Copenhagen, Denmark. Retrieved February 17, 2007 from http://www.samford.edu/ctls/evidencebasedpracticeinteachingcb.pdf

Coe, R. (1999). A manifesto for evidence-based education. Retrieved March 12, 2007 from http://www.cemcentre.org/RenderPage.asp?LinkID=30317000

Davies, P. (1999). What is evidence-based education? British Journal of Educational Studies, 47, 108-121.

Davies, H.T.O., Nutley, S.M. & Smith, P.C. (Eds.). (2000). What works? Evidence-based policy and practice in the public services. Bristol: Policy Press.

Greenhalgh, T. (1999). Narrative based medicine. British Medical Journal, 318, 48-50. Retrieved January 29, 2007 from http:/www.bmj.com/cgi/content/full/318/7175/48

Hammersley, M. (2001, September). Some questions about evidence-based practice in education. Paper presented at the symposium on "Evidence-based practice in education" at the Annual Conference of the British Educational Research Association, University of Leeds, England.

Hargreaves, D. (1997). In defense of research for evidence-based teaching: A rejoinder to Martyn Hammersley. British Educational Research Journal, 23, 405-419.

Hargreaves, D. (1999). Revitalising educational research: Lessons from the past and proposals for the future. Cambridge Journal of Education, 29, 239-249.

Hunter, K. (1996). Don't think zebras: uncertainty, interpretation, and the place of paradox in clinical education. Theoretical Medicine, 17, 225-241.

Kennedy, M. (1999). A test of some contentions about educational research. American Educational Research Journal, 36, 511-541.

Maxwell, J. A. (2004). Causal explanation, qualitative research and scientific inquiry in education. Education Researcher, 33, 3-11.

Morrison, K. (2001). Randomised controlled trials for evidence-based education: Some problems in judging 'what works'. Evaluation and Research in Education, 15, 69-83.

Oxford Centre for Evidence-based Medicine. (2001). Levels of evidence. Retrieved April 3, from http://www.cebm.net/index.aspx?o=1025


2007 Pan SIG-Proceedings: Topic Index Author Index Page Index Title Index Main Index
Complete Pan SIG-Proceedings: Topic Index Author Index Page Index Title Index Main Index

Last Main Next

[ p. 6 ]