Book Review
This book review appeared on this week's TCRecord website. Comments?
International Research on the Impact of Accountability Systems: Teacher Education Yearbook XV
reviewed by Laura Hamilton — September 25, 2007
Title: International Research on the Impact of Accountability Systems: Teacher Education Yearbook XV
Author(s): Louise F. Deretchin and Cheryl J. Craig (Eds.)
Publisher: Rowman & Littlefield, Lanham
ISBN: 1578865476, Pages: 251, Year: 2007
Search for book at Amazon.com
This Yearbook addresses a timely topic. In the United States, a debate is under way to reshape the No Child Left Behind (NCLB) legislation as the law’s reauthorization approaches. There is a clear need for research to inform this process: Much of the praise as well as criticism for the law is based on anecdotes regarding its effects on schools and classrooms rather than on a systematic review of evidence. At the same time, several other nations are experimenting with new systems for motivating improved student and educator performance, and many of them are looking to the experiences of countries with long histories of test-based accountability to determine the best way forward. In addition, there is a resurgence of interest in pay-for-performance programs that would use test scores alone or in combination with other criteria to determine teacher or principal pay. A volume that brings together international evidence on these effects is a welcome contribution to the ongoing policy discussion surrounding various forms of test-based accountability (TBA). Although this book does not provide a complete or fully balanced review of existing evidence, its chapters cover a large number of important issues and examine TBA in a wide variety of contexts, so the book should be useful to anyone thinking about how to create accountability systems that promote high-quality teaching and learning.
The major strength of this volume is the attention given to the experiences and perspectives of teachers, both in the chapters and in the commentaries that introduce and conclude each section. Whether TBA polices eventually lead to positive or negative outcomes will depend not only on how those systems are designed, but on how the educators who are subjected to their provisions respond to them. Research on school reform implementation has demonstrated the importance of teacher support and buy-in, and the broader literature on performance-based accountability in a variety of sectors reveals the need for incentive systems to include goals that are perceived as fair, important, and attainable. This book provides teacher perspectives gathered in a variety of contexts, and together the chapters suggest some important lessons about how teachers respond to accountability and what really matters to them as professionals.
A majority of the chapter authors express skepticism about the value of standardized tests and the accountability policies that rely on them. Some of the skepticism is informed by the research, but in several cases the authors’ negative views appear to be unsupported by evidence and shaped by a somewhat simplistic view of measurement and of current TBA systems. For example, references to the consequences NCLB imposes for failure to meet AYP tend to focus on the punitive aspects and fail to recognize the support systems the law is intended to promote. Recent research on NCLB implementation suggests that districts are providing extensive support to schools identified as needing improvement, and that some of the law’s more severe sanctions are rarely imposed even when schools fail to meet their targets for several years. In addition, despite the volume’s title, the United States is given much more emphasis than any other nation, except in the final section that discusses promising approaches to accountability. This emphasis is not surprising, given the recent focus on NCLB and the upcoming reauthorization, but readers should be aware that the representation of international material in the book is somewhat limited.
The Yearbook is divided into four sections. Each section is introduced by an overview and framework section written by educators from K-12 schools or from university-based teacher education programs, and the same set of authors provide concluding remarks at the end of the section. These commentaries provide a valuable practitioner perspective and help bring coherence to each section.
The first section, “Philosophy, History, and Design of Accountability Systems,” includes three chapters that address some of the broad influences of externally mandated accountability on classrooms, schools, and school systems. All three chapters discuss the limited value of the kind of measurement that is central to today’s test-based accountability systems, and describe some of the problems that arise when measurement becomes the primary motivator. Some of the assertions appear to emphasize extreme rather than typical responses to accountability, such as in Chapter 2 when the author describes an environment of “performativity” in which “Raising standards (as indicators of quality) is the aim and nothing else seems to matter” (p.16), and criticizes scripted curricula by claiming that “Teachers following this scripted curriculum hardly have to do any thinking themselves—not even in terms of adapting the materials or the learning activities to the particular characteristics and needs of their actual students and the particular circumstances in the school” (p.17). There is evidence that in many schools, the response to standards and accountability has been more nuanced and that multiple goals continue to drive practice. And even if complete standardization were the goal of the curriculum developers and of those who decided to adopt the curriculum, the reality in schools is that teachers make a large number of decisions, do a lot of thinking themselves, and implement even heavily scripted curricula differently than their colleagues next door or down the hall. The tendency of some chapter authors to use overly strong language or to focus on extreme responses diminishes the utility of the book to some degree. At the same time, all three chapters make important points about the ways accountability can negatively affect instruction, and the experiences as described by teachers in the chapters and the commentary are worth consideration.
The second section includes five chapters that focus on the consequences of accountability systems. The chapters address a range of outcomes and populations, and together they provide a good sampling of the kinds of issues that should be attended to by those who implement TBA systems, though all but one of these focus exclusively on the United States. This section contains one of the only chapters in the volume that provides a reasonably favorable view of TBA. Chapter 4, by Fletcher, Strong, and Villar, illustrates how the growing availability of longitudinally linked achievement test scores can facilitate program evaluation, and provides a helpful discussion of what conditions are necessary to support good use of data in evaluation. Although the authors mention some of the concerns raised in other chapters—such as the narrow emphasis of many standardized tests—for the most part the chapter emphasizes the value of this information for promoting better decision making. The following chapter, by Lee, provides another analytic example, this time using TIMSS data to study the effects of high school exit exams on student outcomes. Although the TIMSS data are of limited value for answering questions about country-level policy effects, the analysis provides some suggestive evidence of pathways through which high-stakes exams might influence affective outcomes.
The remaining three chapters in this section address important outcomes related to testing. Unfortunately the authors’ arguments, which are often compelling, are in some cases tainted by misrepresentations of the research or of the accountability systems themselves. In Chapter 6, for example, Klug makes some important points about the ways NCLB fails to address the needs of Native American students, but she also makes some overly strong statements about the punitive nature of NCLB, which fail to acknowledge that although the law imposes sanctions, there are also technical assistance provisions and school supports that are supposed to be provided.
Section three, “Perceptions and Perspectives of Accountability Systems,” includes one chapter that discusses teacher education in the U.S. from the perspective of an author who had previously worked in Canada; the author provides an interesting outsider’s perspective and presents some good ideas for improving teacher education while accommodating the current policy environment (e.g., by improving value-added modeling systems through the incorporation of higher-quality assessments into those systems). The second chapter presents an analysis of teachers’ drawings expressing their reactions to NCLB; the findings are consistent with those presented in earlier sections, emphasizing feelings of frustration resulting from seemingly unattainable goals. The third chapter applies a framework for democracy to education and analyzes the extent to which NCLB fails to conform to a democratic notion of public education. The findings described in these chapters should not be interpreted as broadly representative of teachers’ perspectives, but can serve as useful case studies to complement some of the larger, survey-based research that has been published recently.
The last section, “Underpinnings of Powerful Accountability Systems,” presents a broader conceptualization of accountability than the test-based variety that is the focus of most of the earlier chapters. Interestingly, it is also the only section that consists exclusively of non-U.S.-based research, which is consistent with the generally critical view of U.S. accountability policy that pervades the volume. Together, the three chapters in this section point out the value of recognizing the place of education in the broader society, and the need to provide teachers with support and opportunities for collaboration, self-reflection, and continuous growth. These chapters point out many of the conditions necessary for promoting high-quality teaching, and for the most part they are consistent with other research, emphasizing ongoing, school-based (rather than external) professional development, principals’ instructional leadership, strong professional communities, a social safety net for students, and continuous self-evaluation using a variety of formative and summative approaches. Given the heavy U.S. emphasis in previous sections, it would have been informative to include at least one example of districts or schools in the U.S. where these conditions are present and where responses to NCLB have been more productive.
The volume does not attempt to answer some of the most fundamental questions about TBA, but gives readers a wealth of information and opinions to help them reach their own conclusions. Some of the key questions that are raised and that may be informed by the chapters, though not answered definitively, include whether it is possible to design TBA systems that promote rather than restrict teacher autonomy and professionalization, whether elements of other nations’ TBA systems should be adopted by the U.S. in the upcoming NCLB reauthorization, and the extent to which standardized test scores should continue to play a role in evaluating the success of educational interventions. The volume’s editors raise additional questions in their concluding remarks, and provide some useful suggestions for needed research. Although this book has limitations and does not provide a complete summary of what is known about the effects of TBA, it should be helpful for informing the continuing debate on how to create TBA systems that lead to beneficial outcomes for students.
Cite This Article as: Teachers College Record, Date Published: September 25, 2007
http://www.tcrecord.org ID Number: 14622, Date Accessed: 10/10/2007 10:51:42 AM