Running Randomized Evaluations: A Practical Guide - Softcover

Glennerster, Rachel; Takavarasha, Kudzai

Softcover

ISBN 10: 0691159270 ISBN 13: 9780691159270

Verlag: Princeton Univers. Press, 2013

Inhaltsangabe

A comprehensive guide to running randomized impact evaluations of social programs in developing countries This book provides a comprehensive yet accessible guide to running randomized impact evaluations of social programs. Drawing on the experience of researchers at the Abdul Latif Jameel Poverty Action Lab, which has run hundreds of such evaluations in dozens of countries throughout the world, it offers practical insights on how to use this powerful technique, especially in resource-poor environments. This step-by-step guide explains why and when randomized evaluations are useful, in what situations they should be used, and how to prioritize different evaluation opportunities. It shows how to design and analyze studies that answer important questions while respecting the constraints of those working on and benefiting from the program being evaluated. The book gives concrete tips on issues such as improving the quality of a study despite tight budget constraints, and demonstrates how the results of randomized impact evaluations can inform policy. With its self-contained modules, this one-of-a-kind guide is easy to navigate. It also includes invaluable references and a checklist of the common pitfalls to avoid.Provides the most up-to-date guide to running randomized evaluations of social programs, especially in developing countries Offers practical tips on how to complete high-quality studies in even the most challenging environments Self-contained modules allow for easy reference and flexible teaching and learning Comprehensive yet nontechnical

Über diesen Titel

Über die Autorin bzw. den Autor

Rachel Glennerster is executive director of the Abdul Latif Jameel Poverty Action Lab (J-PAL) at the Massachusetts Institute of Technology. Kudzai Takavarasha holds degrees in chemical engineering and economics from the Massachusetts Institute of Technology, and was a policy manager at J-PAL from 2004 to 2012.

Von der hinteren Coverseite

"The popularity of randomized evaluations among researchers and policymakers is growing and holds great promise for a world where decision making will be based increasingly on rigorous evidence and creative thinking. However, conducting a randomized evaluation can be daunting. There are many steps, and decisions made early on can have unforeseen implications for the life of the project. This book, based on more than a decade of personal experience by a foremost practitioner and a wealth of knowledge gathered over the years by researchers at J-PAL, provides both comfort and guidance to anyone seeking to engage in this process."--Esther Duflo, codirector of J-PAL and coauthor of Poor Economics

"Randomized impact evaluations, pioneered by Rachel Glennerster and her colleagues, have emerged as a powerful new tool in the fight against poverty. This book gives us a critical guide for how to actually do them. It is required reading for students, professionals, and researchers committed to using evidence to guide policy."--Dean Karlan, coauthor of More Than Good Intentions

"I have been searching for a book to train my students, research assistants, and field staff to design and implement social policy experiments. My search is over. Running Randomized Evaluations provides practical guidance covering all stages of an experiment--wisdom that previously could be acquired only by working directly for an evaluation expert."--Jeffrey B. Liebman, John F. Kennedy School of Government, Harvard University

"Impact evaluation is a key resource for improving development effectiveness, yet until now it was inaccessible to people--policymakers, NGO staff, and development professionals of all kinds. Oftentimes, we get stuck in the theoretical discourse and philosophical debate and miss the chance to talk about the practical toolkit needed to actually do impact evaluation. Running Randomized Evaluations will significantly contribute to accelerate advancements in the field by making the tools of the trade accessible and user-friendly. This is a must-read guide for policymakers, practitioners, and donors alike."--Jodi Nelson, Bill and Melinda Gates Foundation

Auszug. © Genehmigter Nachdruck. Alle Rechte vorbehalten.

Running Randomized Evaluations

A PRACTICAL GUIDE

By RACHEL GLENNERSTER, KUDZAI TAKAVARASHA

PRINCETON UNIVERSITY PRESS

Copyright © 2013 Princeton University Press
All rights reserved.

ISBN: 978-0-691-15927-0

Contents

Preface....................................................................
vii
Abbreviations and Acronyms.................................................
ix
1 The Experimental Approach................................................
1
2 Why Randomize?...........................................................
24
3 Asking the Right Questions...............................................
66
4 Randomizing..............................................................
98
5 Outcomes and Instruments.................................................
180
6 Statistical Power........................................................
241
7 Threats..................................................................
298
8 Analysis.................................................................
324
9 Drawing Policy Lessons...................................................
386
Appendix...................................................................
421
Glossary...................................................................
443
Index......................................................................
453

CHAPTER 1

The Experimental Approach

This chapter provides an example of how a randomized evaluationcan lead to large-scale change and provides a road map for an evaluationand for the rest of the book. The modules in this chapter are asfollows:

MODULE 1.1: The Power of Randomized Evaluations

MODULE 1.2: A Randomized Evaluation from Start to Finish

MODULE 1.1 The Power of Randomized Evaluations

This module provides an example of how a small nongovernmental organization,by subjecting its program to rigorous evaluation, can generateevidence that can change the lives of millions of people.

In 1994 I went with Michael Kremer to visit the family he had livedwith for a year in rural Kenya. We met up with many of Michael's oldfriends, including Paul Lipeyah, who told us of the work he was doingwith International Child Support (ICS) Africa, a nongovernmentalorganization (NGO) helping government schools in Busia, a neighboringdistrict in Kenya's Western Province. Paul asked us what advicewe might offer for improving the effectiveness of ICS programs. CouldMichael help evaluate what they were doing? Michael suggested randomizedevaluation: if ICS wanted to understand the impact of theirprograms, they could randomly choose the schools in which theyworked and the order in which they phased in new programs.

Over the following years, ICS randomly evaluated many approachesto improving education outcomes, including providing additional inputs(classrooms, textbooks, teachers); reducing the cost of going toschool (providing free school uniforms and school meals); and offeringperformance incentives (merit scholarships for girls, bonuses forteachers who attended school regularly). Sometimes the programshad the expected impact, and sometimes they did not. But ICS, inpartnership with a growing number of researchers, kept innovatingand testing in the areas of education, agriculture, women's empowerment,clean water, and health. Their knowledge of how to improvelives and how to evaluate programs kept growing.

In 1997, ICS (in line with World Health Organization recommendations)phased in a program of treating children en massefor intestinal worms (such as hookworm and schistosomiasis). Theresults were astonishing. Deworming reduced the absenteeism ofchildren in local schools by 25 percent, making it the most cost-effectiveprogram for increasing schooling that ICS had tried. Long-termresults showed that women who had been dewormed as girlsreceived more education and were more likely to grow cash crops,whereas men who had been dewormed worked 3.5 hours longer perweek and were more likely to hold manufacturing jobs and earnhigher wages.

On the strength of the evidence, in 2009 Prime Minister RailaOdinga announced a program to deworm 3 million of Kenya's mostat-risk children. In 2012 the program was expanded to include preschoolchildren, in part on the basis of further evidence of cogitativegains for young children from deworming. In 2013, programs to deworm40 million children are being implemented in Kenya andaround the world. ICS could never have hoped to reach so many childrenwith their own programs, and yet through the influence of theirevaluation they have helped millions of children in Kenya—andaround the world.

Since 1994 we have learned a lot about which programs work andwhich do not and also about how to run randomized evaluations.Until that point, most randomized evaluations of social programswere performed in rich countries in partnership with governmentsand at very high cost. But the experience of partnerships betweenresearchers and organizations such as ICS showed that it was possibleto conduct high-quality randomized evaluations with small organizationson limited budgets and in very poor settings. Although manygovernment programs are being evaluated with randomized evaluations,NGOs have also proved to be flexible and innovative partners inthis effort to learn. The challenge of working with new partners onnew questions and on modest budgets has spurred innovation in randomizedevaluation methodology.

We have learned how to introduce randomization into programsin creative ways, account for and measure spillovers, reliably measuredifficult-to-measure outcomes like corruption and empowerment,get the maximum statistical power from a very limited budget, minimizeattrition, and design evaluations that answer fundamental questionsabout why humans behave the way they do and how to motivatechanges in behavior.

In this book we have gathered many of the practical innovationsfrom this large body of work. Our goal is to enable more people andorganizations to undertake and commission high-quality randomizedimpact evaluations and thus to build a wider and deeper evidencebase on how best to combat poverty. Our hope is that we will see thegeneration and application of rigorous evidence grow even faster thanin the past two decades. By innovating and testing and by feeding theevidence back into even more innovation, evaluators and practitionerscan improve the effectiveness of policies and programs and makea real difference in people's lives.

MODULE 1.2 A Randomized Evaluation from Start to Finish

In this module we provide an overview of the steps in planning and runningan evaluation and indicate where these topics are covered in therest of this book. We tell the story of a randomized evaluation of an educationprogram in India that I experienced firsthand from inceptionthrough implementation and analysis to scale-up.

Starting right: Choosing the right question to test

Ten years after my trip to Kenya described in Module 1.1 , I was workingwith a group of researchers at the Massachusetts Institute of Technology(MIT) planning our next evaluation. We had recently startedwhat is now the Abdul Latif Jameel Poverty Action Lab (J-PAL) withthe objective of promoting the...

Details zur Ausgabe

• Verlag: Princeton Univers. Press
• Erscheinungsdatum: 2013
• Sprache: Englisch
• ISBN 10: 0691159270
• ISBN 13: 9780691159270
• Einband: Taschenbuch
• Anzahl der Seiten: 480
• Kontakt zum Hersteller: Princeton University Press; https://press.princeton.edu/books; ; 7th Floor, New South Building; PRINCETON; NJ; 8544; USA
• Verantwortliche Person: GPSR Kontakt; gpsr@libri.de; ; Europaallee 1; Bad Hersfeld; 36244; Deutschland