Transparent and Reproducible Social Science Research: How to Do Open Science - Hardcover

Miguel, Edward; Christensen, Garret; Freese, Jeremy

Hardcover

ISBN 10: 0520296931 ISBN 13: 9780520296930

Verlag: University of California Press, 2019

Inhaltsangabe

Recently, social science has had numerous episodes of influential research that was found invalid when placed under rigorous scrutiny. The growing sense that many published results are potentially erroneous has made those conducting social science research more determined to ensure the underlying research is sound. 

 

Transparent and Reproducible Social Science Research is the first book to summarize and synthesize new approaches to combat false positives and non-reproducible findings in social science research, document the underlying problems in research practices, and teach a new generation of students and scholars how to overcome them. Understanding that social science research has real consequences for individuals when used by professionals in public policy, health, law enforcement, and other fields, the book crystallizes new insights, practices, and methods that help ensure greater research transparency, openness, and reproducibility. Readers are guided through well-known problems and are encouraged to work through new solutions and practices to improve the openness of their research. Created with both experienced and novice researchers in mind, Transparent and Reproducible Social Science Research serves as an indispensable resource for the production of high quality social science research. 

 

Über diesen Titel

Über die Autorin bzw. den Autor

Garret Christensen is an Economist at the U.S. Census Bureau and was formerly a Research Scientist at the Berkeley Institute for Data Science and Berkeley Initiative for Transparency in the Social Sciences. His research focuses on the impacts of social safety-net programs. Jeremy Freese is Professor of Sociology at Stanford University, and Co-Principal Investigator of the General Social Survey and Time-Sharing Experiments in the Social Sciences. His research focuses on topics that connect social inequality, health, and social change. Edward Miguel is Oxfam Professor in Environmental and Resource Economics in the Department of Economics at the University of California, Berkeley, and Director of the Center for Effective Global Action. His research focus is on African economic development.

Von der hinteren Coverseite

“In this profoundly important book, Christensen, Freese, and Miguel outline a pragmatic, seemingly modest plan for rehabilitating the traditional scientific norms of integrity and transparency. Many people drawn by these norms into social science are troubled by institutional incentives that increasingly reward quantity over quality, tradeoffs over rigor, and impact factors over insight. Students preparing for a career in the social sciences should read it carefully.”—Paul Romer, corecipient of the 2018 Nobel Memorial Prize in Economic Sciences

“An excellent one-stop-shop introduction to the topic, it summarizes key advances in this area and provides important practical advice for researchers.”—David McKenzie, World Bank

“This book comes at just the right time. As norms shift around the question of what it means to conduct social science with integrity, this book offers a how-to guide and a thoughtful reflection on the ethos of our research. How can science be more open and transparent? How can researchers make sure, across the hundreds of choices they make regarding research design, analysis, and interpretation, that they don’t ‘fool themselves’ into believing findings that aren’t real? This book is part of the vanguard of efforts to establish new norms of social science research. The authors use clear, compelling, and occasionally funny examples and explanations that help to outline a modern vision of scientific integrity.”—Betsy Levy Paluck, Professor of Psychology & Public Affairs, Princeton University

“Social science can help people make better decisions. P-hacking, publication biases, and related practices do the opposite—they mislead other researchers, students, policy makers, and the public. We have to do better. Christensen, Freese, and Miguel’s first-of-its-kind guidebook is loaded with compelling insights and concrete advice about how to better serve those who depend on us. It is an essential reference for social science’s next generation of innovators.”—Arthur Lupia, Hal R. Varian Collegiate Professor of Political Science, University of Michigan

“The authors will convince you and your students to change how you do your work. Only by reforming our practices can we move forward together to produce social science that is both cumulative and credible.”—Stephen L. Morgan, Bloomberg Distinguished Professor, Johns Hopkins University

“This is the first work to draw together the exploding area of methodological and practical work to increase the credibility of social-science research through transparency and reproducibility.”—Graeme Blair, Assistant Professor of Political Science, University of California, Los Angeles

“A clear-eyed take on the state of social science and how it can live up to the promise of providing quality evidence regarding fundamental questions about humans and society.”—Simine Vazire, Associate Professor of Psychology, University of California, Davis

“An essential handbook for anyone engaged in empirical research in the social sciences. Situating open and transparent research practices as a fundamental component of ethical research practices, the authors provide an interdisciplinary, inspiring, and practical guide for students, instructors, and experienced researchers alike. It is chock full of helpful tips, concrete examples, and exceedingly persuasive evidence that it is part of our duty as social scientists to support new norms aimed at maximizing learning from research.”—Susan D. Hyde, Professor of Political Science, University of California, Berkeley, author of The Pseudo-Democrat’s Dilemma

Auszug. © Genehmigter Nachdruck. Alle Rechte vorbehalten.

Transparent and Reproducible Social Science Research

How to Do Open Science

By Garret Christensen, Jeremy Freese, Edward Miguel

UNIVERSITY OF CALIFORNIA PRESS

Copyright © 2019 Garret Christensen, Jeremy Freese, and Edward Andrew Miguel
All rights reserved.

ISBN: 978-0-520-29693-0

Contents

List of Figures, 
List of Tables, 
Acknowledgments, 
PART ONE. INTRODUCTION AND MOTIVATION, 
1 Introduction, 
2 What Is Ethical Research?, 
PART TWO. PROBLEMS, 
3 Publication Bias, 
4 Specification Searching, 
PART THREE. SOLUTIONS, 
5 Using All Evidence: Registration and Meta-analysis, 
6 Pre-analysis Plans, 
7 Specification Searching Solutions: Sensitivity Analysis and Other Approaches, 
PART FOUR. PRACTICES, 
8 Reporting Standards, 
9 Replication, 
10 Data Sharing, 
11 Reproducible Workflow, 
12 Conclusion, 
Appendix, 
Notes, 
Bibliography, 
Index, 

CHAPTER 1

Introduction

THE NEED FOR TRANSPARENT SOCIAL SCIENCE RESEARCH

Contemporary society is complex and rapidly changing. Leaders of government, corporate, and nonprofit institutions all face a constant stream of choices. Thankfully, these leaders are increasingly investing in data acquisition and analysis to help them make good decisions. Researchers are often charged with providing this information and insight, in areas ranging from environmental science to economic policy, immigration, and health care reform. Success often depends on the quality of the underlying research. Inaccurate research can lead to ineffective or inappropriate policies, and worse outcomes for people's lives.

How reliable is the current body of evidence that feeds into decision making? Many believe it is not reliable enough. A crisis of confidence has emerged in social science research, with influential voices both within academia (Manski 2013) and beyond (Feilden 2017) asserting that policy-relevant research is often less reliable than claimed, if not outright wrong. The popular view that you can manipulate statistics to get any answer you want captures this loss of faith in the research enterprise, and the sense that too many scientific findings are mere advocacy. In this era of "fake news" and the rise of extremist political and religious movements around the world, the role of scientific research in establishing the truth as common ground for public debate is more important than ever.

Let's take, for example, the case of health care reform in the United States — the subject of endless partisan political debate. This tension can be partly explained by the simple fact that people feel strongly about health care, a sector that affects everyone at one time or another in their lives. But there are also strong ideological disagreements between the major U.S. political parties, including the role government should play in providing social services, and the closely related debate over tax rates, since higher taxes generate the revenue needed for health programs.

What role can research play in such a volatile debate? The answer is "It depends." Some people — and politicians — will hold fast to their political views regardless of evidence; research cannot always sway everyone. But data and evidence are often influential and even decisive in political battles, including the 2017 attempt by congressional Republicans to dismantle the Affordable Care Act (ACA), or Obamacare. In that instance, a handful of senators were swayed to vote "Nay" when evidence from the Congressional Budget Office estimating the likely impact of ACA repeal on insurance coverage and health outcomes was released. Media coverage of the research likely boosted the program's popularity among American voters.

The answers to highly specific or technical research questions can be incredibly important. In the U.S. case, findings about how access to health insurance affects individual life outcomes — including direct health measures, as well as broader economic impacts such as personal bankruptcy — have been key inputs into these debates. How many people will buy insurance under different levels of subsidies (i.e., what does the demand curve for health insurance look like)? How do different institutional rules in the health insurance marketplace affect competition, prices, and usage? And so on.

When the stakes are this high, the accuracy and credibility of the evidence used become extremely important. Choices made on the basis of evidence will ultimately affect millions of lives. Importantly, it is the responsibility of social science researchers to assure others that their conclusions are driven by sound methods and data, and not by some underlying political bias or agenda. In other words, researchers need to convince policymakers and the public that the statistical results they provide have evidentiary value — that you can't just pick out (or make up) any statistic you want.

This book provides a road map and tools for increasing the rigor and credibility of social science research. We are a team of three authors — one sociologist and two economists — whose goal is to demonstrate the role that greater research transparency and reproducibility can play in uncovering and documenting the truth. We will lay out a number of specific changes that the research community can make to advance and defend the value of scientific research in policy debates around the world. But before we get into the nitty-gritty or "how," it is worth surveying the rather disappointing state of affairs in social science research, and its implications.

HOUSTON, WE HAVE A PROBLEM: RESEARCH FRAUD AND ITS AFTERMATH

If you thought we'd have research methods all figured out after a couple centuries of empirical social science research, you would be wrong. A rash of high-profile fraud cases in multiple academic disciplines and mounting evidence that a number of important research findings cannot be replicated both point to a growing sense of unease in the social sciences. We believe the research community can do better.

Fraud cases get most of the headlines, and we discuss a few of the most egregious cases here. By mentioning these examples, we are not claiming that most researchers are engaging in fraud! We strongly believe that outright fraud remains the exception rather than the rule (although the illicit nature of research fraud makes it hard to quantify this claim or even assert it with much confidence). Rather, fraud cases are the proverbial canaries in the coal mine: a dramatic symptom of a much more pervasive underlying problem that manifests itself in many other ways short of fraud. We will discuss these subtler and more common problems — all of which have the ability to distort social science research — at length in this book.

The field of social psychology provides a cautionary tale about how a lack of transparency can lead to misleading results — and also how the research community can organize to fight back against the worst abuses. In recent years, we have seen multiple well-publicized cases in which prominent tenured social psychologists, in both North America and Europe, were caught fabricating their data. These scholars were forced to resign from their positions when colleagues uncovered their misdeeds. In the circles of scientific hell, this one — simply making stuff up and passing it off as science — must be the hottest (Neuroskeptic 2012).

Perhaps best known is the case of Diederik Stapel, former professor of...

Details zur Ausgabe

• Verlag: University of California Press
• Erscheinungsdatum: 2019
• Sprache: Englisch
• ISBN 10: 0520296931
• ISBN 13: 9780520296930
• Einband: Gebundene Ausgabe
• Anzahl der Seiten: 266
• Kontakt zum Hersteller: Nicht verfügbar
• Verantwortliche Person: Nicht verfügbar