Machine Scoring of Student Essays: Truth and Consequences

ISBN 10: 087421632X / ISBN 13: 9780874216325

Verlag: Utah State University Press, 2006

Beschreibung

Satisfaction Guaranteed or your money back.

Bestandsnummer des Verkäufers 087421632X

Bibliografische Details

• Titel: Machine Scoring of Student Essays: Truth and Consequences
• Verlag: Utah State University Press
• Erscheinungsjahr: 2006
• Sprache: Englisch
• ISBN-10: 087421632X
• ISBN-13: 9780874216325
• Einband: Softcover
• Zustand: New

Über diesen Titel

Inhaltsangabe

The current trend toward machine-scoring of student work, Ericsson and Haswell argue, has created an emerging issue with implications for higher education across the disciplines, but with particular importance for those in English departments and in administration. The academic community has been silent on the issue—some would say excluded from it—while the commercial entities who develop essay-scoring software have been very active.

Machine Scoring of Student Essays

is the first volume to seriously consider the educational mechanisms and consequences of this trend, and it offers important discussions from some of the leading scholars in writing assessment.

Reading and evaluating student writing is a time-consuming process, yet it is a vital part of both student placement and coursework at post-secondary institutions. In recent years, commercial computer-evaluation programs have been developed to score student essays in both of these contexts. Two-year colleges have been especially drawn to these programs, but four-year institutions are moving to them as well, because of the cost-savings they promise. Unfortunately, to a large extent, the programs have been written, and institutions are installing them, without attention to their instructional validity or adequacy.

Since the education software companies are moving so rapidly into what they perceive as a promising new market, a wider discussion of machine-scoring is vital if scholars hope to influence development and/or implementation of the programs being created. What is needed, then, is a critical resource to help teachers and administrators evaluate programs they might be considering, and to more fully envision the instructional consequences of adopting them. And this is the resource that Ericsson and Haswell are providing here.

Auszug. © Genehmigter Nachdruck. Alle Rechte vorbehalten.

MACHINE SCORING OF STUDENT ESSAYS

Truth and Consequences

UTAH STATE UNIVERSITY PRESS

Copyright © 2006 Utah State University Press
All right reserved.
ISBN: 978-0-87421-632-5

Contents

Introduction Patricia Freitag Ericsson and Richard H. Haswell....................................................................................11 Interested Complicities: The Dialectic of Computer-Assisted Writing Assessment Ken S. McAllister and Edward M. White..........................82 The Meaning of Meaning: Is a Paragraph More than an Equation? Patricia Freitag Ericsson.......................................................283 Can't Touch This: Reflections on the Servitude of Computers as Readers Chris M. Anson.........................................................384 Automatons and Automated Scoring: Drudges, Black Boxes, and Dei Ex Machina Richard H. Haswell.................................................575 Taking a Spin on the Intelligent Essay Assessor Tim McGee.....................................................................................796 ACCUPLACER's Essay-Scoring Technology: When Reliability Does Not Equal Validity Edmund Jones..................................................937 WritePlacer Plus in Place: An Exploratory Case Study Anne Herrington and Charles Moran........................................................1148 E-Write as a Means for Placement into Three Composition Courses: A Pilot Study Richard N. Matzen Jr. and Colleen Sorensen.....................1309 Computerized Writing Assessment: Community College Faculty Find Reasons to Say "Not Yet" William W. Ziegler...................................13810 Piloting the COMPASS E-Write Software at Jackson State Community College Teri T. Maddox.......................................................14711 The Role of the Writing Coordinator in a Culture of Placement by ACCUPLACER Gail S. Corso.....................................................15412 Always Already: Automated Essay Scoring and Grammar-Checkers in College Writing Courses Carl Whithaus.........................................16613 Automated Essay Grading in the Sociology Classroom: Finding Common Ground Edward Brent and Martha Townsend....................................17714 Automated Writing Instruction: Computer-Assisted or Computer-Driven Pedagogies? Beth Ann Rothermel............................................19915 Why Less Is Not More: What We Lose by Letting a Computer Score Writing Samples William Condon.................................................21116 More Work for Teacher? Possible Futures of Teaching Writing in the Age of Computerized Writing Assessment Bob Broad...........................22117 A Bibliography of Machine Scoring of Student Writing, 1962-2005 Richard H. Haswell............................................................234Glossary..........................................................................................................................................244Notes.............................................................................................................................................246References........................................................................................................................................251Index.............................................................................................................................................262

Chapter One

INTERESTED COMPLICITIESS The Dialectic of Computer-Assisted Writing Assessment Ken S. McAllister and Edward M. White

She knew how difficult creating something new had proved. And she certainly had learned the hard way that there were no easy shortcuts to success. In particular, she remembered with embarrassment how she had tried to crash through the gates of success with a little piece on a young author struggling to succeed, and she still squirmed when she remembered how Evaluator, the Agency of Culture's gateway computer, had responded to her first Submission with an extreme boredom and superior knowledge born of long experience, "Ah, yes, Ms. Austen, a story on a young author, another one. Let's see, that's the eighth today-one from North America, one from Europe, two from Asia, and the rest from Africa, where that seems a popular discovery of this month. Your ending, like your concentration on classroom action and late night discussion among would-be authors, makes this a clear example of Kunstlerroman type 4A.31. Record this number and check the library, which at the last network census has 4,245 examples, three of which are canonical, 103 Serious Fiction, and the remainder ephemera. (Landow 1992, 193-194)

This excerpt from George Landow's tongue-in-cheek short story about "Apprentice Author Austen" and her attempts to publish a story on the international computer network, thereby ensuring her promotion to "Author," suggests a frightful future for writing and its assessment. The notion that a computer can deliver aesthetic judgments based on quantifiable linguistic determinants is abhorrent to many contemporary writing teachers, who usually treasure such CPU-halting literary features as ambiguity, punning, metaphor, and veiled reference. But Landow's "Evaluator" may only be a few generations ahead of extant technologies like the Educational Testing Service's e-rater, and recent developments in the fields of linguistic theory, natural language processing, psychometrics, and software design have already made computers indispensable in the analysis, if not the assessment, of the written word. In this chapter, we approach the history of computer-assisted writing assessment using a broad perspective that takes into account the roles of computational and linguistics research, the entrepreneurialism that turns such research into branded commodities, the adoption and rejection of these technologies among teachers and administrators, and the reception of computer-assisted writing assessment by the students whose work these technologies process.

Such a broad treatment cannot hope to be comprehensive, of course. Fortunately, the field of computer-assisted writing assessment is sufficiently well established that there exist numerous retrospectives devoted to each of the roles noted above-research, marketing, adoption, and use-many of which are listed in the bibliography at the end of this book. Our purpose here in this first chapter of an entire volume dedicated to computer-assisted writing assessment is to offer readers a broad perspective on how computer-assisted writing assessment has reached the point it occupies today, a point at which the balance of funding is slowly shifting from the research side to the commercial side, and where there is-despite the protestations of many teachers and writers-an increasing acceptance of the idea that computers can prove useful in assessing writing. This objective cannot be reached by examining the disembodied parts of computer-assisted writing assessment's historical composition; instead, such assessment must be treated as an extended site of inquiry in which all its components are seen as articulated elements of a historical process. This complex process has evolved in particular ways and taken particular forms in the past half century due to a variety of social and economic relations that have elevated and devalued different interests along the way.

In the following sections we trace this web of relations and suggest that theoretically informed practice in particular circumstances-what we will be calling...