According to Michael Winerip of the New York Times, "A recently released study has concluded that computers are capable of scoring essays on standardized tests as well as human beings do." Pearson graders can grade at most 30 writing samples in an hour, whereas e-Rater (an automated grader developed by the Educational Testing Service) can grade 16,000 in 20 seconds. Does this mean humans should be replaced by e-Rater?
Les Pereleman, a director of writing at Massachusetts Institute of Technology, doesn't think so. Mr. Pereleman, in between teaching undergraduates, has researched algorithms like e-Rater, and found many flaws and loops in their development. He noticed that essays that are simply more wordy (although not as well-written) will be graded with a higher score. Longer sentences, paragraphs, and sentences not starting with the words "and" and "or" will score higher. Also, an essay which uses more complex words, no matter in what context, will get a higher score than an essay which doesn't.
Looks like we humans will not be replaced by robots in the world of teaching and testing just yet.
Link to article: Facing a Robo-Grader? Just Keep Obfuscating Mellifluously
Justyna, very clever title! I am strongly against electronic test taking. Not only do I despise testing as a whole, but I believe that there are many flaws with electronics handling test scores than I can even count, especially in our fields. Since each teacher is looking for a different thing in the questions they ask, the only way that this strategy would be remotely applicable is if EVERY teacher was to code their own electronic grading system; i.e. not use a standardized one.
ReplyDeleteAnd something tells me that will not be happening.
-Joe