On the Grading of Writing

Not too long ago, my next-door neighbor who is a retired history professor from the University of Chicago sent me an article from The New York Times titled “Essay Grading Software Offers Professors a Break.” He and I are both avid news watchers, especially when it comes to stories related to education, and we often stand in our driveways and compare notes on what we read.

This particular New York Times article explains that a nonprofit called EdX, founded by folks from Harvard and MIT, has developed software that will grade written portions of standardized exams and give the student test-taker instant feedback, which would replace having the writing sent to professional academics, who apparently take too long and work too hard. I see a couple of problems here, right off the bat . . .

The developer is quoted in the article as saying:

“There is a huge value in learning with instant feedback,” Dr. Agarwal said. “Students are telling us they learn much better with instant feedback.”

Yes, of course, they do. (If you’re not picturing Veruca Salt right now, you should be.) However, most educators I know agree that this generation of students is marked by expecting instant gratification— and it’s not a necessarily good thing to give it to them just because they want it. I’m not surprised that young people prefer not to wait and have what they want now. Young people also love the rush of energy given to them by sugary snacks, like candy bars, but that doesn’t change the lack of nutrition in them.

The software also allows students to get their scores, and then go back and try again to improve their scores! (This time, I’m picturing a kid in a front yard football game yelling, “Do over!” because the long pass hit the telephone wires.) As a teacher who wants to prepare my students for the way life really is – we have to wait on things we want sometimes and there aren’t always do-overs – I didn’t get a warm fuzzy feeling about what Dr. Agarwal was proclaiming as one of the benefits of his software, partially because I can hear some students saying, “You don’t grade as fast that computer program, so will you just run it through there? It also lets me try again when I don’t make a good grade.”

Thankfully, the opponents, a group called Professionals Against Machine Scoring of Student Essays in High Stakes Assessment, were also quoted:

“Let’s face the realities of automatic essay scoring,” the group’s statement reads in part. “Computers cannot ‘read.’ They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others.”

Even beyond those obvious concerns that computerized grading software won’t do a good job, I have more concerns— and it regards the future of teaching writing. My first concern is that this program might be more widely applied and used and thus turn writing instruction into the “low score = try again” method, where the teacher is a mere proctor and the computer is the “expert.” I don’t think the EdX folks fully understand just how egregiously some people could misuse what they are producing, and I believe that because of this statement in the NYT article:

The EdX assessment tool requires human teachers, or graders, to first grade 100 essays or essay questions. The system then uses a variety of machine-learning techniques to train itself to be able to grade any number of essays or answers automatically and almost instantaneously.

But Dr. Agarwal is very proud of his work. He claims:

“This is machine learning and there is a long way to go, but it’s good enough and the upside is huge,” he said. “We found that the quality of the grading is similar to the variation you find from instructor to instructor.”

First of all, if you aren’t thinking about Hal 9000 by now, you should be.

Second, other companies are following EdX’s lead, according to the article. For example, Hewlett-Packard has versions of similar software in development. Of course, their supposed intention is that it will relieve stress from writing instructors and help students get feedback that is hard to get in overcrowded classes.

And that’s the point in my chagrin: the folks touting this software aren’t connecting the dots that constitute the actual problems and current realities in education. Students don’t get timely feedback, because classes are overcrowded— lots of students means lots of papers. Schools don’t need to pump dollars into more computer workstations and new software to solve the problem of giving feedback faster; they need to pump dollars in hiring more writing instructors to teach smaller numbers of students, which will allow them to give feedback in a timely manner.

What I want people to see about EdX’s proposal is the same basic principle that gun-rights advocates make when it comes to opposing background checks: if you open the door just a little bit, you never know who might exploit that tiny window and what flood might result.

The National Council of Teachers of English has already published a lengthy rebuttal of machine-scored writing proposals. The position statement comes out in blunt terms against machine-scoring, listing nine bullet points of how computers cannot do what a well-educated, thoroughly trained human writing instructor can do. Please read it.

Poor old Dr. Agarwal may mean well. He probably does. But he’s a little too much like Victor Frankenstein for my taste: well-informed people are telling him, Don’t do this, it won’t turn out well, and he and others like him apparently aren’t listening. I also want people to be clear about this: educators work in that career field because we want to educate people, and software developers get into that career field because they want to create software that people will buy! Writing instructors are saying no to this, despite the developers claiming that they made it to help us. What I see is a company that is projecting the possibilities for a fortune to be made in government contracts when their software replaces human employees.

Machines and computers provide excellent solutions to some of our difficulties and quandaries in human life. For example, I am thankful that machines wash and dry my clothes and that I don’t have to sit in my backyard with a washtub and clothesline all weekend. I also love my Mac, my Apple products, and the internet— obviously, I’m writing a blog!

But the machine-scoring of writing is a bad idea. Plain and simple.

I receive hundreds of student papers to grade every term. I work many evenings and weekends grading papers, and that work takes away from my leisure time and time with my family. But I still don’t want computer software to do my job for me. As a writing instructor, I enjoy and believe in what I do. I want the work done right, and I want to do it myself to ensure that it gets done right. I didn’t go into this line of work to outsource my grading duties, and I doubt if any of my colleagues in this field did either. We live for this stuff!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.