Marking Essays by algorithms. Thoughts?

[zofromuxo]

Why marking essays by algorithm risks rewarding the writing of ‘bullshit’

Picture this: you have written an essay. You researched the topic and carefully constructed your argument. You submit your essay online and receive your grade within seconds. But how can anyone read, comprehend and judge your essay that quickly?

Well, the answer is no one can. Your essay was marked by a computer. Would you trust the mark you received? Would you approach your next essay with the same effort and care?

These are the questions that parents, teachers and unions are asking about automated essay scoring (AES). The Australian Curriculum, Assessment and Reporting Authority (ACARA) proposes to use this program to grade essays, like persuasive writing questions, in its NAPLAN standardised testing scheme for primary and secondary schools.

But it will not matter what a text is about: whether the argument is ethical, offensive or outright nonsensical, whether it conveys any coherent ideas or whether it speaks effectively to the intended audience.

The only thing that matters is that the text has the right structural patterns. In essence, algorithmic marking might reward the writing of “bullshit” – text written with little regard for the subject matter and solely to fulfil the algorithm’s criteria.

I personally agree that currently I wouldn't want machine marking it due to the ability to game the system, but if this were overcome: "Would you trust an algorithm to mark you school work?"

I'll be interested to see people's response to this. I also hope this is posted in the right board as well, I apologise to the mods and admins,if I posted this in the wrong board.

[Aaron]

For something that has a finite answer (e.g. maths), a marking algorithm would be better suited than an essay or something that is significantly open to interpretation.

A teacher knows his/her students, their capabilities and personal circumstances which can influence how an essay is written. A teacher can interpret why certain words and sentences have been written and can navigate and award partial marks etc. An algorithm does not.

Obviously I am not an English teacher so I can't really comment regarding the effectiveness of such an algorithm, however I know as a teacher that I would rather mark them myself because I know my students.

[zofromuxo]

For something that has a finite answer (e.g. maths), a marking algorithm would be better suited than an essay or something that is significantly open to interpretation.

A teacher knows his/her students, their capabilities and personal circumstances which can influence how an essay is written. A teacher can interpret why certain words and sentences have been written and can navigate and award partial marks etc. An algorithm does not.

Obviously I am not an English teacher so I can't really comment regarding the effectiveness of such an algorithm, however I know as a teacher that I would rather mark them myself because I know my students.

But for the English exam where an external assessor marks them would you still prefer a human as opposed to an algorithm?

Also for subjects like mathematics, I agree that an algorithm is better suited due to having a clear cut answer. However, do you think it would struggle marking student's working out?
Some people like writing out every step and others write only the key components.
Could that lead to students writing only the key components being mistakenly penalised? Due to the machine only knowing to get the answer from working out the question in full as opposed to just the key steps/