Finding cheaters using multiple-choice comparisons

http://jd-mathbio.blogspot.com/2015/02/finding-cheaters-using-multiple-choice.html

by smcgivern • 11 years ago

44 55 11 years ago

jd-mathbio.blogspot.com

MollyR2• 11 years ago

I don't particularly like the idea of harassing possibly innocent students. Innocent students might be under pressure with no proof of innocence, since they assumed to be guilty. They might do a cost/benefit analysis and admit guilt.

Also if cheating is really such a concern, why not just handle it in a similar way to the american GRE's or SAT's ?

mixedmath• 11 years ago

When you suggest handling it like American GREs and SATs, what do you mean? Do you mean computer-given multiple choice tests? Or is there something about how they handle paper and pencil exams that you think should be widely adopted?

bodyfour2• 11 years ago

A good start would be to not let students pick which seat they take the test in. They can still attempt to copy their neighbor's work but can't collude with someone to make it easy.

MollyR2• 11 years ago

I'm going more by there security practices in general, not really one in particular . https://www.ets.org/gre/institutions/about/security/

I think if academic integrity is huge issue, perhaps we should adopt some formal rules for all tests, rather than being at the whims of different professors, or universities.

Essentially we need some kind of check and balance before we try to discredit students on a metric like the one being described this article.

The GRE's as a specific example have a high degree of trust and a fairly proven track record for being secure and preventing cheating. It might not be that much of stretch to apply whatever they seem to be doing to combat academic integrity issues.

I think I rambled a bit, but I hope that made sense

bodyfour2• 11 years ago

I agree -- visually judging outliers is a pretty poor test of statistical significance. The fact that they sat near each other does give them some weight though.

The big problem is that it only works for people who share a fairly unique answer set. In other words, they need to all have a decent number of wrong answers which correlate. What about people at the bottom-right of the graph who had a high correlation simply because they answered nearly all of the questions correctly? Certainly most of them are innocent, but how would we know? It seems this technique can only finger people who cheat and still end up with a mediocre grade, not those who cheat and end up acing the exam.

xkcd-sucks• 11 years ago

Based on the policies of the two universities at which I've taught, it seems like there's no possible benefit to admitting guilt.

The only foolproof way to "convict" someone of cheating is to have (several) people witness/record the incident. If a student accused of cheating categorically denies everything, they're usually in the clear (although faculty will say nasty things in private).

navait• 11 years ago

Breaking connections to faculty has consequences, namely in letters of recommendation.

dubya• 11 years ago

This varies between schools. At my current school it's fairly straightforward to say "I saw this student looking at X's paper" and show that the answers are more similar than would be expected. Multiply choice would be harder of course, but free response or incorrect math calculations are pretty convincing. Students have recourse, but the appeals process here is not much work for the faculty member making the accusation until maybe the final level.

dfgsdfgs• 11 years ago

I did my undergrad in this gentleman's university. This author exemplifies the problem with Mac's education:

Student aren't cheating on multiple choice tests. Professors too lazy to do their jobs are.

sago2• 11 years ago

What is Mac? I didn't get that from the piece, he just started using it.

F3TTT• 11 years ago

Apparently McMaster Uni? https://www.blogger.com/profile/03197609136976213552

quadrature• 11 years ago

Its not just the tests, plagiarism is absolutely rife in Canadian Universities and the institutions are doing very little to combat it.

CPLX• 11 years ago

Today I learned that "invigilator" is a word.

http://www.merriam-webster.com/dictionary/invigilate

DanAndersen• 11 years ago

A few years ago, I was an ESL teacher at a university in southern China. As an American I remember having a fair amount of confusion when first talking with one of the Chinese teachers at the university, who had been taught with more British English, and kept using the word "invigilate" that I had never heard of.

spikej• 11 years ago

What word do you use instead? "Exam Administrator"?

jackmaney2• 11 years ago

Likewise. I mainly use "proctor" for that purpose.

MAGZine2• 11 years ago

I'm a proctor, and I invigilate exams.

littletimmy5• 11 years ago

Cheating is a natural consequence of society putting a huge value on grade-point-averages and other dullard measures of achievement.

If I had to sit through a lecture room with 200 students (TWO HUNDRED!) and I was absolutely confident that the professor cannot possibly give a shit about my learning in this crowd, I too would cheat.

hurin3• 11 years ago

It's probably a 1st year lecture, those students that are actually concentrating in biology or a related-field already took an AP in high-school, and the rest need to fill graduation requirements with a science course. It's a test to see if you read a textbook for 20 hours.

chucksmash3• 11 years ago

"a natural consequence of society......"

You could use this justification to rationalize any sort of unethical behavior you like. Embezzlement is a natural consequence of society putting a huge value on owning a Benz and other dullard measures of achievement!

That is to say, I disagree wholeheartedly.

littletimmy5• 11 years ago

There is a world of difference between the importance of a GPA and the importance of a Benz.

Low GPA means no good job straight out of college (because apparently you need a 3.5+ from an ivy league school to model in excel) means in a lot of cases lesser income for the rest of your life.

No Benz means you can drive a Toyota. Big whoop.

hnnewguy• 11 years ago

You could use this justification to rationalize any sort of unethical behavior you like.

It isn't a justification; it's an explanation.

Because, I agree: if there wasn't such a societal emphasis on material goods (ie the Benz), there would be less embezzlement.

pavel_lishin3• 11 years ago

Wouldn't that only harm you? Wouldn't you only be cheating yourself?

I don't understand why your professors' interest in your personal learning would impact your own desire to learn.

(Thinking about it a bit more, I can vaguely understand this if it's a required class that you have no actual interest in - an elective you had to take because all others were taken, or if you were retaking a class that you were already very familiar with - but at that point, cheating would be redundant.)

littletimmy5• 11 years ago

I understand the difference between our thoughts. Where we differ is that I think college is not a place of learning. In general, it is a sequence of hoops to sort out the labor force.

mannykannot2• 11 years ago

Cheating harms every honest student, by casting doubt on their efforts and achievements.

speakeron• 11 years ago

"I did put all 6 of them through a bit of an ordeal, though, and at least half of them deserved it."

The author sounds like a really nasty piece of work.

afandian• 11 years ago

How widespread is multiple choice at university level? I've never come across it.

jackmaney2• 11 years ago

For exams, it's nearly unheard of in small classes (ie <= 30 or so people), but in large lecture halls with hundreds of students, it's not uncommon (especially for general education large lecture courses).

ska• 11 years ago

Right, for service courses with lots of sections, the marking load can be pretty high (e.g 1000+ students taking the same course) and the difficulty in maintaining even marking policies both tempt towards scantron and the like.

It is limiting, but if you put a lot of work into it you can even design pretty decent multi-part questions that reward partial understanding/success. Mostly people don't put that time in, though, and I've never been a fan of the practice.

dweinus• 11 years ago

I wonder if the front-back case is the top-most red dot. This would be explained if the front person was innocent and the person behind them copied. Not having the benefit of collusion, they have more shared wrong answers. Also, they have less answers in common overall, which supports the non-collusion theory.

bildung• 11 years ago

Multiple choice tests simply are a horrible way of assessing knowledge in any non-trivial domain. Instead of wasting time with writing R scripts the author really should grab himself a book about education.

Edit: On second thought that's probably worded too harsh, but multiple choice testing for a final exam in biology? There's not the slightest chance the results will correlate with the capabilities of his students at all.

hashmymustache• 11 years ago

I'm surprised he didn't break it down to probabilities according to answer distributions instead of right vs wrong. For instance, given 2 questions with their answers and the percent of the class that answered for each choice:

1. A is the answer A - 60% B - 20% C - 5% D - 15%

2. C is the answer A - 10% B - 5% C - 70% D - 15%

The probability of randomly selecting 2 students that both chose:

- A and C (both correct) is 0.1764 - D and B (both wrong) is 0.000056

* Appreciate corrections

potatoyogurt• 11 years ago

That doesn't necessarily work because wrong answers can have natural correlations. Let's say you have two questions about finding a slope on an exam. Anyone who thinks that slope is change in x over change in y is going to get both of those questions wrong in the same way, even if those wrong answers are both individually rare. It would be really hard to distinguish between when these correlations are expected and when they're unexpected.

Finding cheaters using multiple-choice comparisons

Related Stories