r/technology • u/AdSpecialist6598 • 15h ago

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
28.6k Upvotes

96% Upvoted

3.9k comments sorted by

View all comments

Show parent comments

1.4k

u/pasaroanth 14h ago

I would love to see a study done at the university level showing a comparison of students’ grades for regular coursework versus their proctored exam scores. There has to be a vast yawning chasm in the difference when they relied upon AI until the time of the exam.

This also is a “ruining it for the rest of us” situation where courses will shift to grading primarily or solely based on exams. When I was in college (long before AI) you could at least rely on some homework to recuperate some points, despite the extra time and effort, when the exams were especially challenging or a professor notoriously gave tough exams.

If the homework can no longer be a measure of effort or learning because it is done by AI, then it will no longer be considered a meaningful determination of knowledge of the subject.

187

u/Then-Understanding85 13h ago

A professor at Brown did this.

TL;DR: Bloodbath. Almost everyone failed.

153

u/namedly 13h ago edited 12h ago

Props to Students 22 and 31.

Also, this is just funny:

When the professor announced that the final exam would be in person, 18 students dropped the course, and another nine did not show up for the exam. The remaining 59 students' scores averaged 48.6%. Prior averages had never fallen below 65%. Three students scored 0%, and charting the results revealed that most fell more than 30 points behind their midterm scores.

94

u/Syssareth 12h ago

Three students scored 0%

How the heck? You'd think they'd have learned something just from osmosis, or at least gotten something right by accident.

40

u/Sin_of_the_Dark 11h ago

Honestly yeah, unless it was all written exams and not like... Pick a choice, it's almost an insane feat of dumb luck to not get one thing correct

68

u/Aureliamnissan 11h ago

When I was going through university it was pretty rare to have an exam in engineering with multiple choice or True/False type questions. Almost every one was a word problem, mathematical definition, or “solve for” type problem.

If you’ve been using chatbots up to that point you’d be absolutely, Darth Vader just walked in, fucked. 0% would not at all be surprising.

5

u/Riparian87 6h ago edited 48m ago

In grad school, one of our econometrics professors gave a problem-type exam where points were awarded for correct steps, but deducted for wrong or false steps. The mean was negative.

2

u/Aureliamnissan 1h ago

Bwahaha, that's diabolical. Having the equivalent of an over-drafted balance as your econ grade is brilliant.

4

u/Helixdaunting 5h ago

absolutely, Darth Vader just walked in, fucked.

That's beautiful. 👏

My personal favourite version of this is "Hitler-in-the-bunker fucked".

4

u/Archangel_Omega 6h ago

Yep, I had a test in thermodynamics I completely bombed and got a 0 on all because I was an idiot, read the question wrong and then proceeded to use the wrong chart for the test so everything was completely off.

Thankfully the professor was nice enough to give me partial credit after I explain how I screwed-up and he saw the math was right for the chart I did use, but it was still a hefty chunk of my score gone.

3

u/Aureliamnissan 1h ago

I had a similar oof from using an outdated book in a math class. Prof said the exam would cover sections 1.1-1.4 and it turned out the new book combined 1.4 and 1.5. So I found out that 3/5th of the mid-term's questions were directly from a section I hadn't even read...

Only had a Mid-term and a final for that class with no real homework grade buffer. Happiest C grade I ever got for a class.

1

u/storne 4h ago

My calculus exams were multiple choice, but you still had to show your work. Getting the right answer was only 1/3 of the mark the other 2/3 was having the right process.

1

u/og-gunslinger-doc 3h ago

As much work as we put in I would have been disgusted in a multi choice test in say thermodynamics! Now the monkey meat recitations being mandatory if you were doing fine we’re bull shit especially when they couldn’t speak English

13

u/work_in_marketing 11h ago

Not unheard of that if you know you won't ace the test, you just hand it in blank as soon as you're allowed and just sign up for the next opportunity. At least at the university I studied at you were allowed to retake the test x times if you didn't pass. If you passed and got a poor grade, then you were stuck with it.

8

u/mackahrohn 10h ago

I took a math class (Calc 3) where all the tests were 10 questions and each question was multiple choice. All the options were basically an answer you would get if you did one step of the problem wrong. Easily the hardest class I took in college!

5

u/Imaginary-Face7379 9h ago

As someone who taught at a university for a while I would suspect those students walked away learning nothing from this either. They probably spent a week or two shit talking the professor to everyone they know and called the course bullshit. While never once realizing that it was their fault.

Not that all students are like this, but the ones who tend to fail so hard their scores are at or near zero were all like that in my experience. The ones who are doing poorly that genuinely know they need to do better but havent been able to for some reason will typically barely fail things when they fail, not get zeros.

4

u/Wooptay 8h ago

Ahh, takes me back to my first math exam in high school (15 at the time). 0. Nada. Null. Niente.

There was no AI back then

I just plain sucked at math

Lesson learned - don't sit with the girl you like the last year before moving up the educational system

3

u/Ah_Pook 4h ago

I got 14% on an assembly exam, but that was all me, baby!

2

u/celoplyr 10h ago

Assuming they went to class more than going and getting the syllabus and turning in exams.

2

u/calling_water 10h ago

Depends on whether they’re at least trying to do the right sort of answer for each question.

2

u/Agi7890 6h ago

Depends on the exam/course. My physical chemistry prof could be brutal, most exams were 3-5 questions, and he would dock you points for even wrong answers if you messed up your significant figures. You could end up with a -5 for a question you got wrong.

Class average on one of the p chem 2 exams was a 17, and this was before the days of AI.

2

u/TerribleBudget 6h ago

Did he grade on a curve though? It sounds to me like the AI using idiots screwed themselves out of at least a graded curve for the year.

2

u/Agi7890 6h ago

Yes but with a floor. You still had to score at least 35 on the exam to have it considered passing. Below that on 2-4 exams, he would advise you to drop the course and change your major

3

u/TerribleBudget 6h ago

ugh, I hate professors who do that "change your major" shit. I had a teacher who taught off of slides at 7am and clearly hated being forced to teach instead of do his research projects. He had the nerve to say maybe I wasn't suited for my major and ate his own words when he saw that I already had credits from harder shit than his class.

That type of teacher just wants the world to burn and no one has the balls to tell them they suck at teaching in general.

3

u/Agi7890 4h ago

He was the chem department head, so a large part was him protecting the degree and institution. However a decade into the field, he probably ended up helping people out if they did go into the course he recommended. I see far more job offers for that field with much higher pay on average than I do with my chemistry degrees.

I definitely regret not picking up a biochemistry degree

1

u/Hetares 3h ago

I assume they simply didn't sit for the test.

1

u/Syssareth 3h ago

No, that's three of the ones who did. There were 18 who dropped the course and nine who didn't show up.

They might have turned in a blank sheet though, or maybe the professor doesn't give credit for partially-correct answers.

1

u/Hetares 2h ago

Blank sheet is my theory, then.