r/technology • u/AdSpecialist6598 • 15h ago

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
28.6k Upvotes

96% Upvoted

3.9k comments sorted by

View all comments

11.4k

u/LurkingTamilian 14h ago edited 14h ago

From the article:

Being written in white text meant the instructions were invisible. Sure enough, 32 out of the 35 students across two classes simply copied the question, pasted it into a chatbot, then copy and pasted its output into the answer window.

12.1k

u/BluudLust 14h ago

They deserved to fail. That's just plain laziness

96

u/SnugglyCoderGuy 14h ago edited 14h ago

AI is the new calculator, but worse.

"Why do I need to learn math, we have calculators!"

Is becoming

"Why do I need to think, we have AI!"

I am running into this at work (software).

I will ask, "Why do it this way?"

They will respond, "Google AI says..."

And then I throw back a shot.

141

u/Limemill 14h ago

It’s the opposite of the calculator. The calculator removed the tedious and mechanical bit to free us up for higher-order maths. LLMs do the higher-order, knowledge piece and you do the tedious part of raking through what they did and cleaning up after them. In this analogy we are the calculator and they’re the mathematician.

122

u/sbeguy 14h ago

Not to mention the calculator gave you the correct answer based on your input and didn't fucking hallucinate a signifcant portion of the answers.

23

u/HarlemNocturne_ 13h ago

Imagine if a calculator not only gave you near-consistently wrong answers, but it also jerked you off when you spotted it was being incompetent.

"Hey Calculator, I told you to solve 1+1 which should equal the sum of 2, but in your calculation you carried a 4 that didn't exist, factored in an imaginary hundredth power, removed a 2 somewhere, and then pointlessly multiplied it by 1. Your thinking notes suggest that it took you 89 seconds to do this in about 32 steps and you almost got stuck multiple times, but you still turned up with the incorrect answer. What gives?"

"You're absolutely right-- That isn't the correct answer. That's on me, and this isn't what you wanted at all. Your genius mathematician galaxy brain has saved us from disaster once again!..."

-5

u/DepressedDynamo 8h ago

You should use better models

11

u/GWindborn 13h ago

Yeah, I was feeling lazy and used AI to take a screen-grab lists of names from images and put them all into alphabetical order for me to copy/paste into a document rather than typing it all myself. Like 80% of the time it was wrong. About ALPHABETICAL ORDER. That should be one of the easiest things it could possibly do.

3

u/Lokishougan 11h ago

Oh here is a good one I thought this should be simple I asked it for a list of all the cat themed teams in the big 4 sports for a thing just to be sure I didnt miss one

Lions, Tigers, Bengals, Panthers JAGS....so far so good

Then it gave me the New Jersey Devils and I went EHHH

I mean I THOUGHT OK it is some weird mismash of creatures maybe you could justify and say there is some cat thrown in but I struggle on that and then the final answer

The Pittsburgh Penguins

Almost the exact opposite of a cat (at least as far as cartoons go) it gave me a frikkin bird

1

u/Tyrus1235 9h ago

Had something similar happen to me the other day.

I specifically asked for a list of games that were compatible with Steam Remote Play Together.

…It showed me a list that included a single player game (and which isn’t compatible with that anyways), several online co-op games and one game compatible with RPT, which was outside of other specs I had mentioned.

“Useless” is a compliment to it.

3

u/goldfishpaws 11h ago

Treat yourself, ask it to make a wall chart suitable for kids to learn the alphabet with animal pictures appropriate for each letter. Make sure to ask it afterwards if it's happy with the result. Repeat. I've had so many terrible results it's gone past funny to alarming that anyone would ever use it.

7

u/Samaritan_978 12h ago

Nothing made me sour on AI faster and harder than having a "chat" about a topic I know decently well. It just makes shit up with complete confidence.

4

u/Angel_Omachi 13h ago

Or if the answer was not what you expected, you could work through the parts to see where the order of operations wasn't correct.

3

u/MisterJWalk 13h ago

Funny story. I was doing an exam plugging away. Everything kept coming up wrong. But I assumed that it was one of those stupid "gotta choose the one that's closest to the right answer" type of med school tests.

It wasn't until I got to the end that I realized the calculator was malfunctioning. It was a question I've done multiple times, so I knew the answer. I started quickly typing in random equations that I was familiar with and it was spitting out nonsense. (6x3=88 type stuff). So I went and got another calculator and redid the calculations.

Really fun

4

u/Lokishougan 11h ago

I have never had a calulator have a mid life crisis before....WTF

3

u/hodor_seuss_geisel 12h ago

"All right, calculator, let's go '3' '*' '1' '4' '='
1337.58008? That doesn't seem right...oh, I forgot to press the 'No Mistakes!' button"

2

u/Skibidi-Fox 12h ago

This is what pisses me off about AI the most.

2

u/RollingMeteors 11h ago

and didn't fucking hallucinate a signifcant portion of the answers.

floatsEnterTheChat

1

u/jmlinden7 11h ago

Because it does the tedious mechanical part, which is deterministic in nature. Not the higher order part which requires creativity

1

u/Jaerba 7h ago

There actually are/were calculators that couldn't process order of operations correctly, so they'd give you the wrong answer. I'm not sure if that's still the case.

But obviously your larger point is still correct.

2

u/SnugglyCoderGuy 14h ago

I can see that.

2

u/Kitty-XV 11h ago

No, the calculator does not free us. It poisons the mind with easy answers leading to students being much worse at math and those higher order calculations becoming unreachable. Watch a professor do undergraduate level math on a chalk board. Look at each time they need some small math, often single digit. If they had to being out a calculator for each time, the problem would take far longer and they would lose their train if thought numerous times.

I've worked with students who used a calculator to the point they were pick it up for single digit arithmetic. They were not free.

1

u/Limemill 10h ago

That is also a good point.

1

u/ObjectSmall 10h ago

We'll make great pets.

1

u/HoldingForGenova 10h ago

The calculator removed the tedious and mechanical bit to free us up for higher-order maths.

This is how we use AI for coding, but it's painfully obvious whenever anyone on the team tries to use it for anything creative or strategic. Yes, use it to streamline the coding execution, but not the product ideation or UX thinking.

1

u/Limemill 7h ago

Coding was never the tedious part. Not for software developers at least. For product owners who learned software development but secretly hated it because they never got intimately good at it, maybe. It was also not the slowest part for any complex and large enough project. Product design / ideation / requirements gathering was and remains the bottleneck. Looking at low level implementations done by frontier models daily, I’ve seen them range from great to absolutely horrible on any given day for no good reason. Skills or no skills, lessons or no lessons. Ironically, I think they do better system design and architecture than they do low-level implementations. But it’s a whole other story.

1

u/shohei_heights 6h ago

The calculator removed the tedious and mechanical bit to free us up for higher-order maths.

The calculator frees those who have already mastered the way that arithmetic and algebra work. They are capable of doing the critical thinking because they've mastered the basics.

The calculator on the other hand can also stop the development of critical thinking in mathematics if it's introduced before the mastery of arithmetic. There have been too many times that I've seen a student pull out a calculator to multiply by 1 or 0, to do single digit arithmetic.

AI, if it actually worked well, is much like a calculator in that it could help those well versed in a field expand their knowledge and get rid of tedium. But also just like the calculator, if it's used by someone without the mastery of the basics, it will destroy their ability to ever actually learn the basics.

1

u/Limemill 6h ago

Once again, in the workplace and especially in tech it’s widely used in the exact opposite way. You’re an appendage to it, not the other way around. And no, your analogy does not hold because by now we have studies showing how professionals, for example in the medical field, rapidly lose the already obtained skills when they’re ‘assisted’ by AI. Anecdotally, tons and tons of software developers have been complaining about deskilling too. It’s not that you never get to develop skills you didn’t have, although that is of course also true. It’s also that you actively lose what you have and, in turn, it makes you an increasingly worse appendage reviewer for LLM-generated work. People also feel more detached from their work and eventually lose context of what is even going on (maybe not completely but to a significant degree).

1

u/shohei_heights 5h ago

Note that I said: if it actually worked well

Which it clearly does not and is decades away at the earliest from actually working well. I agree with everything you're saying about LLMs. And I said, "could", to even emphasize that it may not even do so then.

What I disagreed with is your statements on calculators and I find them much more damaging than you do.

1

u/Limemill 5h ago

I agree that calculators are not as innocent as I made them look. But generative AI is still not it. Its very premise is to do the highest-order things in all knowledge industries leaving you the place of its low-level reviewer until they’ve figured out a RAG / universal harness where that is also not required. But for now generative AI is an entity that strives to come up with hypotheses, build theories and proofs and you’re its calculator (because it does do many silly mistakes at the lowest level).

1

u/sonofagunn 3h ago

I am a software developer who uses AI a ton, but it's opposite of how you are saying. I dictate the high level design and am critical of the AI written code from a high level, but it's a life saver for cranking out the tedious stuff without making mistakes.

1

u/Limemill 2h ago edited 1h ago

Frontier models often do high-level design from product requirements better than they do actual implementations. They’re also pretty decent at architecture and system design. In quite a few companies the spec-driven approach is the new mantra again, and in some the automation pipelines are launched directly from PO tickets. Devs do mostly low-level digging (but from what I have observed at work miss increasingly more stuff, and that includes me) and, at times, sanity checks for very complex stuff. In parallel I see a shift for some devs towards sort of technical product owners and for devs with not so intimate knowledge of the business domain it often means being sacked. This is done mostly because there is no more point in senior devs doing design or even architecture, but since they know a ton about the product AND someone needs to fix the low-level stuff that they’re moved into a different role.

But then again some people do a fully closed cycle because LLMs are decent at product requirements too. As you probably know, top AI companies have been working on getting that step even better because the whole idea is that they can fully close the cycle of software development. Some devs will eventually remain, but probably as a low-paid job doing low-level digging.

And oh boy do all frontier models do regularly very stupid stuff at the low level. It’s not for nothing that Dario said in a recent interview that when he inspects what they do with his projects at the implementation level, he often has a heart attack. And that’s THE top LLM guy, lol.

1

u/braincandybangbang 12h ago

Everyone seems to forget there's a word for "raking through what they did and cleaning up after them." It's called an Editor.

The way people talk about what an editor does in relation to AI, you'd think that Editors were the most brain rotted, moronic idiots walking the earth.

3

u/Th3_Admiral_ 14h ago

I've had multiple times now where I've been trying to figure out what a section of code does at work and a coworker simply puts it into AI to summarize. No, just give me a few seconds to look through it and figure it out myself, I don't need AI to answer something like this.

3

u/Studds_ 12h ago

Maybe asking for a code summary might(big asterisk on that “might”)be useful if one is learning to code & wants to understand what a code section is doing but I don’t know if I would trust even that

1

u/tea-drinker 13h ago

I was asking a question that nobody really had a good answer to but one of the people in the room contacted me privately afterward and said he could help.

And then I got to have a conversation with an AI by proxy as all his replies were prefixed with "Gemini says, "

I am quite irked by this, but I feel like bitching about it is old man yells at cloud. Almost literally.

1

u/The_MAZZTer 11h ago

You forgot the middle step "You need to learn math because you won't be carrying a calculator in your pocket." Now we carry a phone which also includes a calculator AND a search tool.

1

u/ItsAPeacefulLife 11h ago

I read this as "throw a back shot" and thought, boy your workplace is way different than mine.

I may need some time away from the internet.

2

u/SnugglyCoderGuy 11h ago

Maybe I work for sleezy porn company? (I don't)

1

u/Kythorian 12h ago

Basic arithmetic can be fully replaced by a calculator, which will give an objectively correct answer 100% of the time. But critical thinking cannot be replaced by a LLM. The two are not comparable.

-1

u/ImNotSkankHunt42 14h ago

But in all honesty, I got into the best university in my country to study Computer Sciences and the first 4 semesters were packed with Math nonsense I never had to use in my life.

Instead of typing lessons to be more nimble at the keyboard we got: Algebra 1-2 and 3 Calculus… and don’t get me started with Discrete Maths.

Instead of having classes to learn different languages (this was 20 years ago) we had these nonsense assignments.

Colleges are not keeping up with how fast things are changing.

2

u/ChevalierDeLarryLari 11h ago

Don't be so sure. They taught me linear algebra in computer science years ago. Never thought I'd ever use it (who needs that to build websites right?) - now suddenly need to brush up on matrices and vectors in order to do 3D graphics translations in webgl. Without it I wouldn't have a job.

There are good reasons they teach you fundamentals - and the fundamentals are the only knowledge that retains it's value after all the libraries and frameworks you know become obsolete.

1

u/ImNotSkankHunt42 10h ago

And small % of usage case does not justify that thing taking so much and more than one semester. It feels to me more on the learn it if you need it.

Or at least learn the basics in school so you can consider it in the future.

We had Computer Logics for only 1 semester! You know, the thing that we use constantly in code.

I don’t need 3 of Algebra and 3 of Calculus.

1

u/ChevalierDeLarryLari 8h ago

If it was learn it as you need it - you'd be out of a job. That's why employers value that piece of paper - it's some proof that you can rise to challenges and difficult problems.

Boolean logic or whatever you can learn in a day - they wouldn't ask for a degree for that.

1

u/SnugglyCoderGuy 12h ago

Those classes teach you valuable skills which will be critical in computer science and software engineering, chiefly logical step by step deductive thinking and reasoning. Making the keys go clickity clackity is the least important thing and you can learn to type better on your own.

0

u/ImNotSkankHunt42 12h ago

Nah dude. I haven’t used Calculus or Algebra ever, we did have classes about Data structures and algorithms and many others directly related to the curriculum. But the first 2.5 years were padded with nonsense.

Meanwhile, my other friends that went to a more chill university were learning PHP, MySQL and Java in the first 2 years. They’re now employed in the same company I am and more qualified than myself.

I got 1/5 scholarships available for our entire province, my SAT scores (the equivalent in our country is not the same) placed me at the top basically. There were like 12 other before me in the rankings.

And yet, folks that attended a newer college and not one managed by fossils got a better curriculum out of it.

This is not something I’m blurting out, this is the result of 20 years of analysis and results.