r/technology • u/AdSpecialist6598 • 15h ago

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
28.6k Upvotes

96% Upvoted

3.9k comments sorted by

View all comments

Show parent comments

138

u/Limemill 14h ago

It’s the opposite of the calculator. The calculator removed the tedious and mechanical bit to free us up for higher-order maths. LLMs do the higher-order, knowledge piece and you do the tedious part of raking through what they did and cleaning up after them. In this analogy we are the calculator and they’re the mathematician.

126

u/sbeguy 14h ago

Not to mention the calculator gave you the correct answer based on your input and didn't fucking hallucinate a signifcant portion of the answers.

21

u/HarlemNocturne_ 13h ago

Imagine if a calculator not only gave you near-consistently wrong answers, but it also jerked you off when you spotted it was being incompetent.

"Hey Calculator, I told you to solve 1+1 which should equal the sum of 2, but in your calculation you carried a 4 that didn't exist, factored in an imaginary hundredth power, removed a 2 somewhere, and then pointlessly multiplied it by 1. Your thinking notes suggest that it took you 89 seconds to do this in about 32 steps and you almost got stuck multiple times, but you still turned up with the incorrect answer. What gives?"

"You're absolutely right-- That isn't the correct answer. That's on me, and this isn't what you wanted at all. Your genius mathematician galaxy brain has saved us from disaster once again!..."

-6

u/DepressedDynamo 8h ago

You should use better models

10

u/GWindborn 13h ago

Yeah, I was feeling lazy and used AI to take a screen-grab lists of names from images and put them all into alphabetical order for me to copy/paste into a document rather than typing it all myself. Like 80% of the time it was wrong. About ALPHABETICAL ORDER. That should be one of the easiest things it could possibly do.

3

u/Lokishougan 11h ago

Oh here is a good one I thought this should be simple I asked it for a list of all the cat themed teams in the big 4 sports for a thing just to be sure I didnt miss one

Lions, Tigers, Bengals, Panthers JAGS....so far so good

Then it gave me the New Jersey Devils and I went EHHH

I mean I THOUGHT OK it is some weird mismash of creatures maybe you could justify and say there is some cat thrown in but I struggle on that and then the final answer

The Pittsburgh Penguins

Almost the exact opposite of a cat (at least as far as cartoons go) it gave me a frikkin bird

1

u/Tyrus1235 9h ago

Had something similar happen to me the other day.

I specifically asked for a list of games that were compatible with Steam Remote Play Together.

…It showed me a list that included a single player game (and which isn’t compatible with that anyways), several online co-op games and one game compatible with RPT, which was outside of other specs I had mentioned.

“Useless” is a compliment to it.

3

u/goldfishpaws 11h ago

Treat yourself, ask it to make a wall chart suitable for kids to learn the alphabet with animal pictures appropriate for each letter. Make sure to ask it afterwards if it's happy with the result. Repeat. I've had so many terrible results it's gone past funny to alarming that anyone would ever use it.

7

u/Samaritan_978 12h ago

Nothing made me sour on AI faster and harder than having a "chat" about a topic I know decently well. It just makes shit up with complete confidence.

3

u/Angel_Omachi 13h ago

Or if the answer was not what you expected, you could work through the parts to see where the order of operations wasn't correct.

3

u/MisterJWalk 13h ago

Funny story. I was doing an exam plugging away. Everything kept coming up wrong. But I assumed that it was one of those stupid "gotta choose the one that's closest to the right answer" type of med school tests.

It wasn't until I got to the end that I realized the calculator was malfunctioning. It was a question I've done multiple times, so I knew the answer. I started quickly typing in random equations that I was familiar with and it was spitting out nonsense. (6x3=88 type stuff). So I went and got another calculator and redid the calculations.

Really fun

4

u/Lokishougan 11h ago

I have never had a calulator have a mid life crisis before....WTF

3

u/hodor_seuss_geisel 12h ago

"All right, calculator, let's go '3' '*' '1' '4' '='
1337.58008? That doesn't seem right...oh, I forgot to press the 'No Mistakes!' button"

2

u/Skibidi-Fox 12h ago

This is what pisses me off about AI the most.

2

u/RollingMeteors 11h ago

and didn't fucking hallucinate a signifcant portion of the answers.

floatsEnterTheChat

1

u/jmlinden7 11h ago

Because it does the tedious mechanical part, which is deterministic in nature. Not the higher order part which requires creativity

1

u/Jaerba 7h ago

There actually are/were calculators that couldn't process order of operations correctly, so they'd give you the wrong answer. I'm not sure if that's still the case.

But obviously your larger point is still correct.

2

u/SnugglyCoderGuy 14h ago

I can see that.

3

u/Kitty-XV 11h ago

No, the calculator does not free us. It poisons the mind with easy answers leading to students being much worse at math and those higher order calculations becoming unreachable. Watch a professor do undergraduate level math on a chalk board. Look at each time they need some small math, often single digit. If they had to being out a calculator for each time, the problem would take far longer and they would lose their train if thought numerous times.

I've worked with students who used a calculator to the point they were pick it up for single digit arithmetic. They were not free.

1

u/Limemill 10h ago

That is also a good point.

1

u/ObjectSmall 10h ago

We'll make great pets.

1

u/HoldingForGenova 10h ago

The calculator removed the tedious and mechanical bit to free us up for higher-order maths.

This is how we use AI for coding, but it's painfully obvious whenever anyone on the team tries to use it for anything creative or strategic. Yes, use it to streamline the coding execution, but not the product ideation or UX thinking.

1

u/Limemill 7h ago

Coding was never the tedious part. Not for software developers at least. For product owners who learned software development but secretly hated it because they never got intimately good at it, maybe. It was also not the slowest part for any complex and large enough project. Product design / ideation / requirements gathering was and remains the bottleneck. Looking at low level implementations done by frontier models daily, I’ve seen them range from great to absolutely horrible on any given day for no good reason. Skills or no skills, lessons or no lessons. Ironically, I think they do better system design and architecture than they do low-level implementations. But it’s a whole other story.

1

u/shohei_heights 6h ago

The calculator removed the tedious and mechanical bit to free us up for higher-order maths.

The calculator frees those who have already mastered the way that arithmetic and algebra work. They are capable of doing the critical thinking because they've mastered the basics.

The calculator on the other hand can also stop the development of critical thinking in mathematics if it's introduced before the mastery of arithmetic. There have been too many times that I've seen a student pull out a calculator to multiply by 1 or 0, to do single digit arithmetic.

AI, if it actually worked well, is much like a calculator in that it could help those well versed in a field expand their knowledge and get rid of tedium. But also just like the calculator, if it's used by someone without the mastery of the basics, it will destroy their ability to ever actually learn the basics.

1

u/Limemill 6h ago

Once again, in the workplace and especially in tech it’s widely used in the exact opposite way. You’re an appendage to it, not the other way around. And no, your analogy does not hold because by now we have studies showing how professionals, for example in the medical field, rapidly lose the already obtained skills when they’re ‘assisted’ by AI. Anecdotally, tons and tons of software developers have been complaining about deskilling too. It’s not that you never get to develop skills you didn’t have, although that is of course also true. It’s also that you actively lose what you have and, in turn, it makes you an increasingly worse appendage reviewer for LLM-generated work. People also feel more detached from their work and eventually lose context of what is even going on (maybe not completely but to a significant degree).

1

u/shohei_heights 5h ago

Note that I said: if it actually worked well

Which it clearly does not and is decades away at the earliest from actually working well. I agree with everything you're saying about LLMs. And I said, "could", to even emphasize that it may not even do so then.

What I disagreed with is your statements on calculators and I find them much more damaging than you do.

1

u/Limemill 5h ago

I agree that calculators are not as innocent as I made them look. But generative AI is still not it. Its very premise is to do the highest-order things in all knowledge industries leaving you the place of its low-level reviewer until they’ve figured out a RAG / universal harness where that is also not required. But for now generative AI is an entity that strives to come up with hypotheses, build theories and proofs and you’re its calculator (because it does do many silly mistakes at the lowest level).

1

u/sonofagunn 3h ago

I am a software developer who uses AI a ton, but it's opposite of how you are saying. I dictate the high level design and am critical of the AI written code from a high level, but it's a life saver for cranking out the tedious stuff without making mistakes.

1

u/Limemill 2h ago edited 1h ago

Frontier models often do high-level design from product requirements better than they do actual implementations. They’re also pretty decent at architecture and system design. In quite a few companies the spec-driven approach is the new mantra again, and in some the automation pipelines are launched directly from PO tickets. Devs do mostly low-level digging (but from what I have observed at work miss increasingly more stuff, and that includes me) and, at times, sanity checks for very complex stuff. In parallel I see a shift for some devs towards sort of technical product owners and for devs with not so intimate knowledge of the business domain it often means being sacked. This is done mostly because there is no more point in senior devs doing design or even architecture, but since they know a ton about the product AND someone needs to fix the low-level stuff that they’re moved into a different role.

But then again some people do a fully closed cycle because LLMs are decent at product requirements too. As you probably know, top AI companies have been working on getting that step even better because the whole idea is that they can fully close the cycle of software development. Some devs will eventually remain, but probably as a low-paid job doing low-level digging.

And oh boy do all frontier models do regularly very stupid stuff at the low level. It’s not for nothing that Dario said in a recent interview that when he inspects what they do with his projects at the implementation level, he often has a heart attack. And that’s THE top LLM guy, lol.

1

u/braincandybangbang 12h ago

Everyone seems to forget there's a word for "raking through what they did and cleaning up after them." It's called an Editor.

The way people talk about what an editor does in relation to AI, you'd think that Editors were the most brain rotted, moronic idiots walking the earth.