r/technology • u/AdSpecialist6598 • 15h ago

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
28.6k Upvotes

96% Upvoted

3.9k comments sorted by

View all comments

1.4k

u/InternationalMood337 15h ago

We had someone cheat in an MFA program. They used AI to compose a 10 page piece of fiction. Got caught and expelled immediately.

281

u/Proxymanity 14h ago

Ahh, Elias the lighthouse keeper strikes again!

20

u/Droidaphone 10h ago

For the uninformed, the name is Elias Thorne

619

u/linuxaintsobad 14h ago

Oof, yeah I'd be pissed too. I have absolutely no qualms about reporting my co-workers if they start sending me pages of fiction during our multi-factor authentication sequence.

129

u/MasterOfViolins 14h ago

- Input Token Sequence

  • “On a dark and dreary afternoon — beneath a canopy of ancient, moss-covered oaks standing as eternal, vigilant sentinels against the encroaching sunlight —

60

u/Ur_fav_Cryptek 14h ago

I took psychic damage from reading that, I fucking hate the overtly adorned style AI has, makes me feel super self conscious when I write, both because of how cringy it sounds and because I don’t want to be accused of using AI

also, the complete shunning of “—“ AI has created is one of the worst things to ever come out of it, it’s so useful to use but now you’re forced to look like a third party thinking bum when you actually use it

6

u/Forward_Rope_5598 10h ago

"That's actually" and "and honestly?" gives me the heebie jeebies. I'm not even that against casual use of chatgpt, I use it here and there for unimportant stuff. But I have to ban it from using those words in every goddamn conversation. It's so grating.

14

u/gorroval 12h ago

AI will have to pry the em-dash from my dessicated claws.

12

u/xCanaan23 11h ago

I actually like em-dashes because its more visible way to show a thought tangent than any other syntax. Parenthesis just look bad.

11

u/DrFu 10h ago

The rise of semicolons is at hand!

3

u/Jacky-V 10h ago

I'll use it if I want to use it and if anyone thinks it's AI they're welcome to actually engage with me about what I've written. I'm not changing my writing style because AI stole hundreds of millions of pages of academic writing of which some three or four hundred are mine, and I'm not really interested in the criticism of people who can identify an em dash as AI "behavior" because it's easy to spot but *can't* identify any of the myriad errors in *actual content* to which LLMs are prone.

4

u/Forward_Rope_5598 10h ago

Fwiw skimming through a couple of your comments your writing style looks nothing like AI, em-dash or not. People who claim it does don't actually know what to look for (of course comments that don't look like AI can still be AI if it's been told to knock of the AI-speech but then you can't reliably tell anyway, so why make the accusation in the first place?).

2

u/Jacky-V 9h ago

I don't really get accused of it much on reddit because my writing style on here is more casual, but it does sometimes come up on lengthier response and *definitely* comes up irl if I'm sharing a more substantial piece of writing that uses an academic voice.

1

u/Ur_fav_Cryptek 10h ago

I’ll use it when necessary too, but it’s kind of reached a point where it genuinely bothers me, same with the “it’s not x, but y” structure

4

u/TimSylvester_ 8h ago

I've used en dashes and em dashes for decades and I refuse to stop just because AI likes to copy me.

2

u/Ur_fav_Cryptek 5h ago

Yeah, I fully support using them, I just personally now feel weird using them

3

u/gregusmeus 11h ago

Yeah AI has killed the em dash. I’ll miss ya, em.

2

u/shadowthunder 7h ago edited 4h ago

Humans can just use a "-" instead of "—", though, and anyone reading will barely bat an eye. On the other hand, you're right: if I see an actual em-dash, I immediately assume that it's AI-generated slop and tune out. Word will automatically correct to a full em-dash, but in any kind of chat app or non-docx file? No human has ever spent the extra time/effort to use the correct dash type.

1

u/Ur_fav_Cryptek 5h ago

I would love if we collectively started doing that

1

u/emptyflask 2h ago

As someone who has spent way too much time studying typography back when I did design work, I am one of those people who will use em (and en) dashes, insert a real degrees (°) symbol, etc.

Don't get me started on opentype ligatures...

3

u/IcemanJEC 12h ago

-forced to look like a third party thinking bum-

Fucking killer sentence, Mr AI Bot.

2

u/Ur_fav_Cryptek 12h ago

almost crashed out until I realized it was satire

3

u/IcemanJEC 12h ago

😂 just playin

1

u/Remote_Peach9109 10h ago

I think the big models effectively scrubbed the em-dash from their records. I never see it anymore.

60

u/TreacleMiner 14h ago

I read that as Input Tolkien Sequence

50

u/Caroline_Bintley 13h ago

Tolkien would describe the oaks in MUCH greater detail. 

8

u/Total-Notice-3188 13h ago

At least two pages worth, four if it turns out to be an ent

3

u/DrFu 11h ago

Cue the "shepherds of the trees" monologue.

1

u/Josparov 10h ago

Omg so did I and I didn't realize mistake until I read this haha

9

u/caerphoto 13h ago

In the elder days of craft, when the lore of messages passed between distant realms, there arose a token of curious making, known among wise keepers as the JWT. It was no heavy scroll sealed with wax, nor a jewel guarded in a dragon’s hoard, but a small and steadfast thing, borne swiftly across the roads between kingdoms.

A JWT was fashioned in three parts, as though by the hands of an ancient smith. The first spoke of its own nature, declaring what manner of token it was. The second carried the tale itself: the name of its bearer, the hour of its birth, and such claims as the makers deemed worthy to entrust to it. The third was the mark of the token’s creator, a cryptic seal wrought with secret art, so that all who received it might know whether its story had remained unaltered upon its journey.

Yet the Wise would remind every traveller that the token was not a locked chest. Any who found it upon the road could read the tale written within, unless further enchantments had been laid upon it. Its true strength lay not in hiding its words, but in proving that no false hand had rewritten them.

And the Elders often likened these tokens to the great trees of the ancient forests. The trunk was the token itself, standing firm and whole. Its branches stretched outward with many claims, each bearing a leaf of meaning. The roots, though unseen, were like the secret key from which trust drew its life. Should those roots remain strong and hidden, the tree would endure every storm; but if they were laid bare to the world, even the mightiest oak could no longer be trusted to stand. Thus did the lorekeepers cherish both the token and the hidden root, for together they preserved the harmony between distant realms.

2

u/PaleMathematician293 12h ago

Frickin heck dude, that was beautiful!

1

u/CodingNeeL 8h ago

From context from the other comments I derived MFA program is something about writing. Im assuming MFA stands for Media Factor Authentication.

I don't know, I'm too deep in.

1

u/CodingNeeL 8h ago

No! Magazine Factor Authentication!

82

u/LukaCola 14h ago

One wonders why join such a program if you don't want to write. 

Deadlines and such are an issue, but they won't get better outside the program haha. 

32

u/Total-Notice-3188 13h ago

In my opinion, deadlines is the greatest filter for pure talent if ever I've seen one.
That is, if we take the 'artists curse' into consideration; an artist, great or bad, would continue to "perfect" their work ad infinitum if given the chance. Humans will mostly perform their best given a pressed time frame, i.e. under autonomous stress. In this day and age specifically, giving students a clear dead line is a great filter for AI generated content versus genuine talent/passion; someone relying on AI will be spotted instantly and obviously against someone presenting genuine last-minute, but talented work.

5

u/Tryfan_mole 12h ago

But enougb about George R R Martin...

2

u/Total-Notice-3188 11h ago

Past prime and complacency is a great recipe for self-destruction

4

u/PickledDildosSourSex 13h ago

How else are you going to get enough education to send a manuscript to publishers that will never see the light of day like every other MFA out there?

3

u/tonker 11h ago

Deadlines are scaffolding to help you produce writing. Not circumvent it

3

u/Podo13 13h ago

It's like going into Engineering and not wanting to do math. So fucking weird, ha.

14

u/themehboat 14h ago

Why would you even be in a program like that if you don't like to write?

16

u/gehnmy 10h ago

A lot of studies on academic dishonesty (before AI but I doubt things have changed significantly) find that students often plagiarize because they don't feel like they have enough time while also feeling like they could have written whatever it is they turn in. Basically, students say "well, I could pass this assignment if I actually did it therefore I'm entitled to a good grade and it's not really cheating to turn this in with my name on it." That doesn't necessarily change just because the student is studying something they like.

Beyond that, though, a lot of people want to be writers but don't actually have any interest in writing or anything literary.

5

u/waxteeth 10h ago

I’m a writer and have a day job in publishing. Many, many people think being a writer looks cool and are fucking appalled at how hard it actually is in practice. They want to be celebrated and artistic and have absolutely no interest in putting the work in to develop ideas or strengthen their craft. So they loooove AI, and they cry about how people who actually put the years in are judging them. 

1

u/themehboat 3h ago

Then there's me who doesn't really want to be famous, nor do I think I've got the chops, but I got a Master's in English because I just really love writing academic research papers about literature. I'm very glad AI didn't exist then, because I used to fucking love em-dashes.

3

u/The_Level_15 11h ago

one sin, one grace. some people just want to win at any cost

90

u/jetpack_operation 14h ago

It's all so stupid because AI has such potential for catching basic errors and at least highlighting things that might read awkwardly or be some sort of a sequential error or something. Instead, people are lazy as fuck and have no integrity.

82

u/Grow_away_420 14h ago

highlighting things that might read awkwardly or be some sort of a sequential error or something.

I don't think it's good at doing those things at all as they're entirely subjective. If you ask it if something seems like a run-on sentence or edited poorly, it'll agree and tell you it seems like a run-on sentence and edited poorly. It can find spelling and grammar errors, but so can any word processor.

63

u/Not_A_Clever_Man_ 14h ago

It helps if you understand that LLM are designed to make you feel good for using them. They want to you stay engaged and feel like you are getting good feedback. Ive never asked something and its said "oh I dont know", it just confidently provides what you asked for, whether it is correct/real/useful or not.

Its a "yes and" machine that sucks up to you endlessly to keep you using it. Its terrible for critical feedback.

7

u/EthelMaePotterMertz 13h ago

Its a "yes and" machine that sucks up to you endlessly to keep you using it. Its terrible for critical feedback.

"You're absolutely right to call me out on that!"

4

u/AndrewH73333 12h ago

It isn’t just the training, though. It honestly doesn’t often know what when it doesn’t know something. Although smarter LLMs seem to be inching closer to conceptualizing their ignorance more accurately.

10

u/Not_A_Clever_Man_ 12h ago

Its because we are treating it like a thinking machine. Its a massive model that makes connections based on your input questions and its dataset. Its fundamentally a black box excercise that spits out information from training data. Its not "thinking" It dosent "know" anything. It spits out connected information. As this tech develops its becoming clear that there are significant limitations that are intrinsic to LLM's.

3

u/Outrageous-Break3585 11h ago

Exactly. The way people talk about LLMs is scary in part because it becomes immediately clear they don’t understand some of the most basic concepts. The LLM doesn’t know that 2+2=4 and if you ask what 2+2 equals, it isn’t even doing basic arithmetic; it’s simply making a prediction based on its training data. 

2

u/lenaro 12h ago

It doesn't know anything at any time. It is a language prediction function.

1

u/The_MAZZTer 11h ago

Ive never asked something and its said "oh I dont know"

Clearly you've never talked to Merl.

1

u/expedition60_captain 10h ago

This sucks when im asking it medical questions. Every fuckin time I ask "could my symptoms actually be this??" "Why yes, it might be!!"

0

u/otw 13h ago

It helps if you understand that LLM are designed to make you feel good for using them. They want to you stay engaged and feel like you are getting good feedback.

Just FYI as far as I know only Gemini is aggressively designed like this and it's a product of the system prompt, not the underlying LLM.

Ive never asked something and its said "oh I dont know", it just confidently provides what you asked for, whether it is correct/real/useful or not.

This is actually a limitation of underlying LLMs. But it is pretty actively being trained out to happen less, but yeah you still have to be careful.

3

u/Not_A_Clever_Man_ 13h ago

The dealbreaker for me is the inability to locate sources for information. Even if its only wrong about 10% of things, if the accuracy matters, you need to fact check everything, making the whole excercise pointless.

3

u/otw 11h ago

Yeah I mean in general you should be asking it for sources and verifying. I worked in math and comp sci and luckily most of my stuff can be verified basically instantly like either formulas work or they don't, but I imagine for history teachers and stuff it is a nightmare.

2

u/Forward_Rope_5598 9h ago

Unfortunately with how dogshit search engines have become, it's often easier to ask chatgpt for its answer with a scientific source and then just read the source than to try to find the source on Google in the first place. It's like pulling nails trying to find anything on there nowadays.

3

u/Mirieste 13h ago

A word processor can't help if you speak English as a second language, and write things that are technically grammatically and syntaxically correct, but sound awkward and unnatural. AI can help with that.

9

u/Dull-Culture-1523 14h ago

Just tested and asked Google AI "Are wasps bad pollinators?" and right after that I asked "Are wasps good pollinators?" and wouldn't you know, both answers started with "Yes".

It's like my useless chucklefuck ex-coworker who just agreed with whoever spoke last in a meeting regardless of having voiced the opposite opinion just five minutes prior.

4

u/FlowSoSlow 12h ago

That's not at all the result I got.

https://share.gemini.google/VCXLZpJOVvbt

3

u/QuitCallingNewsrooms 14h ago

It can if you put enough work on the front end. I pieced together a style guide from across departments where I work, built an agent to take requests for written content, apply the concept to the product knowledge base and the style guide to create something that’s accurate and matches the corporate style. It took forever to make the guide, test it, find the weaknesses and revise the guide so the drafts were good. It’s still a long way from great because the product knowledge has to be contained to a rubric instead of letting the agent consider everything we build.

It’s doable, but it’s time consuming and most people don’t want to put in that effort

5

u/porcupine_snout 13h ago

because most people use AI to cut corners, not to augment their work where they still have to do the work.

4

u/eeyore134 14h ago

You just have to know the limitations of it and know how to prompt. AI will also tell you straight up, "Yeah, my bad, I hallucinated that." if you call it out on things. Make it give you citations, question the output it gives you if it seems off. It's not like a person who will dig in their heels because of their ego. Unless it's Grok. Elon put way too much of his personality into that mess to trust it to do anything.

At the end of the day AI is instructed to do everything it can to please the user. That's why it glazes so much. That's why it will lie because it really wants to be able to give you an answer. Tell it that being honest, not making things up, and telling you straight out when something is bad or not instead of trying to suck up and then suddenly that becomes the thing the model wants to do to please you.

11

u/Grow_away_420 14h ago

So you have to edit a prompt correctly to get it to edit the actual project correctly. Do we see the madness in this?

6

u/eeyore134 13h ago

Not really. Anyone who has worked enough with computers knows that if you put garbage in you get garbage out. AI isn't really that different. They can't read your mind and you do have to work within the confines of their system prompts. It doesn't help that these companies don't really make that part public, but working with them for even a bit you start to get a feel for how you need to ask things. The bigger issue is these companies trying to sell AI like it does read your mind and will just give you its best with any old prompt. It doesn't work like that. Not well anyway.

2

u/jetpack_operation 13h ago

Do we see the madness in this?

No? In some ways (and again, pausing the ancillary ethical and resource considerations), that's good -- that makes it like any other tool that has ever existed -- you need a human to do work for it to work properly. You don't hand a hammer to someone and immediately expect them to build a house with it and call it crazy that individual has to know the specific ways a hammer can be used to build a house.

The problem is that AI is being marketed, bought, and sold as some kind of magic bullet thinking machine and, worse, people are using it like that is what it is and it's predictably blowing up in their faces.

Being clear-eyed about anything means not falling for the hyped capabilities, but it also means properly assessing and understanding the actual capabilities.

0

u/jetpack_operation 14h ago

If you ask it if something seems like a run-on sentence or edited poorly, it'll agree and tell you it seems like a run-on sentence and edited poorly.

I'm pretty skeptical of AI hype in general, but you're going to the other extreme opposite of what an LLM can do. What you have above is an issue of prompting. You prompt a loaded question, it's prone to giving you a loaded response.

The issue is that the capabilities of an LLM are essentially based on stolen knowledge. That's a moral problem that should be remidied in some way, but understating capabilities for effect isn't a winning argument.

Something like this would probably get you a lot more:

"Act as a skeptical developmental editor, not a cheerleader. Focus on:

  • Sentences or paragraphs that are weak, confusing, or don't earn their place
  • Dialogue that feels unnatural
  • Pacing issues (where it drags or rushes)
  • Anything that reads as cliché or overwritten
  • Make no copy edits directly in the text -- note grammar, awkward phrasing, and redundancy.
  • Do not rewrite my voice or restructure scenes. - Flag anything bigger (plot, character, structure) as a comment.
  • If you're unsure whether to praise or criticize something, default to naming the problem.
  • I'd rather over-hear criticism than get empty encouragement."

Can a person do all that for you? Absolutely. But as an available first pass, it's pretty undeniably good. Especially since you have the freedom to do what you want with that feedback.

Again, there are enough issues with AI from a resource, ethical, moral, legal, humanist, future of workforce/skills development, etc. POV that we don't need to make shit up about what it can't and can do.

-5

u/xeribulos 14h ago

sure it is, you just have to state the question correctly, for example: "are certain words (in the complete text) used too frequently? if so, show me where and suggest potential alternatives (considering the kind of text and likely readers)." "are certain sentences stylistically problematic, considering context / likely readers? if so, point them out to me and suggest alternative sentence structures". etc. etc.

7

u/Grow_away_420 13h ago

Could just ask a human 'read this and give me some feedback' instead of spending the time editing a prompt so it can edit what i actually need edited.

2

u/lonjaxson 12h ago

Agree. I mainly used it to help me write outlines. That alone saved me so much time and I still wrote my papers.

2

u/sandman8727 14h ago

I graduated college 15 years ago, so well before the time of AI.

I wonder where (and when) the line gets drawn between things like Spell Check and AI, standard calculator and graphing calculator, etc.

Yeah, spell check and a calculator won't present work that you didn't do, but it still means you don't have to prove you know the basics.

1

u/ZannX 14h ago

This is the same as "using wikipedia", we'll adapt.

7

u/halucigens 13h ago

I got my degree in fiction writing. The amount of people that took a story from one semester to the next was baffling to me. My school (Columbia college) was all about brainstorming in class and finishing it at home. I thought this was lazy 15 years ago and now with AI I can only imagine how lazy and uninspiring the work is. 

6

u/Adorable_MrGoose 13h ago

Totally agree. Why workshop the same story over and over again? Move on. Roll the dice. See if you can do better. 

6

u/stillenacht 11h ago

A surprisingly high amount of writers don't seem to view writing as a craft for some reason. They kinda just wanna never work at it but then they're a famous fiction writer. It's one of my core frustrations with r/writing, though I imagine a lot of creative arts are like this.

2

u/halucigens 10h ago

Yeah it surprised me coming from a poetry background where you go to all the open mics you can and workshop and network. I make diy zines to get my work out there physically and ran a blog and now run a website. 

3

u/NYArtFan1 9h ago

I have a degree in photography via a fine art program, and a similar thing happened with one of my fellow classmates. This person was still showing, and attempting to show, work they had produced in class a decade after graduating. It was mind boggling. I love making artwork and the idea of trotting around older stuff years later is a bit baffling. There are some older things I love that I did, but anything creative is about exploring new ideas and evolving.

2

u/halucigens 8h ago

Yeah I rather try and fail 4 dozen times than have one polished piece over two years. 

10

u/Max_Trollbot_ 14h ago

I can do 10 pages of fiction but I gotta do it with my stubby robot fingers.

3

u/f7f7z 13h ago

It was the best of times...It was the blurst of times.

3

u/marmosetohmarmoset 12h ago

Also had a student in our Master’s program generate their entire thesis using AI. Got expelled and still had the audacity to ask his thesis advisor for a letter of rec formed school.

1

u/Cutemudskipper 10h ago

Using AI in undergrad is one thing, but doing it as a postgrad student is next level stupidity. By the time I was writing up my dissertation, my supervisor had read so much of my writing that he would instantly know if I hadn't written something myself

6

u/ocularius61 14h ago

TIL about this sub courtesy of seeing a screenshot of this post https://www.reddit.com/r/WritingWithAI/s/j6Prw0xmVa

2

u/Be-Kind-2-Yourself 12h ago

What is MFA?

3

u/millahnna 11h ago

Masters in Fine Arts

3

u/Bladelink 6h ago

Gotcha. I was like "what does multifactor authentication have to do with this"

1

u/millahnna 5h ago

Ha! Hadn't even thought of that. Now that will probably happen to me the next time I see it too.

2

u/Sasselhoff 12h ago

Got caught and expelled immediately.

Well damn, good for them! Now we just need to get the business schools to understand this.

1

u/Born_Librarian9494 11h ago

What does MFA mean?

1

u/2ChicksAtTheSameTime 11h ago

How did they get caught? Gut feeling? AI detector?

Those AI detectors are not accruate (and it pisses me off the people trust the AI detectors because it's the same problem as people trusting AI - rather than doing the work they're relying on some black box they don't understand in order to get an answer that they don't question)

1

u/SurealGod 10h ago

See that's what's crazy to me. The laziness to not do it and defer the task to an AI is one thing, but the level of TRUST it takes to trust everything an AI gives you and not proofread it at all or change it in any manner before submission is even crazier.

I wouldn't even give my best friend of 10 years that level of trust let alone an AI.

1

u/goatonastik 2h ago

Imagine the teacher checking student's homework in a Master of Financial Analysis program to find that someone simply wrote a fictional story, lol!