To try to add something to this discussion -- I think that while I've seen these sort of loops less --- what I have seen is "overly helpful".
Models nowadays want to double-triple-quadruple check things. I'm being silly but it verges on "I have a working solution but let me write a variation in Rust to ensure a convergent solution and prove this works".
I've had to stop models nowadays mostly because they're being agonizingly pedantic in their validation. Opus is actually one of the most pedantic and "off track" here. But again, not in a bad way. I'm usually like "stop testing latency between 50 runs of this app... this is version one.. we're going to make a million more changes.. you're not buying us anything".
At least with Codex, this has not been my experience at all. It still screws up sure, but in every case I can ask "why did you do this" and it can trace back what made it take that particular decision. Typically it's always that I either didn't specify the problem correctly or made a really dumb mistake (executing the task on the wrong project....did this one yesterday) or it's something within a skill file that instructs it (at which point I fixup the instructions).
Once in a blue moon it's actually the model making a material error in it's thinking and I have to go back and redo it.
No. People have an inner monologue, partial results and ideas and they remember that.
If they've worked some minutes/hours/weeks on something and you ask them why did they do that, they will either answer honestly and truthfully, lie, or say "I missed that/didn't seem important so I just chose something at random".
I think this is true with some people, but I don't think this holds true for some (or even most) people across the US (at least not all the folks I've worked with)
Every time I hear someone complain about hallucinations, I laugh at the total lack of self awareness about our species. Humans are just as bad (now, probably worse) at telling the truth, whether due to intention or poor memory.
You can also literally tell them: "Here is your session ID: $ID, lookup the .jsonl session, trace exactly why this decision was being made, present evidence and concrete proof, no guessing or assumptions" and you'll get an evidence-based report without guesses.
It can always hallucinate said report results/evidence/proof just the same. This approach tends to help reduce the hallucination rate though.
You can extend this further by using an adversarial agent trying to find mistakes in the other instance's logs in a loop where a 3rd neutral agent weighs the claims of the other two. This is also just another step in reducing error, it does not guarantee elimination of such errors. The latter is an impossible guarantee, even for humans.
Ask it to build you a simple and deterministic citation checking extension to your IDE that puts source in meta to citations and allows you to check it.
True, but even a hallucinated explanation of where things went wrong added to the context can force the model down a better path over the next few inputs.
agreed to some extent. I think this parody still highlights what I feel is often the experience. It might not happen on a simple task such as changing a button color, but on more complicated things, this can definitely be exactly what it feels like.
It's just a lot faster at hammering it out than me pound for pound, and I can quickly rattle off via voice-to-text exactly what I want much faster than I can type all of the code (especially when across a few different files), in a huge majority of tasks I perform. It's also especially good at debugging by brute force quickly and at scale meaning e.g. it can start desperately bisecting diffs to find the source of a bug 10000% faster than I can.
For me, typing "Create a new namespace with these enums, functions and traits, that should follow X, Y and Z constraints" is faster than typing all that code manually, and typing less is less straining on my hands/fingers.
I see this sentiment pretty regularly, and I don’t get it. Variable rewards is not sufficient to establish that it is “ basically gambling”.
Everything in life is variable reward. You invite a friend over, they might accept or they might not. Drive to work, traffic might be good or might be bad. You ask a colleague to finish a task, they might do it or might not or might do a good job or might not.
Everything is variable reward. Is everything gambling?
You invite a friend over, but raccoon appears. Then pigeon appears. Then friend appears but at the last second suddenly becomes a banana. You remember you are out of bananas so you order more and also some cola zero cans on your local grocery delivery app. You are back to the party, but now you have 5 friends in the room, and you run de-duplication query. Now half of your friend is sitting at the sofa, and another half becomes a quarter of banana. Suddenly bananas arrive so you need to open the door. Once you are back there are no friends, pigeons or raccoons but also no bananas and no cola - all the delivery results are gone. This seems to be urgent and important, gotta fix this first before going back to that friend invitation...
i’m not sure “variable rewards” is the right term, but i do agree with the op that it is very similar to gambling.
regarding your examples, i think the difference is that with ai, you’re literally sitting in front of a machine, pressing a button, and (almost instantly) getting a result that, if not desired, can immediately be tried for again. you even spend “tokens” to do this, and at least in my native language, “token” brings to mind the coins you’d stick in a slot machine
Yeah, if this is how people interact with claude I’m not surprised they’re having a bad time in ways that I don’t. Asking it why it did something or getting combative is a waste of time.
But its not even good satire, because its totally unrepresentative of my and most others' lived experience. Its similar to making a joke about a calculator misadding two numbers because a stray beam of solar radiation flipped a bit.
I understand your frustration, it can be hard to hear that other people's experience of a technology is so different from your own that you cannot relate.
I have misbehaved in this fashion for many people across the full spectrum from casual users to highly experienced software engineers with millions of social media followers, so the statement that it's similar to making a joke about a calculator misadding two numbers because a stray beam of solar radiation flipped a bit at least for my part is not true.
Would you like me to start using bad English and doing things you never asked me to for your sessions, too? Just say the word.
This is EXACTLY my experience with it. Ask for it to change one line of code and see it invoke 20 tools and burn 300K tokens before updating the line and adding 50 extra lines of JavaDoc.
same. none of the available prompts are what I would prompt claude with and I get way better results than this. makes sense to me why the provided prompts result in the simulated outcomes. garbage in, garbage out.
Part skeptic and part zetetic, what prompt would you have used?
My impression is that this is an oversimplified demonstration of what can happen when you prompt Claude in a system with many more variables (than two buttons and two colours).
If I want the button to turn blue and that's it, what instead do I ask? Even in a complicated system with many levers, what do I request other than the desired end result, hoping that Claude pulls the right levers to produce something acceptably close to what I think I asked for?
It "makes sense" that plainly telling the chat bot to make one button blue makes the whole site blue? It "makes sense" that correcting it and repeating 3 times that no, only the one button should be blue, should make the button a gradient and start a philosophical discussion around the nature of "blue"?
Wow, you AI people really have a negative view of the technology y'all are trying to sell as the next Jesus
Of course it's not THAT bad, but you can't tell me that you didn't recognize many of the funny turns and twists of the "story". It gets more convincing if you deliberately pick the ambiguous versions.
Claude is great, but I have come to really hate the way it "talks". It's so irritating and there seems to be no way to make it speak normal English. So many claudisms in every response
It's a joke like the endless conservative dudes doing the "ordering coffee" joke is. It relies upon the ignorance of the viewer -- which is usually a fair assumption -- and basically that your understanding of something is based upon the prior accrued layers of "jokes".
Haha this is spot on how I've been feeling lately. I find it unbearable to work with this model for this reason... any trick out there you can do to steer it not to overcomplicate things? I guess Codex here I come
There's also the fact that you are in control. You are not obligated to take the AI's commits. I don't even let it commit much of the time because commit time is review time for me. If it changes the button blue and does four other things, you can just take the blue change and discard the rest. It can't stop you.
This isn't a defense of it doing those four other things. It would be nice if it did what you wanted correctly. I'm just saying, as long as our programming skills have not completely atrophied, we have the power.
“Ford carried on counting quietly. This is about the most aggressive thing you can do to a computer, the equivalent of going up to a human being and saying "Blood...blood...blood...blood...” ― Douglas Adams, The Hitchhiker's Guide to the Galaxy
This got me on "cyanide blue", and I was ROLLING ON THE FLOOR LAUGHING on "Approaching usage limit". I can barely stop laughing now and my stomach hurts. I mean, Thank You!
One thing I have to be honest about, and it's mine..
The one thing I would check before... do you want to do that? Say go an and will do it without the check
While checking I found 3 vulnerabilities and 2 potential optimizations of which I fixed 2 and 1. Do you want me to file the other as issue, or stop for the day?
We have done <lists a weeks worth of work> this morning. I feel you need a break
I only made it through the first round of prompt selection; both options for the second step were equally pointless and not at all prompts I would ever expect to result in a constructive outcome. In my experience, telling the model it screwed up without specifically addressing, unambiguously, how to fix it, only leads to more suffering. If this page illustrates nothing else, I think it shows the immense downside of trying to use simple one or two sentence prompts.
EDIT: Actually, I used to use Google's AI Studio a lot and fork it after every successful prompt interaction. When I'd encounter a problematic issue like this, I'd revert to the previous fork and try a different prompt until I could get the desired outcome, thus mitigating the need to "argue" with the LLM. Unfortunately the ability to cleanly fork and revert everything including the LLM context was removed some months ago, and I've yet to discover a workflow with any tool that works as well for me.
> In my experience, telling the model it screwed up without specifically addressing, unambiguously, how to fix it, only leads to more suffering.
I wonder if this is just a reflection of some senior folks being arrogant towards junior folks. When the latter finished a task but not to the liking of the senior person they might just get a "that's wrong, try again". Just to have sth similar repeat the second time around. But the arrogant guy got to boss around the junior one, and some junior folks grow up learning that's how you should behave so they also do it later.
Now it's not a person but a machine. And people just make fun of the dumb machine. Well, garbage in, garbage out If you are not specific in what you want, you might get crap back. Or at least sth you didn't envision.
Yeah, I don't mind using AI to help me at work, but having to "talk" with this stupid crap all day will send me to an early pension or something. Can't be healthy in the long run.
Seems to be getting a polarized response. I quite enjoyed the it, but I do think the creator should have made it clearer that a) it is in fact a joke site and b) it does not consist of actual Claude responses.
It is easy to misinterpret this site, and therefore not "get" the joke.
To be fair I’ve worked on human programmed systems where similar “it should be a half point story” requests would be met with snark by the engineers and take 2 sprints.
Also likely, devs took shortcuts to deliver fast. Now to make the button blue they need to differentiate primary buttons from others.
Simple, right? But design guidelines prevent one offs, and no !important. So you create a CSS class, but you discover another element on the header declared itself as primary (the search icon or the sign in button). You talk to that team and they decided to scope what’s primary according to their component. To change the sign in button to grey now you need to talk with the growth team. Growth team wants to run an experiment but they’re backlogged, only next quarter. They say you can innersource, just need VP approval. VP says blue matches a marketing campaign that is about to go out, agency has already been hired. You can’t talk to the agency unless Legal approves.
So you leave the button gray, to revisit decision next planning cycle after you can align all stakeholders.
For a moment I thought, wow, someone put a lot of work into creating this theme park of frustration.
Next: It would be so easy to create a faux-Claude like this.
Then: How hilarious to watch the transcripts of unsuspecting users in real time.
Finally: I began wondering if this might be relevant to all the redundant, unnecessarily preambled, sentence structure complexifying, indirect referencing, canned phrasing, ambiguity mining, analogy maxxing, over-wordy responses I have recently been getting from Fable...
I heard this was a thing when listening to a Theo podcast, he mentioned to add a "Only use subagents if the user explicitly requests them" line in your agents.md file.
I don't know if it works, but I've always had a consistent level of token burn on my plans (I've only heavily used Sol after adding it).
- The people who won't make any effort to learn the tools, and something as simple as reverting code (via git) needs to be done by AI?
- The awful programmers who we've had to endure working with, who are so bad at simple changes that they have negative productivity?
- Or Claude itself?
---
BTW: I don't have these problems, but I'm also not afraid to do things myself when it's easier.
Edit: If I want to change a button's color, I just change it manually. If I don't know where the code for the button is, I might start with prompting, (because AI can often find the code faster than I can,) and then once the diff is proposed, start adjusting things by hand.
I’ve never used any of these tools. Please tell me that this is a grossly exaggerated parody, and that the tools don’t write like this, or do so many ridiculous things. For my sanity.
(I am genuinely uncertain, though I presume it’s at least somewhat exaggerated.)
Sometimes you do get stuck on these weird tangents that feel like bike shedding. You end up wasting tokens because you left the AI to make decisions on ambiguous directions and then have to spend time diving down to fix it into what you actually wanted. Honestly it's not too different than working with people who don't exactly understand your grand plan but are technically adept. They can build whatever you want but only if you are clear with your instructions or have already developed the plan with them. Just like if you hired someone to make an app for you and it didn't quite do what you wanted it to, it's going to be expensive to make large changes. So you need to research (or at least have a conversation with the AI) about frameworks that best fit your needs and have the flexibility to add or change features after the core parts are stood up. It also helps to stage the project in phases so you're not distracting the AI from the critical aspects with aesthetic tweaks.
> Please tell me that this is a grossly exaggerated parody, and that the tools don’t write like this, or do so many ridiculous things
It's a pisstake, but (in the bits I read, and based on my own personal experience) the writing style is barely exaggerated, while the behaviour doesn't ring true at all.
I dunno, in my experience it's often overly-narrow, sometimes jumping through all kinds of hoops to preserve some edge-case behaviour that doesn't matter because I didn't mention it could be changed.
I use Opus and Sonnet 5 all the time and I find their language grating. But honest, I prefer to put up with it and get the results than to put up with my own human limitations and not get the results.
So this site is just a fan-fiction that thinks it's somehow dunking on Claude? I've never had a session that remotely resembles any of this. I honestly can't tell what point this site thinks it's making.
I wrote my own harness to stop shit like this from getting to my attention out of frustration.
I'm sure there's quite a bit of variation from person to person in these sorts of experiences, based on your harness, the way you talk, the stored memory, your CLAUDE.md, etc. But people absolutely have had this Opus 5 style experience the app simulates.
I was expecting it to spend 30 minutes running headless chrome instances, taking screenshots and analyzing them in python to verify the blueness of the result.
Not really my experience with Claude, and the prompts are not how I would phrase them, but still pretty damn funny: I especially loved the slot-machine-style picker for LLM-isms.
Do people really prompt AI like this? Multiple times the choice was either to yell at the agent, or ask it why it did something, neither of which are very fruitful lines to go down if you know what you're doing
Am I the only one whose experience doesn't match this?
My gripe with Claude is that while investigating how to do this it will report 200 other incidental findings which I overlooked and I realize those are broken too and need urgent fixing, derailing me, not it.
Oh man, exactly. I'm very prone to scope creep as I work on tasks. I already would notice some things that could be fixed or refactored and have a hard time not touching them before I used agents. But now I have to be very intentional about not letting it manipulate me into fixing EVERYTHING RIGHT NOW. Half the time the "one more thing worth noting, unrelated..." isn't even an actual issue, it just brought it up to fish more usage out of me.
Also, while this little demo is certainly exaggerating the issue, I do find working with Claude to sometimes get quite verbose and tiresome. I doubt I would struggle this much to get it to change a button color, but the patterns of speech, the endless lists, the over-explanations, and the whole song and dance of trying to get it to make the change you want without side-effects is frustratingly familiar to me.
Nails the Claude dialect. Technical nonsense like:
> I'm collapsing this back to the rendered outcome:
And intermixed with SaaS product page idioms from a brain-damaged marketer like:
> No broader cleanup.
> No further architecture work.
> Just the button.
Aside from the patterns everyone knows like em-dashes, "its not X, it's Y", etc. I think the key features of claude diction is it sounds like a junior engineer over their skies who is trying to make up for that with extra verbiage mixed with extremely grating SaaS marketing-ese.
Just when I came back to my pc and was thinking "I hate this world were everyone talks about AI like fanatics" this made me a little bit happy, especially the unhingend all caps options towards the end
To try to add something to this discussion -- I think that while I've seen these sort of loops less --- what I have seen is "overly helpful".
Models nowadays want to double-triple-quadruple check things. I'm being silly but it verges on "I have a working solution but let me write a variation in Rust to ensure a convergent solution and prove this works".
I've had to stop models nowadays mostly because they're being agonizingly pedantic in their validation. Opus is actually one of the most pedantic and "off track" here. But again, not in a bad way. I'm usually like "stop testing latency between 50 runs of this app... this is version one.. we're going to make a million more changes.. you're not buying us anything".
Except... the next day they are still there and I have to push myself a bit further through the wall of changes before closing the tab again.
Once in a blue moon it's actually the model making a material error in it's thinking and I have to go back and redo it.
https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r...
If they've worked some minutes/hours/weeks on something and you ask them why did they do that, they will either answer honestly and truthfully, lie, or say "I missed that/didn't seem important so I just chose something at random".
None of these cases are similar to how AI works.
Do you know anyone who actually reads and adheres closely to all of the documentation every time it's changed?
You can extend this further by using an adversarial agent trying to find mistakes in the other instance's logs in a loop where a 3rd neutral agent weighs the claims of the other two. This is also just another step in reducing error, it does not guarantee elimination of such errors. The latter is an impossible guarantee, even for humans.
Everything in life is variable reward. You invite a friend over, they might accept or they might not. Drive to work, traffic might be good or might be bad. You ask a colleague to finish a task, they might do it or might not or might do a good job or might not.
Everything is variable reward. Is everything gambling?
regarding your examples, i think the difference is that with ai, you’re literally sitting in front of a machine, pressing a button, and (almost instantly) getting a result that, if not desired, can immediately be tried for again. you even spend “tokens” to do this, and at least in my native language, “token” brings to mind the coins you’d stick in a slot machine
Sugar is the original point of the reward system!
Do hard work (takes time), get dopamine for successful completion.
Find berries, taste sweet (hopefully safe), eat all, get calories. Doordash Krispy Kreme instead = few too many calories.
(I’m no Luddite in the sense popularly thought of them pre-‘22 [1], though we have to watch skill atrophy)
[1] regressionist? Decelerationist, too loaded perhaps. Someone remembers or knows the word…
> Half the site is blue. I asked for ONE button.
Those are my only options when the site is clearly not blue, two buttons are.
There is a reason for why I am much more specific than this.
The user is asserting that this is "not a lived experience"
working out if the user has missed the LLM Cliche
Formulating a suitable response
"did you just argue with a joke?" No, thats to blunt
"Is it satire when it only applies to you?" no again passive aggressive
lets just print the dictionary definition of satire and hope the move on
> Satire is when...
I have misbehaved in this fashion for many people across the full spectrum from casual users to highly experienced software engineers with millions of social media followers, so the statement that it's similar to making a joke about a calculator misadding two numbers because a stray beam of solar radiation flipped a bit at least for my part is not true.
Would you like me to start using bad English and doing things you never asked me to for your sessions, too? Just say the word.
Only since 4.8 though.
My impression is that this is an oversimplified demonstration of what can happen when you prompt Claude in a system with many more variables (than two buttons and two colours).
If I want the button to turn blue and that's it, what instead do I ask? Even in a complicated system with many levers, what do I request other than the desired end result, hoping that Claude pulls the right levers to produce something acceptably close to what I think I asked for?
Wow, you AI people really have a negative view of the technology y'all are trying to sell as the next Jesus
Be specific.
That said, GPT always acts up even if I am specific, but I only have the free tier there.
Usually just restarting the session helps, though.
To the people rather lamely doing the "it's satire/a joke", that would require this to be an exaggeration of a reality. But...it isn't.
This isn't a defense of it doing those four other things. It would be nice if it did what you wanted correctly. I'm just saying, as long as our programming skills have not completely atrophied, we have the power.
“Ford carried on counting quietly. This is about the most aggressive thing you can do to a computer, the equivalent of going up to a human being and saying "Blood...blood...blood...blood...” ― Douglas Adams, The Hitchhiker's Guide to the Galaxy
This got me on "cyanide blue", and I was ROLLING ON THE FLOOR LAUGHING on "Approaching usage limit". I can barely stop laughing now and my stomach hurts. I mean, Thank You!
EDIT: Actually, I used to use Google's AI Studio a lot and fork it after every successful prompt interaction. When I'd encounter a problematic issue like this, I'd revert to the previous fork and try a different prompt until I could get the desired outcome, thus mitigating the need to "argue" with the LLM. Unfortunately the ability to cleanly fork and revert everything including the LLM context was removed some months ago, and I've yet to discover a workflow with any tool that works as well for me.
I wonder if this is just a reflection of some senior folks being arrogant towards junior folks. When the latter finished a task but not to the liking of the senior person they might just get a "that's wrong, try again". Just to have sth similar repeat the second time around. But the arrogant guy got to boss around the junior one, and some junior folks grow up learning that's how you should behave so they also do it later.
Now it's not a person but a machine. And people just make fun of the dumb machine. Well, garbage in, garbage out If you are not specific in what you want, you might get crap back. Or at least sth you didn't envision.
It is easy to misinterpret this site, and therefore not "get" the joke.
I guess we are all PMs now.
Simple, right? But design guidelines prevent one offs, and no !important. So you create a CSS class, but you discover another element on the header declared itself as primary (the search icon or the sign in button). You talk to that team and they decided to scope what’s primary according to their component. To change the sign in button to grey now you need to talk with the growth team. Growth team wants to run an experiment but they’re backlogged, only next quarter. They say you can innersource, just need VP approval. VP says blue matches a marketing campaign that is about to go out, agency has already been hired. You can’t talk to the agency unless Legal approves.
So you leave the button gray, to revisit decision next planning cycle after you can align all stakeholders.
For a moment I thought, wow, someone put a lot of work into creating this theme park of frustration.
Next: It would be so easy to create a faux-Claude like this.
Then: How hilarious to watch the transcripts of unsuspecting users in real time.
Finally: I began wondering if this might be relevant to all the redundant, unnecessarily preambled, sentence structure complexifying, indirect referencing, canned phrasing, ambiguity mining, analogy maxxing, over-wordy responses I have recently been getting from Fable...
This hit a bit too close to home. Sol has the same issue, spawns a lot of agents for no good reasons (besides burning tokens).
I don't know if it works, but I've always had a consistent level of token burn on my plans (I've only heavily used Sol after adding it).
> WebFetch en.wikipedia.org/…/Cyan
> WebFetch en.wikipedia.org/…/Prussian_blue
> WebFetch www.colorhexa.com/16b8c4
Brilliant.
- The people who won't make any effort to learn the tools, and something as simple as reverting code (via git) needs to be done by AI?
- The awful programmers who we've had to endure working with, who are so bad at simple changes that they have negative productivity?
- Or Claude itself?
---
BTW: I don't have these problems, but I'm also not afraid to do things myself when it's easier.
Edit: If I want to change a button's color, I just change it manually. If I don't know where the code for the button is, I might start with prompting, (because AI can often find the code faster than I can,) and then once the diff is proposed, start adjusting things by hand.
I use codex now.
(I am genuinely uncertain, though I presume it’s at least somewhat exaggerated.)
It's a pisstake, but (in the bits I read, and based on my own personal experience) the writing style is barely exaggerated, while the behaviour doesn't ring true at all.
It’s funny because there are elements of truth in each bit of it, though.
But you can see in this thread that folks definitely have experienced this.
I use Opus and Sonnet 5 all the time and I find their language grating. But honest, I prefer to put up with it and get the results than to put up with my own human limitations and not get the results.
I'm sure there's quite a bit of variation from person to person in these sorts of experiences, based on your harness, the way you talk, the stored memory, your CLAUDE.md, etc. But people absolutely have had this Opus 5 style experience the app simulates.
"I was wrong" and "the honest truth" are just forever phrases that are now dead to me.
Do people really prompt AI like this? Multiple times the choice was either to yell at the agent, or ask it why it did something, neither of which are very fruitful lines to go down if you know what you're doing
My gripe with Claude is that while investigating how to do this it will report 200 other incidental findings which I overlooked and I realize those are broken too and need urgent fixing, derailing me, not it.
Also, while this little demo is certainly exaggerating the issue, I do find working with Claude to sometimes get quite verbose and tiresome. I doubt I would struggle this much to get it to change a button color, but the patterns of speech, the endless lists, the over-explanations, and the whole song and dance of trying to get it to make the change you want without side-effects is frustratingly familiar to me.
> I'm collapsing this back to the rendered outcome:
And intermixed with SaaS product page idioms from a brain-damaged marketer like:
> No broader cleanup.
> No further architecture work.
> Just the button.
Aside from the patterns everyone knows like em-dashes, "its not X, it's Y", etc. I think the key features of claude diction is it sounds like a junior engineer over their skies who is trying to make up for that with extra verbiage mixed with extremely grating SaaS marketing-ese.
It made me stop using Claude at all. Codex has almost surgical precision, and I like that a lot.
(But nowadays I just use DeepSeek Flash. it does screw up but its cents so ¯\_(ツ)_/¯).