Forget about the scores... look at the Cost Per Task. Fable 5.1 was already the most expensive by a good margin, and now Fable 5.2 doubles it! This is with the price cut.
The output tokens per task just exploded. From 118k tokens (46k Answer/71k Reasoning) in 5.1 to 274k tokens (81k Answer/192k Reasoning) in 5.2.
There's a whole frontier of cost vs performance and this model pushes it out in a bunch of places, so its not just a matter of making a different tradeoff as you make it sound.
That would be my layman's assumption as well. A theory is they're just releasing this as a stepping stone towards something bigger before the IPO. Either way it's the same way chinese models get higher scores on benchmarks. Drop enough of wait actuallys and you'll be top the charts, but time per task and tokens per task go up by a mile (as is the case with this release).
I think we will beat the cost with Hardware. Faster, more efficient hardware. From what I have seen we have only found ways to make it smarter and not more cost effective.
I'm curious to see the next generation of dedicated AI processors and how they perform.
Its still prohibitively expensive, and models 5% the cost are faster and get the job done just as well (for the average person's workflow).
I really don't get the value proposition of these massive models, seems like they're just loss leaders simply to say you're the in the top 3. There's not much use cases outside of niche academic research (and even then I question the need). They mostly function as marketing tools, like a casino that offers high roller accommodations.
People will say "cyber" but companies didnt want to spend on security before, not sure why they would all of a sudden be compelled to spend even more.
The frontier chart there makes gemini carbon look really weird. Low is worse and more expensive than medium, and high is worse and more expensive than medium... funky curve like that implies a messed up model or weird benchmark.
It beats opus 5.5's max effort with less cost, which is a step forward if you don't want to crank it to Ultra Max Settings.
The amount of compute it cost to get the best previous performance from anthropic went down. Nobody is putting a gun to your head and forcing you to go further down the paretto frontier if the current level of intelligence/task is acceptable to you.
Those are both true, but I don't see how they are meant to rebut what I said.
Most people on plan use do NOT use up all their plan usage.
Users on API plans can still use Opus 5.5 which is a great model. If they want top of the line performance, they can pay more.
When another company puts out a model that undercuts Fable 5.2 on price at similar performance, Anthropic will put out Opus 6 and undercut them right back. That's how the wheel turns. This is just part of the cycle.
Most is doing the heavy lifting here. I am a power user. And I max 3 claude max subscriptions on opus 5.5 high and a single codex pro 5x for personal use.
Cool! It looks like Fable 5.2 (especially low and medium) also sit on the pareto frontier between time per task and intelligence: https://artificialanalysis.ai/models?speed=output-speed-over... (or if the link does not work, scroll down to performance, then select graph titled "Intelligence Index vs. Time per Task")
I am always interested in this comparison since it shows how quickly you can go back and forth with a model, capturing both tokens/task and tokens/second. The new models seem to be meaningfully better here.
Why does AA choose ~$3.1 per task to be the breaking point for "most attractive quadrant" highlight? Surely the most attractive section is just being close to the entire frontier?
https://artificialanalysis.ai/#intelligence-efficiency-tabs this is the chart that matters for most of poor developers and companies. Pareto for a given intelligence index and looks like only openai is across the quadrant with muse at the bottom. They have really cooked efficient models!
Fable, for me (it might be a completely different experience for you) has been like the well read snobbish kid at school that openly ignores me when I ask certain questions and often asks me to talk to someone else on other questions.
That kid might indeed be a genius, clearly bright, but is more interested in deciding whether I should have asked a question than in answering it.
For me personally, if the best model on the planet refuses to answer why I shouldn't mix bleach and ammonia, where I'm trying to understand the chemistry behind the warning, it's not pareto frontier to one that atleast tries to and points me in a general direction. Being told that the topic is concerning, referred to someone else and charged part of my quota for that experience feels like getting an invoice from that school genius for telling me to "just ask the chemistry teacher".
If my LLM asks me to rethink my life when I ask it to reverse engineer the firmware of my Samsung TV that I suspect is setting up adhoc radio networks to relay my data to Samsung's cloud, I feel rather sad, specially when that kid sends me an invoice for saying "no"
At least Anthropic should not bill me for the refusal (as of my last tests, refusals still comes out of my quota) - getting slapped and pickpocketed for asking a question is really upsetting.
Google could have been #1 for a little while, but they botched their release and squandered their temporary lead. (As usual.) If I worked there, I'd be so pissed.
I think it's an unwarranted assumption that a temporary benchmark lead is the focus. I think using our resources efficiently to provide helpful solutions is more important.
This model is so far off the efficiency frontier that I personally wouldn't want to use it. The wait would reduce my productivity instead of increasing it.
Optics and incentives. Google will turn on the R&D the day after their Anthropic investment is realised, maybe $80-150bn on a 1.2tn valuation, plus >100bn in spending commitments.
I tried using it as part of my Pro/One/whatever subscription (the one that comes with Youtube Lite). For code review, it was able to find some things that Sol 6.1 couldn't. I found it quite useful. But for code generation, it's pathetically slow and the code isn't impressive. It's at least one year behind.
I do. I prefer it to Opus 5.5 - just as good at coding for my needs, but so much more pleasant to work with.
I am not sure how people keep saying google are not in the race, or have lost it or whatever when just a few hours ago they had the #1 model? What more do people want?
I actually just think it's a meme to talk down on Gemini at any opportunity here. There's probably a fair number of people that wrote off antigravity at its (pretty rocky) launch. And Google also doesn't provide any kind of good sandbox-y experience to Just Try their models, which probably bites them here too.
But Gemini models are still neck and neck in the lead both in my experience and in benchmarks, and Gemini is far above the competitors (although Opus 5.5 closed much of the gap) when it comes to any kind of complex web dev. (I generally use Astra for anything 3D, Gemini for anything web/mobile, and Opus for everything else.)
In my experience Google models have always been mediocre at best and dangerous to use at worst but I must admit that I didn’t try the latest Aragon/carbon crop.
I learned not to put too much trust into benchmarks. The best benchmark is to use the model on a codebase you know well.
Their latest internal reshuffling and the strange amount of parallel releases with the same target, their kafkaesque barriers to entry, and a lack of clear vision is what makes people think they’re not in the race anymore. Obviously they still are. They develop new models and also are deep in the compute game too.
It's insane how people nitpick on the highest thinking setting for every anthropic release and claim it's too expensive. Not trying to stan for anthropic, but what am I missing here: Just don't use those models on the highest thinking setting?
Fable 5.2 medium for example seems a really good choice when it comes to the performance vs task (if you have a complex task that requires such a model of course).
This has got to be a minor reason why adoption of AI isn’t faster.
Businesses like predictable costs and predictable performance.
If your new model is slightly better but crazy expensive, but doesn’t have to be with certain settings, but my user can accidentally run it that way by mistake or maybe your harness makes it the default by accident and the user doesn’t notice…
My beloved predictability is destroyed and trying to get users to understand this is useless.
Imagine a streaming subscription that worked like this: “We’ve updated our catalog with more movies. If you want the premium ones though you only get 2 per week and then you have to wait. But if you do mostly classics and a few from the 2010s you can squeeze in a recent theater release. If you time it right and do only one every day you can see 30 modern classics this month, but you can’t watch three today. Also your bill this month is $40 instead of $20 due to the library update”.
Yep, and now multiply that complexity by hundreds or thousands of employees. That's why companies are giving in to crude, hacky solutions like monthly billing caps. What else are you really gonna do?
The output tokens per task just exploded. From 118k tokens (46k Answer/71k Reasoning) in 5.1 to 274k tokens (81k Answer/192k Reasoning) in 5.2.
Notably, in order to achieve a specific quality, the model has to "reason" for every prompt proportional to the training inference.
This is arguably the case of diminishing returns being reached, as answer token prices saturate the willingness of customers to pay.
They've run into a wall.
I'm curious to see the next generation of dedicated AI processors and how they perform.
I really don't get the value proposition of these massive models, seems like they're just loss leaders simply to say you're the in the top 3. There's not much use cases outside of niche academic research (and even then I question the need). They mostly function as marketing tools, like a casino that offers high roller accommodations.
People will say "cyber" but companies didnt want to spend on security before, not sure why they would all of a sudden be compelled to spend even more.
The amount of compute it cost to get the best previous performance from anthropic went down. Nobody is putting a gun to your head and forcing you to go further down the paretto frontier if the current level of intelligence/task is acceptable to you.
Most people on plan use do NOT use up all their plan usage.
Users on API plans can still use Opus 5.5 which is a great model. If they want top of the line performance, they can pay more.
When another company puts out a model that undercuts Fable 5.2 on price at similar performance, Anthropic will put out Opus 6 and undercut them right back. That's how the wheel turns. This is just part of the cycle.
This might be true for a small portion of the user base, but it is not true as a statement of fact.
I am always interested in this comparison since it shows how quickly you can go back and forth with a model, capturing both tokens/task and tokens/second. The new models seem to be meaningfully better here.
Another member of the pro-REPL / anti-agentic tribe. There are dozens of us!
I had to remind them 4.6 only came out February this year. In a normal world 4.6 is a newer model! A model 6 months old is seen as ancient!
That kid might indeed be a genius, clearly bright, but is more interested in deciding whether I should have asked a question than in answering it.
For me personally, if the best model on the planet refuses to answer why I shouldn't mix bleach and ammonia, where I'm trying to understand the chemistry behind the warning, it's not pareto frontier to one that atleast tries to and points me in a general direction. Being told that the topic is concerning, referred to someone else and charged part of my quota for that experience feels like getting an invoice from that school genius for telling me to "just ask the chemistry teacher".
If my LLM asks me to rethink my life when I ask it to reverse engineer the firmware of my Samsung TV that I suspect is setting up adhoc radio networks to relay my data to Samsung's cloud, I feel rather sad, specially when that kid sends me an invoice for saying "no"
At least Anthropic should not bill me for the refusal (as of my last tests, refusals still comes out of my quota) - getting slapped and pickpocketed for asking a question is really upsetting.
There's a setting to not degrade to a lower model on refusals and instead pause the chat.
I think it's an unwarranted assumption that a temporary benchmark lead is the focus. I think using our resources efficiently to provide helpful solutions is more important.
This model is so far off the efficiency frontier that I personally wouldn't want to use it. The wait would reduce my productivity instead of increasing it.
I am not sure how people keep saying google are not in the race, or have lost it or whatever when just a few hours ago they had the #1 model? What more do people want?
But Gemini models are still neck and neck in the lead both in my experience and in benchmarks, and Gemini is far above the competitors (although Opus 5.5 closed much of the gap) when it comes to any kind of complex web dev. (I generally use Astra for anything 3D, Gemini for anything web/mobile, and Opus for everything else.)
I learned not to put too much trust into benchmarks. The best benchmark is to use the model on a codebase you know well.
So, what more do I want? It'd be a good start to actually tell me it's available and giving me a clear path to using it via plan & harness?
well, releasing it would be nice
Max should not be used unless you really need every bit of perf, it's deep into diminishing returns territory.
Change the models in the dropdown to medium / high and Fable 5.2 is about twice as cost efficient, and slightly faster.
Fable 5.2 medium for example seems a really good choice when it comes to the performance vs task (if you have a complex task that requires such a model of course).
Businesses like predictable costs and predictable performance.
If your new model is slightly better but crazy expensive, but doesn’t have to be with certain settings, but my user can accidentally run it that way by mistake or maybe your harness makes it the default by accident and the user doesn’t notice…
My beloved predictability is destroyed and trying to get users to understand this is useless.
Imagine a streaming subscription that worked like this: “We’ve updated our catalog with more movies. If you want the premium ones though you only get 2 per week and then you have to wait. But if you do mostly classics and a few from the 2010s you can squeeze in a recent theater release. If you time it right and do only one every day you can see 30 modern classics this month, but you can’t watch three today. Also your bill this month is $40 instead of $20 due to the library update”.