Oracle's recent force majeure invocation shines a spotlight on what seems to be an endemic problem: datacenter builders have been signing commitments to bring a certain amount of capacity online by a certain date, without first conducting sufficient due diligence about whether they would be able to secure all the resources needed to meet those commitments.
This means they are now caught in a mad scramble to source all sorts of stuff, including RAM, potentially as a matter of life and death.
What's less clear to me, though, is whether datacenter builders will be able to make good on these purchase agreements. It seems these projects have every opportunity to fail, and the path to success keeps getting narrower. What are the odds that chip manufacturers ultimately get caught holding a bag of chips that nobody can buy in one hand, and a bag of IOUs from bankrupt customers in the other?
or you can wait until one of those frontier labs IPO goes bust in a really bad way where it pumps 10x on day 1 and then everyone dumps every share out there and half a million people are holding bags. that ll surely wipe out all the AI hype
Not sure I buy that? There's a hardware/process argument that DRAM scaling is hitting a wall (for largely the same reason that logic scaling is, even though the fabs are different).
But DRAM has always been almost completely dominated by consumer device needs, and consumer requirements leveled off hard maybe a decade ago. Once you had a 4G system, things were mostly OK. Pervasive VM use and general bloat pushed that up by maybe a factor of two or three since, but that's it.
AI is different. It wants that capacity for different reasons and puts it in different places. And almost none of us really saw that coming more than ~18 months ago or so. Certainly not to the extend that you'd conclude "we've been ignoring it".
If training and inference is hardware constrained, and you can train and server better and bigger models on the same hardware with memory optimizations, that's exactly what I would expect companies to do.
What was the benefit of all this AI stuff again? Because all I see is middle and lower class being fucked over, repeatedly. Increasing prices of consumer electronics, job losses, suffering quality of goods and services, consolidation of power in favor of elites.
Think about the app sitting on some developer's laptop, necer getting deployed anywhere else! Or the thousands getting uploaded to app stores to slurp and sell data!
This means they are now caught in a mad scramble to source all sorts of stuff, including RAM, potentially as a matter of life and death.
What's less clear to me, though, is whether datacenter builders will be able to make good on these purchase agreements. It seems these projects have every opportunity to fail, and the path to success keeps getting narrower. What are the odds that chip manufacturers ultimately get caught holding a bag of chips that nobody can buy in one hand, and a bag of IOUs from bankrupt customers in the other?
But DRAM has always been almost completely dominated by consumer device needs, and consumer requirements leveled off hard maybe a decade ago. Once you had a 4G system, things were mostly OK. Pervasive VM use and general bloat pushed that up by maybe a factor of two or three since, but that's it.
AI is different. It wants that capacity for different reasons and puts it in different places. And almost none of us really saw that coming more than ~18 months ago or so. Certainly not to the extend that you'd conclude "we've been ignoring it".