I think he doesn't need to understand the technology to point out the books are cooked. a business can sink in either way: the technology flops or the finances flop. he's arguing the /finances/ would flop. he doesn't argue that the /technology/ would flop, only that they can't come up with the money to pay their debters.
There is a piece of this I agree with. That you do not need to be a deep technical expert to notice that a company is burning cash by overcommitting to capex, or relying on heroic revenue projections that may or may not come to pass.
But that is not the full argument he is making. If the claim is that the labs will not be able to pay their creditors because inference is structurally incapable of becoming profitable, then he absolutely needs to be right about the technical economics of inference.
One part of that is the balance-sheet argument (which already shows insanely good margins). But it also depends on how inference-time compute actually works: routing, batching, kv cache reuse, model segmentation, different latency tiers, etc. Much of those details he's just been straight up wrong about in his writing, so as a result I have to call into question the rest of his reasoning as well (in part to avoid Gell-Mann amnesia).
Doesn't this kinda imply its own smoke and mirrors though? Like if the name of the game with inference is already routing things around and caching so you can make money, why is the newest biggest model always the most important critical thing? How does this square with any of their press about it? Also wouldn't that just add more inference? Because you need to pre-judge every prompt to know where to route it?
Also, if there is significant gains from caching, then like.. what are even doing here? Inputting something and then reading cached pieces of text based on their similarity to the input? Kinda like a search engine?
I don't think its smoke and mirrors, though I do have plenty of gripes with how the labs market this product landscape generally speaking.
The newest biggest model can still matter even if you do not run every prompt through it. You'll always have some task where even small amounts of loss are unacceptable and thus you need to make sure frontier intelligence is used for it.
On the router point, yes, routing has some overhead. But the router does not need to run the biggest model to decide which model to use. We've been using tiny classifiers for recommendation engines for ages now, usually on CPU. If routing saves you from sending a large fraction of traffic to the expensive reasoning model, the routing overhead can easily be worth it.
> Also, if there is significant gains from caching, then like.. what are even doing here? Inputting something and then reading cached pieces of text based on their similarity to the input? Kinda like a search engine?
The caching I'm talking about is explicitly the attention/kv cache, so its not input similarity retrieval (that would be more like what you'd use in a RAG/IR system). Prompt caching is generally about reusing already-computed attention scores for repeated prompt prefixes. The idea being you don't recompute the same static system prompt, tool definitions, schemas, long shared context, or repeated boilerplate every time. In more sophisticated systems, you usually store multiple checkpoints so that a small prompt change doesn't result in all-or-nothing hit/miss scenario.
I think the earlier commenter is right. If a parent fails to... well... "parent" that's on them. Locking down the internet to republican-approved sites only is not the answer.
musk's public image is well known, and the statement is reflecting a useful heuristic / business sense; some customers really would use claude less knowing that it's buddies with musk now. Your own immaturity leaks through. I would say that the one weakness for OP is that people okay with using claude but not okay with grok might be a different size overlap than expected (either more people fine with both, or more people disjoint, or it just doesn't factor in for people).
Assuming "slang" is an exaggeration: the basic principle of OSM is that things are mapped and named as they are on the ground. So local names and naming conventions should work; if they don't, it's a bug.
Why do I get the feeling that AI skeptics will treat it as definitive and irrefutable proof that they were right all along even though it’s one data point in an industry that’s hasn’t even been around for 5 years.
You're right, it is tempting to dunk on AI boosters every time an article like this comes out and puts a damper on their sci fi fan fiction fantasies. There's just something about a grown person getting all excited like a child that makes it really satisfying.
You must have a really bleak view on life to think an adult should never get excited like a child.
Adult life doesn’t have to be boring drudgery, you know. I mean, it mostly is, but the rare moments of childlike joy and excitement are some of the best parts.
As far as the putting a damper on anything, nope it doesn’t. And it never will.
The people excited about AI are excited because of the impacts they see on their own jobs and daily lives. We don’t care what Goldman Sachs has to say about productivity.
It’s a grift being perpetuated by the folks at the top, who then sweep along in their slipstream folks under them, and so on. The folks who “need to hear this” are helpless to go against and so can’t back down, and the folks who don’t need to hear this because they’re driving it have their paychecks aligned to it, so they’re not backing down.
This is kinda... rude. Like saying that a GUI doesn't serve a purpose when people could read the TTY.
CI gives you areas for your bash scripts to run in self-contained small runs, that may trigger other runs, in a repeatable fashion on a clean environment, on a GUI anybody in your team can see. It gives you quick integrations into things.
CD lets you repeatedly deploy - without forgeting a step that was only known to Phil, the guy that retired three years ago, remembering all the steps and doing something dependably.
reply