I agree and I think the most concrete demonstration of this for me is actually the current air.
It's basically the idea of the pro of 10 years ago but realised flawlessly, and improved upon actually in most ways, for significantly cheaper.
To do this they had to become an execution machine at huge scale to own and solve every piece that held them back in that pro design, and they did, and it's honestly amazing how it all came together.
I think there's a ton of examples where this is true for lower level stuff like open source where you see the internals.
For commerical products it certainly exists too, for example in those cases where you know the product is built by one person or a small group of people who you absolutely know take extraordinary care to get all the details right, and it shows through as a really nice intangible feeling when you're using the product.
That (kind of rare to be honest) 'oh this is just really well done' feeling.
Seems roughly right, that does seem to be about the boost in the most well-suited cases where you essentially know exactly how to solve the problem, the problem won't change much, and it's truly a matter of just churning out the implementation.
In that case precisely prompting, doing the review & nudge loop, can be a pretty nice (nice, still not game changing) speed boost over literally typing out the code to match the design in your head.
The less optimistic view though is that most things you build aren't like that. Even if they seem like it first. These things get booked as a nice speed boost, but you'll only find out much later they weren't.
A confounding factor is that it seems like many people not in the detail of building software do seem to think of most to all things are like that, even before AI assisted coding. Not much need to say more - see the entire history of the 'agile' movement for evidence of this.
And because most things aren't like that, I actually struggle to see fundamentally how more than 20-40% will ever be achieved (short of the ever-present deus ex machina of AGI argument), simply because the generation is already really good for these types of things. So since things like this aren't going to increase in overall proportion of things to be done, I don't see where the overall extra gains come from by models improving at this point.
I think unfortunately it's not about what seems obvious, or even what seems more likely, but about what seems retrospectively justifiable regardless of outcome.
The incentive structure of this type of decision is 'absolutely under no circumstances existentially mess up'. Ostensibly with respect to the organisation, but in actual reality much more so with respect to the individual(s) involved in the decision.
If everyone else is doing something that kind of obviously makes no sense, and you decide to break from the crowd by instead doing what does make sense, then there's a pretty solid chance of gaining a temporary edge while reality resolves the truth. But those gains probably won't matter all that much for the organisation, or indeed your position within it. It's a solid chance of an unimportant gain.
However on the other hand, there's a tail risk that something very unexpected happens and the thing everyone's doing that makes no sense actually turns out to make sense - sometimes even for entirely unpredictable incidental reasons - and then, well, you're in trouble. Not necessarily 'you' the organisation.. they'll likely be able to catch up and it won't matter that much. But for 'you' personally, the decision maker, it's very much not good.
As a bonus, in the much more likely scenario that the thing that makes no sense turns out to indeed make no sense, you're in the same boat as everyone else, there's no relative loss, and most importantly you don't stick out as someone who did something as risky as to go against the prevailing, albeit pretty clearly nonsensical, sentiment.
So basically, game theory tells you pretty quickly to just go with the thing that makes no sense if you're optimising for some (weighted) cross of what's best for the organisation and yourself as the decision maker.
I was just thinking how it'd be great if there were newer, modern things like this that had sprung up in response to newer technologies.
I guess it's one downside of dematerialisation with digital tech - I can't think of a single thing that would make sense. Everyone's got their own virtual portal to all the new technologies that come out, there's not much to look at out in the world.
Maybe as more progress happens in physical 'world of atoms' type things we'll see a bit of this come back.
As someone that tries to survive (every day more difficult) with just a dumbphone with me, I just fantasized about a parallel universe where all those kiosks still existed, and they were somehow like computer that you can briefly rent, to do the things people do with a smartphone. Perhaps you tap a card, and it picks your accounts, and you can quickly Whatsapp someone, check your email, call an Uber or use Google Maps (maybe even check hacker news, but with time limit?!)
Maybe then many people would stop carrying their own portals, as you can briefly use the public ones for the one-off situation where you need it, but enjoy a portal-free mind the rest of the time. Also quite useful in case emergencies as it seems those portable-portals tend to run out of battery, or get lost or damaged...
There was a brief moment around the turn of the millennium where that was what some of us were expecting. I was in college just after that and there were some free Internet kiosks, which combined with the ubiquitous free-to-use computer labs on campus, did a pretty good job of making this type of lifestyle possible, to the extent you could store your important documents online (much harder though in a pre-Dropbox, pre-Google Docs world!). Or another thing that was a big trend then was Portable Apps. On a flash drive on your keychain, you'd have installs of apps that you needed, together with their data and whatever documents you might need.
There was a brief period before that where some airports had pay phones with text terminals and modems built in, so you could dial up your corporate email or CompuServe. I swear I did not dream this.
a parallel universe where all those kiosks still existed, and they were somehow like computer that you can briefly rent, to do the things people do with a smartphone
In the early days of the Internet, this existed somewhere in western Europe where I was traveling.
There was a phone booth down the block from my hotel and a couple of times I popped in and used it to check my email.
Maybe Vienna or Amsterdam? It was a long time ago. Swiping one's own credit card would have been unusual then, so I have no idea how I paid for it. Especially since I mostly used travelers checks.
In some Asian countries electric scooters with swappable battery packs are very common, and they have roadside battery swap stations where (for a subscription fee?) you can take a freshly charged battery and leave your old one. Seems like a great idea.
Just went through Orlando airport and saw something called "power rod" that is basically that for small power banks: you drop your depleted bank and pick up a fully charged one. If I were a US-only traveller, I can see myself using it.
> - Not sure what they're called, but I've seen a lot of fully automated outdoor "locker stations" for packet deliveries
Drop boxes!
I was part of a team prototyping these some 20 years ago. I highly doubt we were the only team doing so, but we were certainly unaware of any commercially available/deployed stations at the time. I was writing the software, in particular the orchestration of the locks and event bus for the transmissions.
Lots of fun from trying to fathom how undocumented solenoids operated, to trips to various countries for remote and environmental testing, and destructive tests simulating someone driving a truck into an installation (i.e., by deliberately driving a truck into one!)
The nerdiest moment was taking a mainboard model that we were getting intermittent faults with and recreating the exact environmental conditions to recreate the problem. This involved incubating the mainboard in a sealed environment chamber to control temperature, humidity, and atmospheric pressure. The fault was bit-flipping because electrons were jumping rails when the microchip(s) were cold and damp.
Maybe we’re not running the same system version, but I’m quite certain my eyelash to eyeball collision detection works great (and the alert can be anywhere from annoying to quite painful.)
A collision prevention enhancement would be fabulous!
I don't know if they still exist, but for a while when pay phones were disappearing in the US, NYC converted a number of them into free WiFi access points.
I think weirdly, ads embedded in AI search responses actually maybe do have a chance at being helpful (as long as it's clear from the context of the question that I may be willing to pay for a solution) just because they could potentially be quite well matched to the specific thing I want, or if they're not quite as well matched but offer other benefits, explain the difference.
At the moment search ads aren't very helpful because you have neither of those things. You always get them for any type of query, and when you do get them you don't know if the thing being shown will exactly solve your problem, or only approximately, and the work is much more on you to find that out by reading the product's marketing pages further.
If all that could be done for you up front, reasonably honestly, then I could see it being useful. I mean to be sure, in some small percentage of searches I really am looking to buy something and really do want to be usefully, honestly pitched on available options.
interesting thing about the scoring effect narrowing the types of writing that generate the higher average ratings.
I guess also there's something fundamental about next token prediction that is going to narrow in on 'stable' ratings. Let's say for the 4/5 rating you have a wide range of styles, there's also this question of then the style of 4/5 isn't going to be cohesive, so which one do you pick?
Any output that mixes them up is going to get rated lower in later testing, and then if you have somehow to pick one, you've got to stick with it, which seems like it would be complicated to put into next word prediction weights. You'd need a different branch of weights for each style or something.
So not only does the rating process itself push towards a generic style, but the model 'prefers' training on this generic style, i.e. gets to better-seeming results faster.
it's almost certainly not true yet but at some point there might be an equilibrium reached of speed Vs quality (and let's not forget, cost) where it's true for most of what you do.
Perhaps you'd still turn to hosted models for the hardest tasks, but most tasks go local. It does seem like that would make demand go down significantly.
Of course that's all predicated on model advances plateauing, or at least getting increasingly more expensive for incremental improvements, such that local open source models can catch up on that speed/quality/cost curve. But there is a fair amount of evidence that's happening. The models are still getting noticably better, but relative improvement does seem to be slowing, and cost is seemingly only going up.
* local compute isn’t scaling as before, so algorithmic improvements are the only ways models get meaningfully faster and smarter
* all those same algorithmic improvements would also be true for larger models
* hardware manufacturers have an incentive against local LLMs because cloud LLMs are so much more lucrative (+ corps would by desktop variants if they were good enough)
So no it’s not clear quality will ever be comparable. It may be good enough for what you want but there will always be a harder problem that you need to throw more compute and more memory at.
> It may be good enough for what you want but there will always be a harder problem that you need to throw more compute and more memory at.
Sure, but if the “good enough for what you want” consumes the vast majority of cases - data-center ai becomes just for the very extreme edge cases. Like how I can render a 4k rez video game at 60fps on my home pc, but if pixar wants to render their next movie they use data-center compute.
> all those same algorithmic improvements would also be true for larger models
Smaller models run faster. If ten runs of a small model gets me the same quality result as one run of the big model, and the small model runs 10x faster, then they are functionally the same.
Even accepting the premise, it should be obviously true that 10 dumber models running 10x as fast != 1 smarter model. Otherwise engineering would just be a matter of throwing people at a problem when it’s very clear that 1 talented engineer can outperform a team of engineers or accomplish things the team would never have been able to. There’s also the assumption you’re making that a 10x smaller model is 10x dumber when it’s not - it’s a curve and some people seem to struggle with non linear effects
> it should be obviously true that 10 dumber models running 10x as fast != 1 smarter model
If a smaller model tries ten things and comes to the same conclusion as the big model gets first try, then yeah 10x small = 1x big. Is that where we are at now? Idk probably not - but it’s not hard to imagine something like that emerging soon. There is already evidence that smaller models get some things _better_ than bigger models (e.g. https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag... )
> There’s also the assumption you’re making that a 10x smaller model is 10x dumber when it’s not
That is not an assumption i am making. I said “a smaller model” not “a 10x smaller model”. Model speed and model “intelligence” are both non-linear.
I think this instinct is intrinsic, and comes from really caring about detail and wanting to fully understand it and own it.
That's what drives it, and I don't really think the extrinsic things about the way you learned (while helpful) have that much bearing on it. It comes from you and you should take credit for it.
I think if you were learning today you'd probably find have the same feeling and do just fine because of it.
It's basically the idea of the pro of 10 years ago but realised flawlessly, and improved upon actually in most ways, for significantly cheaper.
To do this they had to become an execution machine at huge scale to own and solve every piece that held them back in that pro design, and they did, and it's honestly amazing how it all came together.
reply