If you believe a 128gb machine that is essentially DGX Spark in a laptop chassis can run models comparable to SOTA you either never ran open models on hard tasks, or you aren't scratching the surface of SOTA closed LLM capability in how you're using them.
Can you show me an example of a hard task that can't be achieved using light models? When we don't want the model to work on autopilot without reviewing the code at all. Even SOTA models will produce garbage code, if you don't guide them all the time.
Hard tasks require a lot of guidance and code reviewing, unless you are creating another throw away project where correctness, maintainability and code understanding does not matter.
Boy pricing is pretty nuts these days. I have half a petabyte in Seagate enterprise drives myself and I didn’t pay anything close to that to acquire it. Such a pity about the flash storage. 2 years ago we built 200 TiB or something of flash using Samsung PM1633 or something and it was a fraction of the cost per gigabyte that $1m would imply.
So, I'm not really a mathematician, but the first 3-8 pages reads like nonsense and a bunch of unrelated facts. A bit surreal may be, but if this the norm for this kind of thing, I'm amazed it arrives at any useful result at all.
It didn't seem like nonsense to me. (Recently graduated undergrad with a math degree; probably could have gone to PhD). It seemed like the AI was cycling through a bunch of different possible approaches to tackle the problem. Eventually it finds one and makes more progress there until it reaches the solution
I'm disappointed only that the chain of thought needed to be rewritten. Need to train these LLMs to natively communicate in LaTeX research paper format.
I believe they rewrite the chain of thought to protect their IP, i.e. the chain of thought reveals information about how the model works in a manner that may aid replication
Cargo is spiritually based on NPM so it's not much better.
Go Get is closer to always locking dependencies unless you explicitly upgrade them with a go get, so it's much much better in my view.
Yes, you can lock deps in NPM/Cargo/etc. but that's not the default. It is the default in Go.
In Go projects my policy for upgrading dependencies includes running full AI audit of all code changed across all dependencies, comes out to ~$200 in tokens every time but it gives those warm 'not likely to get pwned' vibes. And it comes with a nice report of likely breaking changes etc.
BTW a curated mirror of <whatever ecosystem> packages, where every package is guaranteed to have been analyzed and tested, could be an easy sell now. Also relatively easy to create, with the help of AI. A $200 every time is less pleasant than, say, $100/mo for the entire org.
Docker does something vaguely similar for Docker images, for free though.
People are already scanning npm constantly. You can limit yourself to pre-scanned packages by setting npm's minimum release age setting to 1 or 2 days (a timeframe that all the recent high-profile malicious package versions were unpublished within).
Note to self: the test suite for vetting a package should include setting the system date some time in the future, to check if an exploit is trying to sleep long enough to defeat the age limit.
It's insane to me you spend $200 on a report you likely rarely read in detail or double check for correctness, yet you're doing it to feel good about security.
If it runs in a harness that will alert me when something dodgy is detected I'm fine to stay at that level.
I don't read it in detail because reading in detail is precisely what I delegate to the harness. The alternative is that I delegate all this trust to package managers and the maintainers which quite clearly is a bad idea.
Whether the $$ pricetag is worth it is.. relative. Also in Go you don't update all that often, really when something either breaks or there is a legitimate security reason to do so, which in deep systems software is quite infrequent.
Funnily enough for frontend NPM code our policy was to never ever upgrade and run with locked dependencies, running few years old JS deps. For internal dashboards it was perfectly fine, never missed a feature and never had a supply chain close call.
What do you when a critical vulnerability gets discovered and you have to update a package? How many critical/high severity vulnerabilities are you running with in production every day to avoid supply chain attacks?
For the stuff in more sensitive deployments it's really quite simple, just setup CORS etc properly and don't do anything overly fancy on the frontend. Worst case the user may force some internal function to eval some JS by pasting scripts into the browsers debug console.
Critical severity vulnerabilities are only critical when they are reachable, but are completely meaningless if your application doesn't touch that code at all. It's objectively more risky to "patch" those by updating dependencies than just let them be there.
It is the default in both cargo and npm, but "npm install" stupidly enough still updates the lockfile, and you need "npm ci" to actually respect it. I think there's some flag to make install work sanely, but long-term I find the best approach is to use anything other than npm.
I ditched npm for yarn years ago because it had saner dependency resolution (npm's peer dependency algorithm was a constantly moving target), and now I've switched from yarn to bun because it doesn't run hooks in dependencies by default. It also helps that it installs dependencies 10x faster.
There was once a 2.5" SSD Mushkin Source 16TB SATA drive. At its cheapest it was ~1700 USD (or 1500 EUR). That was mid 2023 (like 3 years ago!).
Nowadays it feels like that this time and price region is like decades away in the future. I was hoping I can store more data in future on modern tech like SSDs and not less.
Yeah it sucks :( Almost exactly a year ago, I got a brand new 15.36TB Kioxia CD-6R (u.3 pcie4x4 drive) for $1450+tax from serverpartdeals.com - that same drive is now listed for ~$4600 (and it’s also out of stock there)
reply