In such discussions, whenever you mention abstractions are universally "pretty poor", to the extent anyone is listening, I think this hyperbole can do real damage. Maybe it prevents people from getting relevant performance gains, even if not 100% of the optimum, which is anyway unattainable. And what is the alternative? Not many projects can afford to hand write intrinsics for all platforms. And are you aware that Highway is basically a thin wrapper over intrinsics, which you can still drop down to where it helps?
I am aware of Highway. It doesn’t add much value for the kind of SIMD code I write. I have better abstractions because I don’t have to consider portability nearly as much. Some useful constructions don’t have a good expression on weaker SIMD architectures.
To be clear, "better abstractions" here seems to mean macros for assembly language. To each their own.
What bothers me is advocating for this, or denigrating more generally useful alternatives, without mentioning the very narrow niche where this sits.
Video codecs only change every few years. This makes it more worthwhile/feasible to spend eng time on a few kernels.
Even then, not supporting SVE (you don't, right?) gives less incentive for the Arm CPU ecosystem to invest in it, helping keeping us stuck in the NEON local minimum. Not ideal :/
:) I figure there is always something left to improve. For some kernels which really want to keep 30+ live registers, the compiler might not do as good a job as careful manual tuning, so intrinsics can have a bit of a cost. But I also figure optimization time is limited, so better to get 90% of several kernels rather than one to 99%.
Not who you asked but I think the meaning is that since intrinsics for simd are different in each platform, being able to have something that is portable and sometimes works faster is something, while writing for Intel, ARM and a zoo of instruction sets is not an option for some.
Besides Spolsky's law of leaky abstractions, "abstractions" can also result in "lowest common denominator" situations, which are the opposite of performance optimization. Talking negatively about abstractions is not what deals damage; you are shooting the messenger here. It's the abstractions themselves that deal damage when misplaced. "Zero-cost abstractions" is the true hyperbole.
Is this a good faith reply? The particular abstraction we built, and is being discussed, is manifestly and obviously not a lowest common denominator.
Looks like you are deploying a second straw man, that of zero cost. In other comments here I acknowledge a cost to intrinsics.
This works today :) Highway provides such an abstraction for arbitrary vector lengths and maps them to intrinsics. All on the library level, no need to wait years for compiler or language updates.
Have you considered our Highway library? Runtime dispatch need not be a PITA :) It's basically portable intrinsics, and a much more complete set (>300) than the ~50 in std.
I hadn't but it would make sense for doing my own personal programming challenges.
Given the ongoing disasters around the software supply chains I've been fighting the creeping NPM-ism that people are trying to introduce to C++, where you just FetchContent 20 different libraries to build your own app upon.
I do use gtest, fmt and a few others though, so something as broadly used as Highway would probably be fine by that standard as well. But I'd still like it better if there was a Good Enough solution that was part of C++ stdlib to reduce the number of external integrations that are deemed required for a modern C++ program.
Fair point. If it helps, our security team has called Highway critical infrastructure and helped to harden the repo.
The flip side of standardization is that it would be much harder and slower to add ops as the need arises, which we do regularly.
Yes, the EMU128 target is scalar only, with for loops. This is a fun way to see how well autovectorization works, with the same source code.
That works on any CPU. Curious which projects have such concerns, any link?
People reported challenges building V8 (whether upstream or the Node.js variant) on s390x with z13 support. I don't know if it was discussed on the porters mailing list because it's not public: https://groups.google.com/g/v8-s390-ports
Thanks for sharing. The first link seems non public indeed.
I can imagine there is some compile issue we could reasonably fix, with the help of someone who has Z13 access. Please encourage them to raise an issue. I will be back on May 26.
After that, it should at least be able to use the scalar fallback.
The issue with Z14 is that it lacks fp32 support. Would their usage be integer only?
I'll bring it up with some folks. It probably won't change much because the z13 transition has finished by now. It's still good to know because RISC-V is in the same boat regarding Highway support today: we need scalar fallback in Highway until we get RVA23 hardware deployed.
Any suggestions for improvement? We went through >5 iterations of the dispatching and I am fairly confident this is about as good as it gets in current C++.
I suppose "macro hell" is a matter of taste. Objectively, we have six dispatch related macros in the example: https://gcc.godbolt.org/z/KM3ben7E
The ~two dozen lines of boilerplate are generally copied from an example.
But why multi-file?
Highway TL here. I agree with the main points, with a few clarifications:
> tag-dispatched free functions like hn::Mul(d, a, b)
We only require tags for certain ops, mainly memory, casts and reduction; not arithmetic. Operator overloading is supported but until recently compilers didn't allow that for SVE vectors.
> It’s a Google project with Google-scale maintenance, but the bus factor is real — the core development is driven by a small team
We have 101 contributors, including 14 current or former Googlers in several teams.
> being length-agnostic means you can’t easily express fixed-width algorithms that depend on knowing the vector size at compile time, which is common in cryptography and codec work
We explicitly support fixed-length 128-bit vectors, acknowledging that these are common and important.
(Personal opinion)
I get the impression that RISC-V-related discussions often lack of awareness of prior work/alternatives. A large amount of (x86) software actually uses our Highway library to run on whatever size vectors and instructions the CPU offers.
This works quite well in practice. As to leaving performance on the table, it seems RVV has some egregious performance differences/cliffs. For example, should we use vrgather (with what LMUL), or interesting workarounds such as widening+slide1, to implement a basic operation such as interleaving two vectors?
> For example, should we use vrgather (with what LMUL), or interesting workarounds such as widening+slide1, to implement a basic operation such as interleaving two vectors?
Use Zvzip, in the mean time:
zip: vwmaccu.vx(vwaddu.vv(a, b), -1, b), or segmented load/store when you are touching memory anyways
unzip: vsnrl
trn1/trn2: masked vslide1up/vslide1down with even/odd mask
The only thing base RVV does bad in those is register to register zip, which takes twice as many instructions as other ISAs. Zvzip gives you dedicated instructions of the above.
Looks like the ratification plan for Zvzip is November. So maybe 3y until HW is actually usable?
That's a neat trick with wmacc, congrats. But still, half the speed for quite a fundamental operation that has been heavily used in other ISAs for 20+ years :(
Great that you did a gap analysis [1]. I'm curious if one of the inputs for that was the list of Highway ops [2]?
Gemma.cpp has nested thread pools, one per chiplet, and one across all chiplets.
With such core counts it is quite important to minimize any kind of sharing, even RMW atomics.
