KTransformers now supports Kimi K2 for MoE offloading They claim 14 tps for the ...

		chithanh 9 months ago \| parent \| context \| favorite \| on: Kimi K2 is a state-of-the-art mixture-of-experts (... KTransformers now supports Kimi K2 for MoE offloading They claim 14 tps for the 4-bit quant on a single socket system with 600 GB RAM and 14 GB GPU memory.