we're experimenting with our own dynamic GGUF quants of kimi k3, made from the original weights with our llama.cpp fork. Q3_K_S is done and works 1114.76 GiB on disk. Q1 and Q2 are in progress, results on those tomorrow rented box hardware: - AMD EPYC 9554P, 64 cores - 1.5 TB of DDR5 - NVMe in raid0 to store the weights (inference runs fully from ram) - no GPU the run: - 110 threads - pp512: 4.21 t/s we ran a short test for text coherence and image understanding to make sure the qu...