LocalLLaMA
OSCAR RotationZoo - Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
Previously I did post a thread on this. Now with some more details. GGUF downloads: Gemma-4-12B-it: https://huggingface.co/Zhongzhu/OSCAR-LLAMACPP-Gemma-4-12B-it-INT2-KV Qwen3-32B: https://huggingface.co/Zhongzhu/OSCAR-LLAMACPP-Qwen3-32B-INT2-KV Qwen3-4B-Thinking-2507: https://huggingface.co/Zhongzhu/OSCAR-LLAMACPP-Qwen3-4B-Thinking-2507-INT2-KV Code: llamacpp: https://github.com/FutureMLS-Lab/OSCAR/tree/zhongzhu/llamacpp sglang: https://github.com/FutureMLS-Lab/OSCAR/tree/z...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit