LocalLLaMA
Kimi K3 for local use (1.56TB → 594GB) compressed and released by Unsloth
The model was quantized to 8, 4, 2, and 1 bit. Characteristics: Q8 : 8-bit 1.56 TB, lossless Q4 : 4-bit, 1.51 TB Q2 : 2-bit: 861 GB Q1 : 1-bit, 594 GB The smallest Q1 model keeps 78.9% accuracy, while being almost 3 times smaller than the original one. Instruction for running the model is in the model's card submitted by /u/BankApprehensive7612 [link] [comments]
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit
Related Trends in LocalLLaMA
Building an open-source control plane for self-hosted vLLM, what would you want in it?
5090: Windows or Linux for Qwen3.8.27b
RTX Pro 4500 vs. A100 (x4) - purchase being made today
Best llm at understaning middle eastern languages? Arabic/Persian etc
Qwen3.8-27B abliterated FP8: refusal 64–99% → 0–6%, and MMLU/GSM8K move less than 1.3 points
llama.cpp Windows Manager