TrendPulse

i built this on top of my own stack but the code is MIT for everything related. kokoro was pretty fun to explore, i'll likely build something similar for other models. if you have a particular preference, let me know and i'll take a look at it. the specific kokoro code i wrote to enable this is here: https://github.com/wlejon/brosoundml the models, including the bridge model i trained are here: https://huggingface.co/datasets/wlejon/brosoundml-data if you like it enough to want to try...

Hello everyone I wanted to share what I've been working on. I started writing NVFP4 kernels for llama.cpp last year and needed the ability to quantize NVFP4 GGUFs, so this project started as an NVFP4 quantizer. It's since become much larger. I would love to get more help to improve it. This is what I call the advanced-quantizer-tool (MIT license). This is used to create NVFP4 and MXFP6 models into GGUFs directly. But it can do much more. The latest model I've made with it are here: Qwen...

https://huggingface.co/nex-agi/Nex-N2-mini

This is not a diss to Unsloth, they make great quants and really move this community forward. I've been experimenting with quanting specific sublayers based on which ones have the most outliers post Q8 quant. I basically did a BF16 to Q8_0 conversion and looked at the post quant values to compare. I found several layers that had a CRAZY high number of outliers. I'm not certain this is better, but the results are interesting! I still need to upload the Q8 quant to hugging face, but here are s...

The Unsloth Q5_K_XL is officially my main squeeze for local coding. I started out with the Q4_K_XL, but found myself fixing syntax errors a little too often. It wasn't terrible, but I had one file where I had to make 23 edits just for syntax. With the Q4 I was pulling around 61 t/s, and moving to the Q5 dropped me down to 50 t/s, but now most things get one-shotted (not zero-shot, I still had to tell this baby what to build *wink*, looking at you grammar/tech Nazis). The model file sits righ...

I’m posting this as a warning for anyone building multi-GPU local LLM rigs with older workstation/HEDT boards. My setup (Node #04) Gigabyte X399 Designare EX Threadripper 1950X 128GB DDR4 4x RTX 3090 10GbE TP-Link/Aquantia NIC llama.cpp NCCL build vLLM for safetensors models I was getting weirdly disappointing multi-GPU results. The rig worked, all 4 GPUs were detected, VRAM was available, models loaded, but some workloads were underwhelming. Example: Mistral Medium 3.5 1...

BeeLlama v0.3.0 and v0.3.1 are here! Big architectural update to align the fork with upstream llama.cpp and integrate all its additions like MTP and Gemma 4 12B support, while also updating DFlash to handle complex configurations like multi-slot and multi-GPU. Now also recommended by club-3090 ! Thanks to noonghunna for inviting Bee to the club and for their help with testing v0.3.0 on a multi-GPU setup. Not quite a pegasus, but close enough. GitHub | Qwen 3.6 27B Quick Sta...

"We’re going to have the most beautiful laptops, they’ll be the slimmest laptops ever. A total masterpiece, look at that green chip. Unbelievably powerful. They’ll be so slim you won’t even see them from the side…believe me…it’s true. A lot of people are saying it. It’s not like those big, clumsy, failed laptops that Sleepy Joe makes. Total losers. We only make the best. And did you hear about my new ballroom, it’s gonna be the most beautiful ballroom..." submitted by ...

What are you doing AMD & Intel? NVIDIA just released a 550B model after so many tiny/small/medium/big models. Models are becoming(or already?) the commodity for NVIDIA. https://huggingface.co/nvidia/models?sort=created https://huggingface.co/amd/models?sort=created https://huggingface.co/Intel/models?sort=created

Took a while, but Nalthis is finally up and assembled. Specs: Supermicro H13SSL-N AMD EPYC 9575F (64C/128T Zen 5) 768GB DDR5-5600 ECC RDIMM 4× RTX 3090 (96GB VRAM total) 1× 2TB NVMe OS 2× 3.94TB NVMe data 2050W ATX 3.1 PSU Corsair 9000D Planned use: vLLM - high throughput small models llamacpp - larger reasoning models I have been making a space simulation and finally ready to integrate AI into how the NPCs doing planning, hoping to get decent throughput on smal...

It seems like this comment has gone widely unnoticed. https://old.reddit.com/r/LocalLLaMA/comments/1tvtn6m/googlegemma412b_hugging_face/opjj681/ Maybe hold off on testing quantization and wait for it's refinements. The account is Omar from the gemma team.

WARNING: I'm speed typing this, no time to organizea/format, so if short paragraph chunks bother you, just keep it moving. CONTEXT UPDATE: (for those interested, otherwise skip) For those interested in the data points, the task was building an agentic workflow inside of rivet that included an mcp subgraph (with a list of 11 tools) that received json instructions from the main subgraph so that I could shave off 30K tokens from the main agent's memory. The main subgraph included context...

Who needs programming anyway?

Model Summary Total Parameters 550B (55B active) Architecture LatentMoE - Mamba-2 + MoE + Attention hybrid with Multi-Token Prediction (MTP) Context Length Up to 1M tokens Minimum GPU Requirement 8x GB200/B200/GB300/B300, 16x H100, 8x H200 Supported Languages English, French, Spanish, Italian, German, Japanese, Korean, Hindi, Korean, Brazilian Portuguese, and Chinese Best For Frontier reasoning, complex agentic workflows, long-conte...

The KV-cache quant race just got more interesting. Huawei just open-sourced KVarN , a KV-cache quantization method under Apache 2.0, drops into vLLM with one flag. Posting because the tradeoff it's claiming is genuinely different from what's already in the stack, and I'd like to see it stress-tested. The landscape it's stepping into FP8 ( --kv-cache-dtype fp8 ) is the current default: ~2x KV capacity, BF16-level throughput, near-zero quality loss. Hard to beat, and the bar anything ne...

3 different accounts, some even with LinkedIn Gold, made the above posts all on the same day. And clearly all of them followed the marketing team's pointers without even understanding how locally hosted AI works, no way a $249 8GB machine can replace frontier models.