TrendPulse

Supra Title is live! 🦅 We just released Supra Title (experimental) , a purpose-built 350M model for generating chat conversation titles, built on LFM2.5-350M. https://huggingface.co/SupraLabs/Supra-Title-350M-exp-GGUF https://huggingface.co/SupraLabs Most platforms use large general-purpose models to title conversations. Supra Title does only that, and does it fast, in GGUF format, on any hardware. No system prompt needed. Just send the user message and get a title back. Exam...

When we first started experimenting with local LLMs, it was a completely different story! We were using gaming GPUs to tinker around. 8GB or 16GB of VRAM (which wasn't even a given for everyone) was the norm, and so many people could actually get their hands dirty and experiment. Let’s just forget for a second that long crypto-mining phase that bloated the market and caused shortages... but today? Today, if you don't have high-end hardware, experimenting has become way too difficult. I know ...

I just saw this statement regarding Anthropic being hit with an emergency export control directive from the US government. They were forced to pull the plug on Fable 5 and Mythos 5 for all customers globally. The tl;dr is that the government got spooked by a narrow jailbreak (which basically just sounds like asking the model to fix vulnerabilities in a specific codebase), and forced a complete shutdown without a transparent process. Anthropic is pushing back, but the API access is completely gon...

Benchmarked the new Gemma diffusion model against its autoregressive twin on a single H100 (FP8). We gave each the same three tasks: write a Steve Jobs biography, the history of Tetris, and the story of BeOS - every next topic less popular than the previous one. Then we fact-checked every claim in every answer. Gemma4 got 45 facts right, 5 wrong. DiffusionGemma got 33 right, 28 wrong. The less popular the topic, the worse it got: 4 mistakes on Jobs, 12 on Tetris, 12 on BeOS. It named Clara Cll...

I’m just some fucking guy. This is just some fucking opinion. I’ve seen tons of stealth marketing or related topics on this subreddit about how great or how easy it is to use some random subscription api. Why the fuck are we allowing people to so casually talk about how much more affordable their zai subscription is than Claude? Who cares? I don’t give a singular care if the eastern (bless them for their otherwise great contributions to OSS LLMs) companies can offer 35 trillion tokens for 25 c...

If you don't have it on your own drive, someone is going to take it away, enshittify it, bar you from accessing it, censor it, and hike the prices of it sooner or later.

Minimax m3 weights are out !! It has ~428B parameters and ~23B activated parameters.

Kimi K2.7 Code is a coding-focused agentic model built upon Kimi K2.6. With substantial improvements on real-world long-horizon coding tasks, it strengthens end-to-end task completion across complex software engineering workflows while improving token efficiency, reducing thinking-token usage by approximately 30% compared with Kimi K2.6.

I've been experimenting a bit today with letting models reason for creative tasks, rationale being that it might help with keeping track of details and prompt adherence. And predictably, the wall I'm running into is that they all want to draft, check, refine, revise, "um actually...", and draft again before actually giving the output. This is hugely wasteful no matter the reply length, but it's especially bad if you want more than a paragraph or two. Obviously I'm not the first pers...

Just got 100 tps on generation, but in total time it around 45-60 t/s in case of prompt processing waiting. Available memory show: GPU KV cache size: 152,671 tokens Maximum concurrency for 131,072 tokens per request: 1.16x amd-smi monitor for this gpu: GPU XCP POWER GPU_T MEM_T GFX_CLK GFX% MEM% ENC% DEC% VRAM_USAGE 3 0 183 W 82 °C 84 °C 3036 MHz 100 % 5 % N/A 0 % 23.6/ 24.0 GB 5 0 161 W 81 °C 88 °C 3101 MHz 100 % 0 % N/A 0 % 23.7/ 24.0 GB 7 0 165 W 78 °C 86 °C 3095 MHz 100 % 1 % N/A...

I have a budget of 5K, and want to buy some gpus my requirement is 48gb+ vram, because I finetune small language model, perform DPO, in general tinkering/ development is my usecase. if you where in my shoe which among these would you get, on one hand amd is better bang for buck, faster but has low vram. Nvidia has cuda, but is slow af, and the memory although large, it’s speed is trash, making large model training/inference useless. is there any other machine which i can consider, due to tra...

okay after testing DiffusionGemma a bit more internally we genuinely can’t tell if this is the start of something big or if everyone’s just getting distracted by crazy TPS numbers again lol but one thing that stood out REALLY fast for us was how different the H100 vs A100 behavior felt compared to normal transformer inference on some runs the H100s scaled almost exactly how you’d want them to the A100s were still good, but once concurrency started increasing the gap widened way more than w...

Has anyone personally compared the Minimax M3 model against other proprietary models to determine its relative performance tier? I am trying to understand where it currently ranks in the broader Al landscape. Can we say Minimax M3 is better than GPT 5.2 in coding and agentic task?

First Doom, now DiffusionGwmma 4. We are truly living in the future. Who even needs a new Qwen release anymore? /s (Satire - Shaq doesn’t actually make a digital pregnancy test capable of running diffusion-based LLMs) Credit to Obvious Plant for the original Shaq pregnancy test box (that I doctored slightly).

Posting for a friend who keeps on asking me... "So for LLMs, VRAM speed is king. But what if you bought a used server which had, for example, 80 lanes of pcie 5 available, and you bifurcated that to hold 40 SSDs @ 2x lanes, with each NVMe doing 15Gbps, that means a mirror of 40 2TB drives could potentially do 600Gbps for a 2TB model. Or if you did 80 nvme @ 1x pcie lane each, you'd get 1.2TB/sec. That seems pretty good, right? You could get pretty good speeds across any model size. ...

Bonsai LM (1-bit and 1.58-bitLLMs) benchmark on Jetson Orin Nano Super Just released a deep benchmark of 5 Bonsai LM models (1.7B → ~8B) on a $250 Jetson Orin Nano Super 8GB using llama.cpp CUDA - across all 4 power modes: 7W, 15W, 25W, and MAXN A thread! So, Bonsai LM models are new line of 1-bit LLMs released recently and I was wondering how they perform in terms of TTFT, tok/s, tok/J and overall request latency, with incredibly low memory footprint even for 8B models! Thus, I ran a ...

Hi! I am a student from India, this is my first paper that I published. I was curious whether I can combine both Attention and FFN together to save parameters without sacrificing performance, specifically at parameters <= 10M. Basically my intuition was that Attention is dynamic and smart about which information to mix, but it has no strong non-linearity to actually transform that information. SwiGLU has the strong non-linearity but it's static. Same weights for every input. So instead of...

I built a local voice AI realtime music setup where my ESP32 microcontroller talks to my MacBook over WebSockets. The microcontroller is just a tiny Arduino-based device with a mic and speaker, and the MacBook M4 Pro runs Magenta Realtime 2 locally and streams the audio back to the device. The fun part is that it’s agentic and conversational. So I can tap the ESP32, speak into it, and it uses MLX Whisper to transcribe what I said. Then after detecting VAD, it sends that to a Qwen model, which ...

I've been spending the last couple of weeks building a Wispr Flow clone as an open source project. For context, it is a voice dictation app that lets you type faster, by speaking instead of actually typing. I spent the first week building the basic STT capabilities. One of the coolest features that Wispr Flow has is ASR biasing. Wispr Flow calls it its dictionary. I was able to figure out how to implement that for my project and wanted to share how it was done. What is ASR biasing? ASR ...

I kept wanting to talk to my local models instead of typing, but every voice setup wanted a GPU, shipped my audio to the cloud, or was macOS-only. So I built one that's none of those — and I benchmarked it, so these are real measured numbers, not vibes. One command installs the whole stack and wires it into your agent. Then you just talk. Everything runs on CPU and stays off your GPU (your GPU is busy running the actual LLM): Silero VAD — knows when you start/stop talking, no push-...

I was debugging some code and LLM crashed out: ``` The debug_log config defaults to "debug.json" and creates a FileHandler — which appends by default. That file is a log of everything that happened, never cleared. The June 4 errors in it are historical artifacts from before the fix. Conclusion: Both errors are pre-existing and already fixed. No action needed — the debug log just hasn't been truncated since. If Stream error: litellm.MidStreamFallbackError: litellm.InternalServ...

I am using qwen3-vl:8b and ollama for doing OCR on scans of handwritten letters and it is doing a decent job. Any other models I should know about for this kind of OCR?

Any practical advice, what are you guys using due to budget constraint, I cannot use 32 billion 27billion coding models.

I tried to benchmark Google’s new on-device dictation app (Eloquent) and basically couldn’t. It drops about half of my dictations. tl;dr Full results are 👉 here . Background: Google shipped a new fully‑local dictation app yesterday with proprietary new models , so I was excited to benchmark it against the leading open models (Qwen3‑ASR, NVIDIA Parakeet V3, etc). I have a harness that drives a dictation app by playing an audio file through a virtual input device and captures the app’s...

Two numbers on this model that don't sit comfortably with each other. The Pro config posts coding scores near the top of every board, 80.6 on SWE-bench Verified and 93.5 on LiveCodeBench. Then CAISI ran it across a spread of domains and landed on it being roughly eight months behind the US frontier, around where GPT-5 was. DeepSeek's own framing at launch put it two months back, right behind the frontier at the time. Same weights, very different verdicts. The way I read it, both are right and ...

https://preview.redd.it/2o426zap9l6h1.png?width=1080&format=png&auto=webp&s=169e2d511bbf4c4b08a155775d94b0e9f3f931a5 Claude Fable is incredible It one-shotted my usage limits in 1 prompt

https://huggingface.co/mradermacher/gemma-4-31B-it-qat-q4_0-unquantized-i1-GGUF/tree/main https://huggingface.co/mradermacher/gemma-4-31B-it-qat-q4_0-unquantized-GGUF/tree/main I waited a bit before asking this. I have 3060 12GB and 32GB ddr3 RAM. I'm currently using an old version of unsloth's gemma-4-31B-it-UD-IQ3_XXS.gguf which is 11.8GB. With override ffn_down tensors, I can run 16k bf16 context at about 1.3 tk/s last time I used it. When I use the bf16 mmproj I offload it to CPU. Ove...

Is anyone else having trouble with tool call loops in qwen3.6-27b? I've been messing with the temperature, top-k, etc. parameters for two days, but it doesn't solve the problem. It works up to a certain point, but sometimes it gets stuck in an infinite loop of repeated tool calls.