TrendPulse

I have been playing around with LLMs on a dual 5060ti (Windows) rig, and now want to change things up. I built a separate dual 5070ti (Debian) rig and now have that running Qwen 3.6 27b UD Q6 MTP @ 100k context, without any multimodal capacity. That's solid for the text processing and generation stuff I want to do right now (which is mainly IT support tickets triage and PowerShell scripting via OpenCode as the "harness"). This now leaves my dual 5060ti rig ready for repurposing. I'...

Compiled by u/ vramkickedin EDIT : Wrong automatic thumbnail for the link(Check the link for long list)

I've picked up a 3-bit quant of this one (Unsloth - Q3_KS) - the best fit for my config right now. Normally I shy away from 3-bit quants but as this is a giant dense model, I figured... why not. I've tested it a few hours in my latest project, and it found a few things that my daily driver had missed. It seems really good. Slower than an MoE, but not bad for me (8 tok/sec - with KV 80k - quant K = q8_0 quant v = q5_0). I was wondering what other people thought after actually using it with co...

Hello. So, first, ill probably explain the context, and why we (well, I kinda have a team, so ill say we) want to build something like that. We live in Russia. There are many preconditions, that point to the fact that pretty soon, we'll get our internet access (global internet, apparently) completely cut off. And we are preparing to go fully offline for indefinite period. I know, this sounds weird. No one believes that. But I prefer to be ready for the worst case scenario. ...and we kinda wan...

Building a personal rig mainly for running coding LLMs locally (inference,maybe light fine-tuning). Already have the motherboard/rest of the platform sorted — just deciding on the GPU. Three options I keep coming back to: Modded RTX 4090 48GB (Chinese clamshell mod) — I have an eBay offer at $3,500. 48GB GDDR6X, full AD102, ~1TB/s bandwidth, and obviously CUDA. The catch: third-party firmware, no real warranty, blower cooler, and general "is this thing reliable long-term" n...

When building long-running coding agents, terminal output is one of the fastest ways to poison a context window. If an agent runs an intensive build, an install command, or a massive search ( grep / find ), it easily generates hundreds of lines of raw log noise. The agent reads it once to check the status, but you continue to pay for those massive, redundant log tokens on every subsequent turn in the conversation loop. To fix this, we designed a lightweight CLI layer called Boost to act as...

I was curious whether any of the FA-3/4 optimizations transfer to RTX GPUs. vLLM/SGLang attention falls back to FA-2 on consumer cards (FA-3 and FA-4 are datacenter-only), so I wanted to know if there's any performance left on the table, and I rebuilt the attention kernels from scratch. The kernel reaches parity with FA-2 (206us on RTX5090 with batch=1, heads=8, seq_len=4096, head_dim=64), but unfortunately, FA-3/4 optimizations are either not applicable or not helpful on consumer cards. It lo...

Hey Guys, As promised here are the results from running MiniMax M2.7 REAP 139B Q3_K_L on llama-bench on 6x MI50's. Memory Load: https://preview.redd.it/7tp1nhrl79ch1.png?width=1628&format=png&auto=webp&s=ada4c100e1b589ab530e5b3e7b64153533a8a3de Hardware: Asus X99-E-WS ( Modded BIOS to support a large number GPU's ) Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz 128GB DDR4 RAM SSD 6x MI50's 96GB VRAM (Gen3 x8,x8,x8,x8,x8,x8) Results GPU Setup Model Test Resu...

https://github.com/ggml-org/llama.cpp/commit/ccb0c3422394fbbfc28fd91f8c77111b748cfa09 It seems to be time for another llama.cpp rebuild, at least if you are on amds ROCm/HIP. There are no benchmarks included and i am still building, so it would be nice if any of you could report back on the performance changes.

I've grown very frustrated with OpenCode. The web GUI and desktop app ideas are good, but the execution not so much. The GUI is lacking so many basic features. It's clear that the TUI is more important to the devs. Is there anything free that provides a more feature-rich GUI? Having it run in a server is really great for me since I'd like to leave long running jobs. I also use Hermes, but I don't like it for coding.

Maintainer here. OpenMed is an Apache-2.0 toolkit for clinical NLP with one hard rule: patient data never leaves your hardware. No cloud calls, no API keys, works in airplane mode. What shipped in 1.8 this week: OpenMedKit for Android (Kotlin, ONNX Runtime Mobile + ML Kit OCR): read a document, strip every name/MRN/date, entirely on the phone. iOS/Swift and React Native bridges landed too. Browser runtime : de-identification via Transformers.js / ONNX Runtime Web with wasm + WebGPU ba...

MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness. Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as [S01] , [S02] , and beyond. Introduction MOSS-Transcribe-Diarize 0.9B turns real-world long-form audio into structured, speaker-aware transcripts in one pass. Instead...

https://preview.redd.it/ah7aif6od7ch1.png?width=3813&format=png&auto=webp&s=32bdca0eb64d36e1a1097473ed0689b2947610bb I made this simple 3D Geometry Wars-style game using my coding agent, Jarvis Code, with GLM 5.2. You can play it here: https://jarvis-llm-codec.github.io/jarvis-code/geometry-wars-3d.html I was honestly surprised by the result. Most of the game came together in the first iteration, and I only needed about four small follow-up tweaks afterward. I'd be curiou...

I am curious about intended sizings of the main size niches of the popular local LLM models. As in, we can see there is a major niche at 26b-35b, then hardly anything from 36 through 69b, then (formerly) another major niche at ~70b-72b, then another niche at ~120b-123b, then another big gap till ~230b-235b, and then it gets a bit more mixed all over the place after that with 300b-750b being scattered more randomly probably based more on just whatever the best strength per size they could get w...

I just pushed a new audio.cpp update with streaming support and 4 ASR/STT models: Nemotron 3.5 ASR, Higgs Audio STT, VibeVoice ASR, and Hviske ASR (da only). Overall 1.07x to 2.41x faster than Python. I decided to drop Parakeet-TDT since good implementations already exist, and I find the Nemotron model more interesting. Streaming support is also starting to land in audio.cpp. Right now, I added initial streaming support for two models that naturally fit streaming usage: Nemotron ASR and ...

TL;DR: Full GLM-5.2 (753B MoE) quantized to Int4-Int8Mix + NVFP4 4-bit KV cache, TP=4 across 4× DGX Spark (GB10) at 100K context , run on Terminal-Bench 2.1 with the same agent scaffold (Terminus-2) as the official numbers. Result: 63/89 = 70.8% vs the official full-precision 81.0% . Caveat up front: I never ran the full model through my pipeline — the ~10-pt gap bundles quantization plus my 100K-vs-256K context cap, a smaller token budget, and unmatched sampling. So read it as: my wh...

Hey Everyone, here is an update on MTPLX! One month after releasing MTPLX V1 which brought a swift based app and upgraded CLI for coding use I am happy to announce MTPLX V2. The biggest change is Turbo Mode: using custom verify-specialized quantized-matmul kernels plus a compiled verify step we have achieved 82 TPS on a Macbook pro m5 max at a temperature of 0.6 We also released significant changes to SSD KV cache and long context tool calling improvements. here are the preliminary be...

TLDR: 75B-total / 9B-active MoE is the perfect shape for multi-24GB rigs, and almost nobody ships it. Qwen 27B is a great model and punches way above its weight-class, it is a frequent fallback for me. Nemotron-3-Puzzle-75B-A9B, NVFP4, vLLM 0.22.1 (the new Marlin fallbacks run FP4 on Ampere), pipeline-parallel across 3×3090 capped at 200W each. The 4th card runs a speech sidecar untouched - 3 seats × 256K ctx, fp8 KV — hybrid Mamba keeps the cache tiny - 132 t/s decode across 3 streams (~6...

https://artificialanalysis.ai/evaluations/artificial-analysis-openness-index In case you want to support openness, some models are more open than others. Update: K2 think v2 is rated highest because it supplies its training data and training regimen. This allows anyone with enough resources to recreate the model. Deep seek doesn't publish how it trained its model or the training data, so it gets a lower score. If we try to compare software to LLMs. One level of software is that they s...

I’m using Opencode and a computer with 128gb. So maybe the results would be different on system. I’ve exhaustingly tried Qwen3.6 27B and Qwen3.6 33B. I have no idea why but they just fall apart when doing more complex tasks with many tool calls. They’re pretty aggressive, doing slightly more than asked, and end up digging themselves into problems. Gemma4 31B and the 26B are literally the opposite. They can’t simply get things done. I have to sit there babysitting them just saying ok, ok, o...

I don't know where this is headed, but I don't like it. https://futurism.com/artificial-intelligence/open-source-ai-model-scary-mythos GLM-5.2 can be downloaded by anybody, can be run on virtually any hardware, and unlike Mythos or Fable, there’s no vendor playing the middle man between the AI models and the users, raising the cybersecurity stakes considerably. Put simply, while these frontier models can aid researchers in patching holes in commonly used software, the can also be abuse...

Open source AI is a shared asset for all humanity. Chinese open source models such as DeepSeek and Qwen have significantly lowered the barriers and costs of AI adoption. China is committed to further promoting open source AI for industry, academia and research institutions, encouraging innovation, AI empowerment and an inclusive ecosystem through international cooperation, thereby injecting sustained momentum into AI development. [link] [...

Samsung chip division's single-year profits beat its past 40 years of profits, combined, due to increased memory and storage prices — Samsung passes Nvidia to become most profitable company in the world, notches 19x quarterly increase in profit https://www.tomshardware.com/tech-industry/samsungs-chip-division-expects-to-out-earn-its-entire-40-year-history-in-2026

Hello guys. Sharing my experience with deploying DS-4-Falsh with DSpark on HGX-H200 For context of my setups etc (including how the hell i have access to H200), you can read my previous posts So mostly, its about my comparison of marlin moe_backend, that i used on my previous main deploy config with EAGLE (1-1-2), and flashinfer i saw in official blog ( https://www.lmsys.org/blog/2026-07-06-dspark-sglang ). In general i can say, that DSpark absolutely faster then EAGLE . Didnt save ex...

I'm glad to announce that we have completely revamped the site, so that you can now search the over 2000 viewable models instantly! You can also log in with your Hugging Face account to write cool interactive articles that link between hovered layers mentioned in the text to where they appear in the graph! Thanks for all the suggestions you gave last time! It's been so fun to see the community keep growing! You are also very welcome to join the #hfviewer discord if you haven't! https...

Just finished reading the paper: LLM Inference at the Edge: Mobile, NPU, and GPU Performance Efficiency Trade-offs Under Sustained Load I am starting to benchmark LLMs on edge devices, particularly phones thus been reading a lot on the what has been done and what is currently being done and wanted to share you my journey of reading such papers and my takes on them. This is one of the only papers I have read that have benchmarked RPi5-Hailo (Hailo's 10H) iPhone 16 Pro (A19 Pro) S...

TP4+DCP2 for a ~360k kV pool. Prefill increases to 900-1000 t/s with longer prompts. You can also run DCP4 for 660k, but prefill gets shaved to ~400. Dropping DCP raises prefil to ~750. I'm running 4 drafted tokens vs Z.ai's rec of 5. Decode is heavily dependent on prose. Thinking gets ~20 tok/s. Code gets 25-35. Typical turns in Pi get me ~24 tok/s. Pruning the model by 5-10% will probably get you to 1M ctx or more concurrency if you need that. In my daily use, a 10% data-free prune seems t...

This PR adds Q2_0 support for CPU. Main motivation is to support Ternary Bonsai models (1.7B, 4B, 8B) and upcoming models. This PR is CPU only (ARM NEON + generic scalar fallback). This completes the Q1_0, Q2_0, Q4_0, Q8_0 family. We have the x86, Metal , CUDA, and Vulkan backends ready to submit later. https://huggingface.co/collections/prism-ml/ternary-bonsai