TrendPulse

Model Overview Description: DiffusionGemma 26B A4B IT is an open-weights multimodal generative model developed by Google DeepMind that processes text, image, and video inputs to produce text output via discrete diffusion. Built on the Gemma 4 26B A4B Mixture-of-Experts (MoE) architecture with 25.2B total parameters and 3.8B active parameters, the model employs an encoder-decoder design with bidirectional attention that generates tokens in parallel 256-token blocks, enabling high-speed gene...

https://wccftech.com/amd-unified-memory-architectures-open-up-a-world-of-possibilities-shape-product-roadmaps/ Quote: AMD believes that UMA will help shape its next-gen architectures Article mentions Ryzen AI MAX 400 series, which we might recognize better as Gorgon Halo systems. Previous discussions on this topic: https://www.reddit.com/r/LocalLLaMA/comments/1swiylm/comparison_of_upcoming_x86_unified_memory_systems/ https://www.reddit.com/r/LocalLLaMA/comments/1oph7jd/unified_me...

Hey everyone, I'm running Qwen3.6-MTP-27B-MTP (Q4_K_M) with llama.cpp server on a Tesla V100 , and I'm currently getting around 55 tokens/sec . I'm trying to find out whether there are any configuration changes that could increase throughput further without reducing output quality . 55 TPS seems lower than I expected for MTP on a V100, but I may be missing something obvious. Current command: llama-server \ -m ../NewModels/Qwen3.6-MTP-27B-Q4_K_M.gguf \ --port 9932 \ --host 0.0...

hello everyone! im looking for the most powerful open-source coding ai while still fitting my system my specs: CPU: AMD ryzen 7 7700 GPU: RTX 5070 RAM: 32 gb DDR5 OS: windows 11 use case: Writing, Coding, debugging. any recommendations would be great. thanks in advance

I see a lot of conflict comments on this sub and elsewhere on how useful is qwopus compared to for example unsloth quants of qwen3.6 27b. Some say it’s worse some say it’s much better. I tried it and I notice no differences in some of my tests. But maybe because my tests aren’t complex enough. I am only talking about coding. For those who heavily use agentic coding what did you find?

They said in spring they're gonna bake or hardcode a mid tier Llm into their chip. Do anyone have any kind of updates like what model they gonna use or what's the expected time or chip pricing

Another day, another MTP speedup

it's the same U1-8B-MoT base with an extended MT (multi-task) training phase focused on structured visual output. the benchmark jumps are significant: IGenBench I-ACC (infographic accuracy) : 4.2👉17.0 (4x) Chart Understanding: 51.3👉69.5Text Rendering: 39.8👉46.6Overall Aesthetic: 53.8👉53.3 Repo: https://github.com/OpenSenseNova/SenseNova-U1github (infographic model docs): https://github.com/OpenSenseNova/SenseNova-U1/blob/main/docs/u1\_infographic\_model.md submitted by /u...

Today's v10.7 release is the start of an exciting new chapter for the Lemonade project, so I thought I should share an project-level update. Lemonade's roadmap and development is now driven by 6 working groups , 4 of which are led by non-AMDers. Here are highlights from 3 of the groups in the v10.7 release, which had 19 contributors. Local Omni Models True omni-modal chat, including image gen/editing, by seamlessly combining multiple backends and models. v10.7 makes these LMX-Omni vir...

Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate t...

Another open weight model got dropped today, this one's from DeepMind, seems like a good day for the OSS geeks. Released under Apache 2.0 Instead of generating text sequentially token-by-token like almost every autoregressive model on the market, it uses a text diffusion head. - Throws a 256-token "canvas" of random placeholder noise onto the screen. - Uses Uniform State Diffusion to iteratively refine and denoise the entire block of text all at once. - Because every token ca...

I have enough RAM+VRAM to use gemma4 26b a4b up to q6_k quantizations w/ decent performance. Does anyone have any comparisons of the Q4_0 QATs (at 4-bits/wt) vs non-QATs at >4 bits/wt? (ex: q6_K)? KLD vs the originals wouldn't be appropriate IIUC.

What are the best models, quants and llama.cpp versions/forks for CPU inference these days? I have AVX2 but no AVX512 - Intel core ultra 7 165H; 64G RAM This seems to ask for massive MoE (a lot of RAM, not a lot of bandwidth/compute). So Qwen3.6 35B A3B Q4_K_M with standard llama.cpp produces about 10 tps - usable in non-thinking mode, not usable in thinking mode. Is this the best I can get or are there other options? [link] [comme...

Microsoft's Surface with the crappy old Nvidia chip won't keep up with anything from Apple, but Microsoft wouldn't be on board if Nvidia didn't have a roadmap for more and better laptop chips. And Apple can crash the market on a whim by just announcing a line of products that are local-first. Every WWDC video being about local AI (not literally, there are just a lot) and the Github repo full of specs and benchmarks for every company's local AI should have already done it, but finance people ar...

I have set up a public discord bot instance of OpenLumara on openlumara's official discord server (get the server link here https://www.reddit.com/r/LocalLLaMA/comments/1txxgpq/openlumara_a_different_kind_of_ai_agent_written/ or on the github's discussion page) It's running on local models. You have a variety of choices, including an abliterated model that won't hesitate to do whatever you want. Prompt engineering won't get you anywhere, though! Most modules are enabled, and i've set them ...

https://preview.redd.it/68n8w6vcyf6h1.png?width=2047&format=png&auto=webp&s=bcad4afed8739b82acee4d9d3de5fd45ae0855bb https://huggingface.co/MooreThreads/MusaCoder-27B http://arxiv.org/abs/2606.04847

Small: 30 billion parameters, 3B active. Efficient: Benchmarks to 33.4 on the Artificial Analysis Coding Index, competitive among similar sized models. Open Source: Apache 2.0 license HF: https://huggingface.co/CohereLabs/North-Mini-Code-1.0

I'm trying to use Gemma 4 12B — the new encoder-free unified model (audio/vision/text in one) — for a one-pass audio → response voice assistant: feed the recorded WAV + system prompt and get the reply back as text directly, collapsing the separate ASR + LLM steps into a single model (TTS still happens afterward). Works great with a minimal prompt — the model clearly hears and responds to the audio. But once the text prompt gets large/dense (mine is ~21k tokens: detailed instructions + ...

vibe coding meaning 1: Thrown together without care, by dumping it all on the AI, without deeper understanding of, or interest in, how to make code good, modular, robust. vibe coding meaning 2: Significant AI assistance in writing code. Suspicion: When Mr vibecoding himself, Andrej Karpathy, "vibe codes" something, it is very much vibecoded 2, but not very much NOT vibecoded 1. (Unless maybe it's something he just wants to use once, and throw away immediately aftere.) I don't ...

Long time lurker, and I say this as someone who genuinely loves this community and runs many local models myself. I’ve been using LLMs since the early GPT and LLaMA days. Obviously, models have come a unbelievably long way. Local/open models today are dramatically better than what we had a even a few months ago. But I also think the community has developed a strange habit of wildly overstating how close these models are to frontier closed models. We now have very large open models from Deep...

Here are some graphs for the Local LLMs releases, it's strange except for the last month, i thought that this year was very heavy in terms of release, but is seems that the peak was last year. Maybe the hype about the quality improvement this year made it seems that it was richer than last year.

Hey everyone. I'm brand new to running LLMs in general, even more new to running them locally, and the sheer number of tools available is absolutely overwhelming. Regarding applications, I look at github and see so many different options that I don't know what to pick. Can't really fully decipher the differences between the tools either, mostly because their descriptions/taglines are filled with so many AI buzzwords. What's the go-to GUI for Windows? The built-in ollama GUI seems like it's pre...

GGUF for the new Cohere 30B A3B model I haven't had a chance to test this yet, but I think it's related to https://github.com/ggml-org/llama.cpp/pull/24260

For such technology with clear importance and impact on all of us, I believe that making it open source is an ethical duty, otherwise, especially with the 1-sided politics of the US we experience today, they could have already monopolized the technology by now, maybe make it exclusively available to US companies only, and starve the entire world including Europe. You could disagree with China on many topics, but the fact they released a couple of powerful open source LLMs is a direct contribut...

Reason 458 why local LLMs are going to be a necessity

Hey r/LocalLLaMA , We just released Apodex 1.0 , and alongside our flagship API, we are releasing the weights for our Smol models (0.8B, 2B, and 4B) . Our core research focuses on independent verification in long-horizon tasks. Instead of just scaling up parameter sizes for raw generation, we’ve been experimenting with small, highly specialized local models that handle specific sub-tasks in an agentic loop (like source cross-examination, hypothesis testing, and tool-grounded synthesis)....

​ This is south Korean start up all-in on inference chip: https://furiosa.ai/renegade-spec Tsmc 5nm node Hynix HBM3 1.5TB/s 48GB VRAM TDP 180W Already tested on LG LLM. If they opened their programming interface the way NVIDIA opens PTX and Intel opens SPIR-V, and team up with llama.cpp for getting a GGML backend working, it would be a game changer. Rtx pro 5000 48gb (non-hbm) is $5k now. Amd's r9700 32gb is $1.3k Intel B70 32gb is $1k I bet if their RNGD chip is ...

Built a decision-reasoning engine (Orlog) and wanted to fine-tune a local model for it instead of paying per-call forever. The method (DV-DPO): Run a 3-voice council on each question, produce a synthesis Cross-examine: losing voices challenge the synthesis If synthesis gets revised → DPO pair (chosen=post-revision, rejected=pre-revision) If synthesis holds → no pair (good reasoning produces nothing to learn from) Only genuine revisions under adversarial pressure become training...