TrendPulse

Hi all. These are my system specs: dual xeon e5 2696 v2 , 160gb DDR3 ram ECC(1600mhz), 3 gpus: 3060 12gb, p100 16gb, 3050 6gb. And a 400gb nvme sdd RAID0, 3000 mb/s. The model is Deepseek-flash-0731 UD_8_X_XL, loseless, 161gb. Now, I'm not too knowledgeable about llama cpp flags, I wish run it without mmap, because its so slow, and I believe it should fit in my system overall. There's also Dspark and MTP which could help with the speee, but do they work with llama? Any recommendations would help...

https://preview.redd.it/h7dwy09z2rhh1.png?width=1240&format=png&auto=webp&s=7285448ac2d9c895c9a11062557044fc97504fa3 I was evaluating which model to test for text summarization and rewriting in a scenario that also requires general world knowledge. Taking into account the available VRAM and my experience over the past few months, I’ve found Minimax 2.7 Q5 to offer the best balance for my needs and the numbers seem to confirm. What is your favorite model? I’m referring to the mod...

nvidias nemotron omni is open weights and it sees, hears and reasons. theres already a 4bit mlx quant on hugging face but only the text backbone loads with standard mlx tooling. the model card says it plainly, the vision and audio towers need a runtime that implements the c-radio vit-h and parakeet conformer forward passes. so i wrote that runtime in pure mlx. vision tower, audio tower, the processor and the token splicing, all ported from nvidias reference implementation. it runs the mlx-comm...

Code and instructions available here: https://github.com/albertoZurini/echo-dot-2-playground Hello there! After a few days of experimenting I was able to get a completely local voice pipeline running on an Amazon Echo Dot 2. The interesting part for this community is that the device can run `llama.cpp` alongside offline speech recognition even with its very limited hardware. The Echo Dot 2 has an ARMv7 processor and 512 MB of RAM. I compiled `llama.cpp` for `armeabi-v7a` and run the model ...

J'ai consacré beaucoup de temps à l'optimisation de DeepSeek-V4-Flash-0731 GGUF sur une seule RTX 3090. Mon exigence absolue pour chaque configuration était la suivante : Le modèle doit rester utilisable avec une fenêtre de contexte de 128 000 jetons. J'ai testé les différentes combinaisons de déchargement GPU, de placement expert du CPU, de quantification du cache KV, de tailles de lots, de mappage mémoire et de répartition de la mémoire CPU/GPU. Les paramètres ci-dessous ont permis d...

I love to see these impressive models coming out that compete with the giants from companies like Z.ai, Moonshot, Alibaba, etc. A win for the open source/weight community is always welcome. While I am grateful, I worry we might be seeing the slow death of models smaller than 27B. The ones released paling in comparison to Qwen 3.5 4B/9B and Gemma 4 12B. Especially for agentic coding and agentic assistance tasks. Is this because we’ve really hit the limit of what we can accomplish with models in...

Hey peeps. I know you're tired of low quants giving hard to believe numbers. I'm quite skeptical too and from what I tried I'm often left with the impression that the claims fall short. So this model popped up on Twitter for me. Tried it and was lowkey surprised it held its own. I ran some benchmarks with the help of antigravity to at least try to verify it myself. Here is what I got: Axis / Metric Escha (W2 ROCmFPX) APEX (Q5 Balanced) Key Finding / Winner VRAM Memory Alloc...

TL;DR: On a Qwen3.6-35B-A3B Q6 setup sized for 64K context on a 24GB RTX 3090, spilling eight MoE expert layers to CPU freed enough VRAM to increase -b from 512 to 1024 and -ub from 128 to 512. Prompt processing improved by 2.36×, while generation speed remained unchanged within measurement noise. Benchmark Auto-fit baseline Tuned Result PP4K 564.5 tok/s 1330.0 tok/s 2.36× TG4K 97.4 tok/s 97.7 tok/s Within noise TG32K 81.6 tok/s 84.0 tok/...

Just came across this coding benchmark: SciCode Artificialanalysis.ai reports a ranking which contradicts the feeling we've towards those models in real life coding. Is Gemma 4 really that good, or a benchmarking issue? EDIT: The contribution of this benchmark to the Intelligence index of artificialanalysis.ai: Full Intelligence Index v4.1 weights: GDPval-AA v2: 20% Terminal-Bench 2.1: 16% τ³-Bench Banking: 14% Humanity's Last Exam: 12% AA-Omniscience Accuracy: 8% SciCode: 8% GPQA: 6% ...

Press My earlier prediction that Tesla would buy them completely missed the mark. With AMD focusing heavily on the enterprise side, the idea of consumer-facing hot-swappable AI model chips looks pretty much dead. Fast forward ten years, you might find used model blade cards on eBay, except a full model's weights will be split across them, so it'll take multiple blades chained together just to make up a single complete set of weights. ...

I swear AA is not the bipartisan they so claim. An open source mode (Qwen 3.8 max) was number 1 on the agentic index, then they just so happen to launch "v4.1.1" of their index in which they just adjusted the weights of the gdpval and t3 banking so that it would be lower than opus, despite the lead in t3 being a 8% lead over opus while opus only has a 5% lead on gdpval. Highly likely to be paid off imo. You can check other subreddits for the score before and after the change, just made...

NVIDIA Nemotron Parse 2.0 transforms document images into structured, machine-readable representations with text, layout classes, bounding boxes, and reading-order information. Given a Red, Green, Blue (RGB) document image and a task prompt, the model produces formatted text and spatial annotations for document elements such as titles, paragraphs, captions, tables, charts, page headers, page footers, footnotes, pictures, and bibliography entries. Compared with NVIDIA Nemotron Parse v1.2, NVIDIA ...

Link to the article: KV Cache Quantization Benchmarks: KVarN, Precision Tail KLD benchmarks with BeeLlama.cpp v0.4.0 , fork of llama.cpp with more KV cache quantization options. Models: Qwen 3.6 27B Q5_K_S 64k context, Gemma 4 31B Q5_K_S 16k context Standard quants, extended: q6_0 and q6_1, and low-bit types from q2_0 to q3_1 KVarN: Variance-Normalized KV-Cache by Huawei, implemented in BeeLlama Precision Tail: keeping latest X tokens of KV cache in (B)F16, implemented in BeeLlam...

Reading through this subreddit and many more I keep running into what I am calling "AI click bait". Either projects that seems interesting in the description/title but when you open them they're the same AI vide coded slop that does not solve the problem; or apparent discussions about an actual problem that are just undercover marketing ploys to sell you a product that also does not solve the problem. I guess I don't really have a point to this, I just encountered the 100th post o...

GGUFs here: https://huggingface.co/ReadyArt/gemma-4-31B-it-scotoma-2-GGUF Disclaimer: By slop, we are specifically talking about specific tics with the model(sentence structures), but this doesn't include words such as "ozone". Summery of the model: Scotoma-2 is a model made by user https://huggingface.co/AesSedai which aims to reduce common Gemma4 tropes, such as(not limited to): "It's not x, it's y" and "a slow knowing smirk "(stacked adjectives) which t...

🐦‍⬛ Magpie-TTS Multilingual 🦜 Nemotron Speech Streaming EN 0.6B 🦜 Nemotron-3.5 ASR Streaming 🦜 Parakeet CTC 1.1B 🦜 Parakeet TDT 0.6B v3 🥦 NanoCodec Merged PR https://huggingface.co/nvidia/magpie_tts_multilingual_357m#run-magpietts-locally-with-nemo-speechcpp I run open source llm models via AI Desktop XP but how am I supposed to run these models on my phone? Any recc?

In a previous post , I compared MinerU, Granite-Docling, and PaddleOCR-VL. Many commentors suggested I added their favorite parsers. So I did. And also added some new capabilities to differentiate the top models. Here is the full list of parser compared: MinerU 2.5 (1.2B VLM) Granite-Docling (258M VLM) PaddleOCR-VL (0.9B VLM) XBerg 1.0 (text-layer parser, CPU) HURIDOCS PDLA v0.0.35 (VGT layout model + Tesseract) LiteParse 2.11 (Tesseract based, CPU) Chandra (Datalab's OCR model)...

I'm the author, so discount the enthusiasm accordingly. This is an unaffiliated community port, not endorsed by the vLLM project, which it uses to verify its correctness. What started it: I love vLLM, but a vLLM install here is 9.1 GiB of virtualenv, and I wanted to embed inference inside other software, on machines where having an interpreter in the process is a problem. And, honestly, Python dependencies have a different deployment story, in term of security (supply chain attacks), and bloat...

google facing yet another "innovator's dilemma" or some shi smh lgtb ig EDIT - WHEN THE FUCK WILL LE CHATON FAT COME RIGHT ON !!????

This is very important for us when considering local hosting. A lot of people decided not to buy expensive hardware because DeepSeek’s prices made it very difficult to break even given that deepseek was soo cheap. Also some of us use DeepSeek in routing, hosting Qwen and routing some hard tasks to DeepSeek API. what do you think about this? do you think raising prices will ultimately lead to another increase in NVIDIA’s GPU prices, since more and more people will now buy their own hardware...

Certo is open-source infrastructure for issuing, managing, verifying, and exchanging digital credentials. It implements Open Badges 3.0[1] and W3C Verifiable Credentials[2] which are the open standards that make credentials portable, machine-verifiable, and vendor-independent. It is designed to be: - Self-hosted: run it inside your own infrastructure, air-gapped network, or sovereign cloud - Standards-compliant: credentials it issues work with any OB3/VC-compatible system - API-first: everything...

Article URL: https://www.bbc.com/news/articles/c1e1vg0gjl5o Comments URL: https://news.ycombinator.com/item?id=49208314 Points: 216 # Comments: 179

Article URL: https://www.currentaffairs.org/news/the-massive-climate-lie-that-will-destroy-human-civilization Comments URL: https://news.ycombinator.com/item?id=49208686 Points: 39 # Comments: 12

Launched my SaaS this week, first client is a company with 6 users(i bill per seat). small money but it's real money and it felt great. Thing is my plan for this month was marketing. Youtube tutorials, cold emails, all that. haven't touched any of it. Instead i spend every day fixing small bugs and sitting in Posthog checking if they're actually using it(they are, which is a relief), and seeing which features they use most has been genuinely useful for deciding what to build next. But it's eat...

​ I’ve been helping a few small teams improve their internal processes, and one thing keeps standing out: Most early-stage SaaS products focus heavily on building features, but very little on automating the repetitive operational work that happens around the product. The biggest time drains I’ve noticed are: - lead inquiries getting lost in email, - manual trial-user follow-ups, - copying customer data between forms, sheets, and CRMs, - reminding users about onboarding steps...

I'm nosy... who has done this? what is it and how much is it making? did you quit your job? did you get lucky? how'd you market it? I have so many questions...