Tech
Running DiffusionGemma Locally on an RTX 5090 with vLLM: The One-Time Pioneer Cost of Bleeding-Edge Model Bleeding-Edge GPU
Introduction
In the previous post I covered keeping a home RTX 5090 (Blackwell) alive 24/7 against the GSP hang. This time it's about the model that runs on top of it — a record of standing up Google's diffusion language model DiffusionGemma locally on that 5090.
Let me lead with the conclusion. It's genuinely fast — roughly ~300 tok/s by feel. But the surrounding ecosystem is still immature (Open WebUI's web search doesn't integrate, among other things), so it earned a place as an...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to