
LocalLLaMA
Diffusion Gemma is 4x faster, but makes 6x more mistakes!
Benchmarked the new Gemma diffusion model against its autoregressive twin on a single H100 (FP8). We gave each the same three tasks: write a Steve Jobs biography, the history of Tetris, and the story of BeOS - every next topic less popular than the previous one. Then we fact-checked every claim in every answer. Gemma4 got 45 facts right, 5 wrong. DiffusionGemma got 33 right, 28 wrong. The less popular the topic, the worse it got: 4 mistakes on Jobs, 12 on Tetris, 12 on BeOS. It named Clara Cll...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit