EDIT: Added the ability to use any open ai compatible endpoint per many requests! I wanted AI Dungeon but fully local and actually private, so I built it. The narrator is Gemma 4 (QAT Q4) through Ollama, and when a scene is worth showing it draws the picture too, locally, with FLUX. No API keys, no cloud, nothing leaves your machine. The part that surprised me: you can run the 12B at its full 256k context and it still only sits around 7.7GB of RAM, because Gemma 4 barely grows the KV cache. ...