LocalLLaMA
I built a GBNF grammar compiler that makes 8B models reliably call tools - here's how it works (deep dive)
I've been building a local agent in Rust (Eris) that runs on llama.cpp and uses an Obsidian-compatible vault as memory. ~50 tools (vault read/write, memory, reminders, web fetch, email, calendar, vision). The biggest pain was getting small models to emit valid tool-calling JSON. You all know the drill: the model wraps JSON in code fences, invents tool names, forgets closing braces, adds trailing prose after the object. My solution: I compile each tool's JSON Schema into GBNF rules at session...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit