We run a small on-prem inference fleet - four nodes, mixed A6000 and 4090 cards - and last quarter we replaced a pile of hand-rolled docker run wrappers with GPUStack. The pitch is reasonable: one control plane that schedules models across whatever GPUs you happen to own, materialises a backend per deployment, and exposes an OpenAI-compatible endpoint. Two months in, it does what it says. It also has a specific failure boundary that cost us a weekend, and that part is worth writing down. ...