I'm the author, so discount the enthusiasm accordingly. This is an unaffiliated community port, not endorsed by the vLLM project, which it uses to verify its correctness. What started it: I love vLLM, but a vLLM install here is 9.1 GiB of virtualenv, and I wanted to embed inference inside other software, on machines where having an interpreter in the process is a problem. And, honestly, Python dependencies have a different deployment story, in term of security (supply chain attacks), and bloat...