I had a huge LLM server , and now I have a tiny one! I had a Jetson Orin NX gathering dust from a long dead robotics project, from back in the Llama-7B days. I figured now with MoE and smaller models doing well, it was time to mess with it again. Goal: As silent as possible (given they bumped the power from 25W -> 40W) Greater than 10 tok/s TG and 300 tok/s PP at least 65K context for Hermes Agent Must look cool AF 👌🏻 With those constraints, I had to take a hacksaw to the...