Building an AI Runtime Operating System for Commodity Hardware For the last few months I've been working on something that started as a simple question: Why do we still treat AI inference as "load an entire model into GPU memory and hope it fits"? My development machine certainly doesn't make life easy. Dell Precision 5520 NVIDIA Quadro M1200 (4 GB VRAM) 32 GB RAM Ubuntu Like many developers experimenting with open-source LLMs, I quickly ran into the same w...