LocalLLaMA
A handy llama-server launcher with easy model and configuration customisation
I wanted something that I could easily configure to manage a set of sensible defaults, that supports multiple llama-server binaries, with per-model over-rides, and command line over-rides. The utility is here: https://github.com/stew675/start-llama I know that llama-server has its own model loading configuration available via the API end-point, but I just wanted something that I could start from the command line easily in one step. I don't know if anyone else may find this useful or not,...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit