By default, llama.cpp enables pipeline parallelism, presumably to speed up inference. In my testing, I found that pipeline parallelism has no speed benefit and comes at a significant cost of VRAM. This cost can be avoided by compiling llama.cpp with the -DGGML_SCHED_MAX_COPIES=1 option. This prevents llama.cpp from allocating a much larger compute buffer when pipeline parallelism is enabled. Pipeline parallelism is enabled when --split-mode layer is used (the default) and all model layer...