LocalLLaMA
sycl : port multi-column MMVQ from CUDA backend (~45% speculative decoding speedup on Intel Arc) by masonmilby · Pull Request #21845 · ggml-org/llama.cpp
Saw this on other sub so posting here. For Intel ARC card holders. Big boost so update llama.cpp version( b9519 onwards) submitted by /u/pmttyji [link] [comments]
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit