Tech
BPE Tokenization: A Trailing Space in My Prompt Broke 41 of 300 Labels
I spent an evening blaming a quantized 8B model for being dumb. It was classifying support tickets into four labels, and roughly one answer in seven came back as garbage: a double space, a capitalized label, a stray newline, half a word. I tried a bigger quant, a lower temperature, a stricter instruction. Nothing moved.
The bug was one character. My prompt template ended with Label: and that final space was fighting BPE tokenization. When I deleted it, 41 bad outputs out of 300 dropped to 3...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to