Tech
Your Chat History Is Eating the Context Window — Fix It With a Reducer
Turn nine of a support chat feels fine. Turn ninety is a different animal. Every request resends the system prompt, three agent loops of tool noise, the user's third restatement of the same bug, and yesterday's dark-mode tangent. The model does not mind. Your invoice does — and attention quality usually dies before the hard token limit.
messages.TakeLast(20) in one controller is not a policy. The second entry point forgets it. A tool-using agent keeps a function call without its result. Som...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to