Artificial
Why does it feel like big LLM providers are literally hiding prompt caching?
I know the info is there. Somewhere in the pricing pages, docs, or API notes. But for something that can seriously change what you pay in production, it is weirdly under-explained. expeciely for other providers than openai which they do have decent explainer here - https://developers.openai.com/api/docs/guides/prompt-caching So basicly: two prompts can look almost identical, but one can be much cheaper to run just because it is ordered better. Put the changing parts too early, like the...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit