SaaS
At what point do you split LLM traffic by cost sensitivity?
One thing I keep seeing with AI products is that the demo phase hides the real cost structure. At first it is just a few API calls. Then usage grows and suddenly you have realtime user-facing calls, background summaries, evals, enrichment jobs, retries, agent loops, and support tooling all running through the same provider path. Prompt cleanup, caching, and smaller models help, but they still treat all traffic as if it has the same latency and reliability requirements. For people running A...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit