Artificial
When an LLM API silently fails or degrades, how do you find out - and how long does it take?
Asking to developers and power users, as a genuine research question. If you are building on top of multiple LLM APIs or even a single one amongst OpenAI, Claude, Gemini, etc. what do you do when the API starts degrading (slow TTFT, elevated error rates, timeouts). Or even worse, when there are responses but the model is drifting or hallucinating. How do you find this out? I'm trying to understand if this is a widespread pain or just something I've been unlucky with. Three specific questions...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit