What You Cannot See Will Break Your LLM App: A Practitioner Guide to Production Observability

Large language models (LLMs) break traditional application observability, making it difficult to identify issues in production. This is because LLMs are complex and their failures are not always obvious. To address this, practitioners need to adopt a new approach to production observability. This involves monitoring not just the code, but also the model's behavior and interactions. Engineers should be prepared to adapt their monitoring strategies to accommodate the unique challenges of LLMs.