Debugging LLM Calls: How to Trace What Your AI Agent Actually Did

Debugging LLM calls can be challenging due to the lack of logs and reasoning behind model answers. OpenTelemetry can be used to create a 'black box flight recorder' for AI calls, providing a full record of prompts, answers, timing, and cost. This can be set up in under 30 minutes with a simple library and API key. Existing tools in this space include LangSmith, Langfuse, and Arize Phoenix.