I measured every millisecond of my real-time AI pipeline. The LLM was the fast part.

A real-time AI pipeline was instrumented to measure performance. The language model was not the bottleneck, but rather the transcription and delivery stages. The model choice was also important, with a smaller model being faster and more suitable for real-time applications. Engineers should prioritize model speed over intelligence when working with real-time features.