Canary an LLM Evaluation Harness With Queue Overload and a Concurrency Gate

An AI evaluation harness measured high latency due to its own queueing, not the model's performance. This matters for evaluating free or rate-limited model endpoints under load. To fix, use a closed-loop harness with a concurrency gate to separate queueing from endpoint behavior. This ensures measured latency is dominated by the system under test, not the harness's queueing. The fix is structural, not statistical, and involves splitting measurements into two clocks: queue wait time and service time.