What GitHub learned when better tools made Copilot code review worse
GitHub's Copilot code review tool saw a decrease in accuracy and an increase in cost after being migrated to a shared CLI toolset. However, by rewriting the instructions to match the actual workflow of a reviewer, the tool's performance improved, with a 20% reduction in cost without any loss in quality. This highlights the importance of tailored instructions for AI tools, rather than relying on generic guidance. Engineers should be cautious of assuming that shared tooling will automatically lead to improvements.