AI Coding Tools Lift Microsoft Engineers' Output 24% in Week One

Tens of thousands of software engineers at Microsoft began using AI coding tools between January and April 2026. The tools in question are not the button-driven kind but ones that take instructions as text typed into a black command-line window: Anthropic's Claude Code and GitHub's Copilot CLI. The researchers tracked who tried the tools, who kept using them, and how their work output changed.

Workplace relationships were what first prompted people to try them. An engineer was more likely to try the tools when a colleague with whom they regularly exchanged code reviews (the practice of reading and checking each other's code) was already using them. If their direct manager was using them, the odds of trying them rose by 82%; if more than a quarter of their review partners were using them, the odds rose by 54%, where odds refers to the ratio of the likelihood of trying the tools. When more than a quarter of colleagues who shared a manager two levels up had used the tools in the previous two weeks, the odds jumped by as much as 216%.

Trying the tools and continuing to use them were different matters. Whether people kept using them depended on how much code they typically wrote. Busy engineers who wrote a lot of code regularly and submitted pull requests (requests to merge changes into the main codebase) frequently were more likely to see the benefits of AI and stick with it. Conversely, people who had already been using other AI tools in a different development environment were less likely to stick with the new tools, perhaps because they felt their existing setup was better.

The effect on output was as follows. Engineers who used the tools in their first week saw the number of merged pull requests rise by an average of 24% compared with those who did not. The increase was not a short-lived spike right after adoption; it held steady across the four months studied. By tool, Claude Code users improved by 11.4% and Copilot CLI users by 24.9%.

What does this mean? For organizations thinking about adopting such tools, the results suggest that adoption spreads outward from the colleagues and managers who already use them. If the goal is for the tools to stick, the data suggest that getting them to developers who write a lot of code first is likely to be most effective. However, these are internal figures from Microsoft. The paper is a preprint, meaning it has not been peer-reviewed and was posted on arXiv, and it remains to be seen whether engineers outside Microsoft or at other workplaces would see the same results. The 24% figure, moreover, measures the number of merged pull requests, not the quality of the code itself.