It turned out that AI assistants do not increase the productivity of programmers

The authors of the Uplevel study, which covered the activities of approximately 800 software developers, argue that the use of artificial intelligence tools for writing software code has not yet revealed any increase in the productivity of specialized specialists, nor a decrease in their emotional burnout.

Image source: Unsplash, Luca Bravo

The study compared the performance of a sample of 800 developers over two three-month periods. In the first period, programmers worked using traditional tools, in the second they switched to using the GitHub Copilot AI assistant. It is noteworthy that such migration did not particularly affect the productivity of developers in this experiment.

Initially, it was expected that the use of GitHub Copilot would reduce the number of errors in the program code and increase productivity, but in fact in the first case there was an increase in the number of errors by 41%, and attempts to use an AI assistant did not lead to a significant improvement in the situation with emotional burnout of specialists. According to GitHub, using the Copilot assistant can increase the speed of developers writing code by 55%.

Some experts admitted that sections of program code written using AI are difficult to analyze and identify errors, so sometimes it’s easier to rewrite the desired fragment again. In the early stages of ChatGPT’s development, for example, it was noted that this chatbot simply misinterpreted more than half of all requests related to optimizing the work of program code creators. Representatives of Gehtsoft, for example, are convinced that software development is 90% dependent on the functions of the human brain in terms of understanding requirements, designing a system and identifying constraints. Turning generated mutual dependencies into program code is the easiest part of a programmer’s job.

On the other hand, there are those who praise the power of AI in developing software code. Representatives of cloud service provider Innovative Solutions, for example, note an almost threefold increase in productivity after switching to the use of AI assistants such as Claude Dev and Copilot.