The problem is with how people interpret these results.
A paper comes out that says "we did a study of developers and found that AI-assistance had no impact on their productivity (using the state of the art models available in September 2024) and a lot of people will point to that as incontestable evidence that "AI doesn't work".
A paper comes out that says "we did a study of developers and found that AI-assistance had no impact on their productivity (using the state of the art models available in September 2024) and a lot of people will point to that as incontestable evidence that "AI doesn't work".