In a scenario where what we're doing is describing and assigning work to someone, having them paste that into an LLM, sending the LLM changes to me to review, me reviewing the LLM output, them pasting that back into the LLM and sending the results for me to review...
What value is that person adding? I can fire up claude code/cursor/whatever myself and get the same result with less overhead. It's not a matter of "is AI valuable", it's a matter of "is this person adding value to the process". In the above case... no, none at all.
Because we know what the value is without AI. I’ve been in the industry for about ten years and others have been in it longer than I have. Folks have enough experience to know what good looks like and to know what bad looks like.
The Stanford study showed mixed results, and you can stratify the data to show that AI failures are driven by process differences as much as circumstantial differences.
The MIT study just has a whole host of problems, but ultimately it boils down to: giving your engineers cursor and telling them to be 10x doesn't work. Beyond each individual engineer being skilled at using AI, you have to adjust your process for it. Code review is a perfect example; until you optimize the review process to reduce human friction, AI tools are going to be massively bottlenecked.