AI Coding Assistants Produce ‘Bad Quality Code’: Study

K. C. Sabreena Basheer 25 Jan, 2024 • 2 min read

A recent study by GitClear delves into the intricate world of code development, analyzing a staggering 153 million lines of code to unravel the impact of generative AI coding tools on code quality and maintainability. This study sheds light on how the surge in AI tools is influencing programming practices.

Also Read: GPT-Engineer: Your New AI Coding Assistant

GItClear study shows AI coding assistants and tools produce bad quality content

Unraveling the Code Quality Tapestry

GitClear’s analysis focuses on key aspects such as code churn, duplication, and age, spanning the years 2020 to 2023. By scrutinizing code authored in 3400 repositories, the study paints a comprehensive picture of the evolving coding landscape.

Also Read: Leveraging GenAI to Enhance Coding Skills

Key Metrics: Code Churn, Duplication, and Age

Results from the study highlight significant shifts in three key metrics in the latter half of the studied period: code churn, code duplication, and code age. These metrics serve as markers for understanding the evolving nature of code development influenced by the advent of AI programming assistants.

AI’s Influence on Code Churn and Duplication

The increase in code churn, signifying the percentage of code undergoing substantial alterations soon after integration, suggests a rise in the generation of what the study terms as “bad code.” The higher percentage of duplicated code points to developers resorting to “quick patches,” potentially compromising the clarity of projects.

The GitHub Copilot Context: A Game-Changing Era

The year 2023 marked the mainstream launch of GitHub Copilot, propelling AI-generated code into the limelight. GitHub’s CEO, Thomas Dohmke, touts impressive statistics – 15 million more “developers,” a global economic impact of 1.5 trillion U.S. dollars, 55% faster coding, and 46% more code written. GitClear’s study seeks to measure the tangible implications of this widespread adoption.

Also Read: Microsoft GitHub Copilot Chat Revolutionizes Coding Assistance

Analyzing Code Operations and Developer Intentions

GitClear classifies code changes into seven main operations, deciphering developer intentions. From “additions” indicating new features to “deletions” aligning with cleanup, the study aims to understand how AI-generated code influences the intricate dance of code development. The concept of “Churned Code” becomes pivotal, representing changes that were incomplete or erroneous upon creation.

Our Say

As the coding landscape evolves with the integration of generative AI tools, GitClear’s study sparks essential conversations. The rise in code churn and duplication prompts a closer look at the implications of AI-driven coding practices. The challenge now lies in balancing the efficiency gains promised by AI with the potential drawbacks in code quality and maintainability. It seems that the human layer of software developers still remains crucial in the era of GitHub Copilot.

Follow us on Google News to stay updated with the latest innovations in the world of AI, Data Science, & GenAI.