Large Language Models (LLMs) have greatly advanced software development, helping automated code writing and ongoing improvement of programs. Recently, researchers from the National University of Singapore have devised a method to enhance the efficiency of software development through autonomous bug fixes and feature additions. Their approach, AutoCodeRover, combines the potential of advanced LLMs with code search capabilities to generate programme updates or patches.
Unlike conventional methods that view a software project merely as a collection of files, AutoCodeRover focuses on program representation with Abstract Syntax Trees (ASTs). The process facilitates effective context retrieval, enhancing the LLM’s comprehension of the program’s core issues. The strategy relies heavily on utilizing the code’s structural properties and mimicking the human cognitive process that professional programmers typically follow.
The research employs SWEbench-lite, a benchmark constituting 300 actual GitHub issues concerning feature additions and bug fixes. The team’s approach resolved 67 GitHub issues within an average of 10 minutes, outperforming previous AI attempts by over 20%. This is vastly quicker than the average 2.77 days that developers took to settle an issue manually.
AutoCodeRover’s effectiveness was deemed more important than time efficiency, so long as realistic time constraints were maintained. A 10-minute limit was placed on automated repairs, which proved 22% successful in addressing GitHub issues on SWEbench-lite. Further, by combining specific debugging and analysis techniques like test-based fault localization in the code search process, the efficacy of solving GitHub issues rose from 16% to 20%.
The AutoCodeRover approach paves the way for autonomous software engineering by boosting the potential of auto-generated codes from LLMs. Improved overall productivity and optimization of the software development process is expected with the automation of actions related to program improvement. The study establishes the feasibility of self-sufficient software engineering processes and underscores the benefits of extensive exploration of the structural properties of code to enhance the effectiveness of automated coding processes.
This research primes software development for a future where codes auto-generated from LLMs can be automatically enhanced, paving the way for more efficient and intelligent autonomous software engineering. Ultimately, AutoCodeRover promotes higher productivity and an optimized software development process through automated program improvements.
