Software engineering is a rapidly evolving field aimed at systematic design, development, testing, and maintenance of software systems. In recent times, large language models (LLMs) such as GPT-3 have been employed to automate and optimize various software engineering tasks. However, the use of autonomous LLM-based agents has its challenges given their cost and complexity, and the intricacies of tool usage and decision-making.
Currently, many methods for addressing software engineering issues employ autonomous LLM agents that use various tools and perform actions based on feedback from their environment. While this approach is similar to human problem-solving techniques, it has several limitations, including the possibility of errors and imprecise results due to complex tool design and usage, and suboptimal decisions made by the agents.
To address these challenges, researchers from the University of Illinois Urbana-Champaign developed an innovative method called AGENTLESS which aims to simplify the resolution of software development problems. Unlike traditional methods that rely on autonomous agents, AGENTLESS uses a two-phase approach: localization and repair. This method does not require LLMs to make autonomous decisions or use complex tools, making it a simpler and more cost-effective solution.
In the localization phase, AGENTLESS identifies specific files, classes, functions, and lines of code that require modification. The project’s codebase is transformed into a tree-like structure for easy identification of suspicious files. After pinpointing the files, relevant classes and functions are located, thus reducing the complexity and volume of the code to be analysed.
In the repair phase, AGENTLESS generates multiple candidate patches for identified areas requiring modification. A simple diff format is used to create search/replace edits. Patches with syntax errors or failed regression tests are eliminated and the remaining patches are ranked using majority voting.
The effectiveness of AGENTLESS was tested using the SWE-bench Lite benchmark. It successfully resolved 82 out of 300 problems with an average cost of $0.34 per problem. Moreover, AGENTLESS was able to solve unique issues unresolvable by other open-source agents.
In summary, AGENTLESS offers an innovative alternative to complex autonomous LLM-based processes in software engineering. The simplified, two-phase approach to problem-solving efficiently resolves software development issues, paving the way for more accessible, efficient and cost-effective solutions in this domain. While AGENTLESS shows promise, further research and development are needed to boost its potential in autonomous software development.
