Large language models (LLMs) are crucial in advancing artificial intelligence, particularly in refining the ability of AI models to follow detailed instructions. This complex process involves enhancing the datasets used in training LLMs, which ultimately leads to the creation of more sophisticated and versatile AI systems. However, the challenge lies in the dependency on high-quality instruction datasets that are tough to annotate on a large scale. Manual methods require excessive human expertise and resources, hindering consistent improvements across different tasks.
Recent methods like Evol-Instruct address this disconnect by improving dataset complexity and diversity using iterative refinement of data through LLMs. Although effective, this process heavily leans on heuristic efforts and expert-designed evolving rules, making it costly and time-consuming. Further, every new task requires a redesign of the execution evolution methods, adding to the work and cost.
Researchers from Microsoft are proposing a solution to these challanges, the Auto Evol-Instruct. This is an automated framework that relies on LLMs to design evolving methods independently, eliminating the need for human intervention. This innovative method ensures a cost-effective adaptation to various tasks by modifying the input data autonomously.
The process begins with an universal initial evolving method that analyzes the input instructions and formulates evolution rules, which are then optimized iteratively by an optimizer LLM. The optimizer identifies and addresses issues in evolving methods, reducing chances of evolution failure while enhancing the dataset’s complexity and diversity.
The Auto Evol-Instruct analyzes the input instruction and formulates evolution rules suited for the given data, removing the need for human experts to specify the evolution rules. The evol LLM devises a comprehensive plan based on the methods outlined autonomously and executes the evolved instruction. The evol LLM then scrutinizes the evolved instruction and corrects any inconsistencies, ensuring complexity and stability of the final evolved instruction.
Auto Evol-Instruct performance has been evaluated across multiple benchmarks. Highlighted results show the framework surpassing GPT-3.5-Turbo and WizardLM-70B and achieving scores comparable to Claude2.0. It furthermore outperformed GPT-3.5-Turbo, WizardMath-70B, and MetaMath-70B. The framework also surpassed scores of GPT-3.5-Turbo and WizardCoder-34B in code generation.
An important aspect of the Auto Evol-Instruct is its ability to iteratively optimize the evolving method through Evol Trajectory Analysis and Evolving Method Optimization stages. The optimizer LLM provides feedback on potential issues and failures during instruction evolution, leading to subsequent optimization and refinement of the evolving method towards the lowest failure rate.
In conclusion, Auto Evol-Instruct addresses the constraints of manual methods by automating the evolution of instruction datasets and provides a scalable and efficient solution for LLMs. It demonstrated significant strides beyond human-crafted methods, showing promising potential for AI advancement. The framework’s noteworthy results across various benchmarks underscore its essence in enhancing capabilities in following instructions, mathematical reasoning, and code generation.
