Large Language Models (LLMs) are an essential development in the field of Natural Language Processing (NLP), capable of understanding, interpreting, and generating human language. Despite their abilities, improving these models to follow detailed instructions accurately remains a challenge, which is crucial as precision is instrumental in applications ranging from customer service bots to complex AI assistants.
Typically to advance LLMs’ instruction-following capabilities, high-quality training data is manually generated, with human annotators creating instructions and forming corresponding responses. This approach, however, is time-consuming and difficult to scale up. An alternative method, behavior imitation, involves training models, but they can often repeat the errors of the more advanced models from which their behaviors are learned.
Addressing these challenges, Alibaba researchers have developed a new method called AUTOIF. It automatically generates instruction-following training data by employing a type of code verification in the validation process. This method uses execution feedback-based rejection sampling, eliminating the need for manual annotation and making the process more efficient and reliable.
AUTOIF involves three main steps: generation of verifiable instructions, creation of verification codes, and ensuring reliability. Starting with seed instructions created by humans, LLMs generate a diverse set of instructions, with verification codes and unit tests then created for each one. Only instructions that can be verified by code are kept, with responses that either pass or fail the verification code used as training data.
AUTOIF has been successfully tested, outperforming several other models and achieving upwards of 90% accuracy on the IFEval benchmark. This marks the first time a model has achieved this level of accuracy. AUTOIF enabled models like Qwen2-7B and LLaMA3-8B to achieve average performance gains of over 4% in benchmarks, and replacing models with GPT-4 resulted in further improvements.
In summary, AUTOIF is a significant development in improving the instruction-following capabilities of large language models. The automated generation and verification of instruction-following data address scalability and reliability issues, ensuring the accurate execution of complex tasks. The successful testing and marked improvement in benchmarks show the potential of AUTOIF in transforming the development of LLMs. AUTOIF’s ability to deliver high-quality, scalable, and reliable instruction-following abilities could pave the way for more sophisticated and practical AI applications.
