Large Language Models (LLMs) are pivotal for numerous applications including chatbots and data analysis, chiefly due to their ability to efficiently process high volumes of textual data. The progression of AI technology has amplified the need for superior quality training data, critical for the models’ function and enhancement.
A major challenge in AI development is guaranteeing the synthetic data used to train these models is varied and high-quality. This process can require significant human involvement, and without this supervision, the risk of model collapse is heightened. Consequently, this can lead to poor learning outcomes and biased results, restricting the real-world application of these models.
A typical method for synthetic data generation is utilizing robust models like GPT-4 to produce responses to prompts. Despite being effective, human input is still needed to guarantee the quality and relevance of the data. Researchers have devised methods to enhance data quality, however, the process is labor-intensive and can be inconsistent.
Researchers from Microsoft Research are addressing these issues by introducing AgentInstruct, a unique framework that automates the generation of diverse, high-quality synthetic data. This framework uses raw data sources, such as text documents and code files, to generate data, thus reducing the need for human involvement and increasing the overall quality and variety of the training data.
AgentInstruct uses a multi-agent workflow, which includes content transformation, instruction generation, and refinement processes. This approach allows the system to autonomously produce a wide array of data, ensuring complexity and diversity.
The researchers tested AgentInstruct by creating a synthetic post-training dataset of 25 million pairs, used to teach various skills to language models. The results indicated considerable improvements across multiple benchmarks. For example, the model demonstrated a 40% improvement on AGIEval and a 54% improvement on GSM8K.
Within AgentInstruct, the content transformation flow converts raw data into intermediate representations that simplify the creation of instructions. Following this, the seed instruction generation flow generates diverse instructions from these transformed seeds, and finally, the instruction refinement flow improves the quality and complexity of these instructions.
Training Orca-3, a model based on the Mistral-7b model, with the AgentInstruct dataset showed better performance than other instruction-tuned models, such as LLAMA-8B-instruct and GPT-3.5-turbo. These benchmarks highlight the significant progress made by AgentInstruct in synthetic data generation.
AgentInstruct marks an innovative development in creating synthetic data for AI training. The automation of diverse, high-quality data generation addresses vital issues concerning manual curation and data quality. The substantial improvements observed in the Orca-3 model, including a 40% improvement on AGIEval benchmark, underlines the framework’s effectiveness.
