Machine learning, in particular large language models (LLMs), is seeing rapid developments. To stay relevant and effective, LLMs, which support a range of applications from language translation to content creation, must be regularly updated with new data. Traditional methods of update, which involve retraining the models from scratch with each new dataset, are not only time-consuming but also require substantial computational resources. This can make keeping up-to-date with latest models unsustainable due to high computational costs.
Research teams from Université de Montréal, Concordia University, Mila, and EleutherAI have been exploring a number of strategies aimed at simplifying the model updating process. Amongst these, continual pre-training has emerged as a potentially fruitful solution. This approach aims to update large language models with new data without having to start training afresh, thus retaining the knowledge the model has previously acquired. The main challenge with this method is adding new information without erasing existing knowledge, a phenomenon known as catastrophic forgetting.
The selected approach involves adjustments to the learning rate and replaying a portion of the previously learned data. The method can adapt the model to new datasets significantly easing the computational load compared to conventional retraining. Adjusting the learning rate through a process known as re-warming and re-decaying, along with replaying some of the old data, assists the model in retaining information previously learned.
There are several key benefits to the proposed approach. It demonstrates that LLMs can be efficiently updated with new data through a simple and scalable method. Importantly, the model can adapt to new datasets without significant loss of knowledge from earlier datasets. This is achieved through a combination of learning rate adjustments and selective data replay. This method has proven effective across a range of scenarios including transitioning between datasets in different languages. This technique performs just as well as fully re-trained models but requires only a fraction of the computational resources.
The process involves fine-tuning the learning rate to assist the model’s adaptation to new datasets. This is achieved through increasing the learning rate (re-warming) at the start of training with new data, and then gradually decreasing it (re-decaying). A chosen portion of the previous dataset is replayed during training. This two-tiered strategy efficiently integrates new information whilst avoiding catastrophic forgetting.
The research showed the method achieved comparable results to traditional re-training models but did so more efficiently. It presents a viable and cost-effective approach to updating LLMs, reducing the computational requirements and making it more feasible for organizations to maintain current and high-performing models.
In summary, this study offers a novel approach to overcoming the computational hurdles inherent with updating LLMs. Through learning rate adjustments and data replay, the study demonstrates that the relevancy and effectiveness of LLMs can be maintained without starting from scratch each time. This not only significantly improves the efficiency of machine learning but also opens up new possibilities for developing and maintaining cutting-edge language models.
