Natural language processing has been making significant headway recently, with a special focus on fine-tuning large language models (LLMs) for specified tasks. These models typically comprise billions of parameters, hence customization can be a challenge. The goal is to devise more efficient methods that customize these models to particular downstream tasks without overwhelming computational costs. This calls for innovative parameter-efficient fine-tuning (PEFT) approaches that prioritize performance while minimizing resource usage.
LLMs’ customization to specific tasks is often resource-consuming. Traditional fine-tuning methods usually update all model parameters resulting in exorbitant computational expenses and overfitting. With the colossal scale of modern LLMs, like those incorporating sparse architectures distributing tasks across numerous specialized experts, there’s an urgent demand for more effective fine-tuning techniques. The difficulty lies in enhancing performance while controlling computational burden.
Existing PEFT strategies in dense-architecture LLMs comprise low-rank adaptation (LoRA) and P-Tuning which usually involve adding new or selectively updating existing parameters. For example, LoRA disassembles weight matrices into low-rank components, decreasing the parameter count needed for training. However, these methods largely concentrate on dense models and don’t fully leverage sparse-architecture LLMs’ potential. In sparse models, various tasks stimulate different parameter subsets, rendering conventional methods less effective.
DeepSeek AI and Northwestern University researchers have devised a new method named Expert-Specialized Fine-Tuning (ESFT) specifically designed for sparse-architecture LLMs using a mixture-of-experts (MoE) architecture. ESFT aims to fine-tune the most task-relevant experts while other experts and model components are frozen. This boosts tuning efficiency and maintains the experts’ specialization, essential for peak performance. ESFT leverages the MoE architecture’s inherent capacity to allocate different tasks to experts, ensuring only necessary parameters are updated.
The ESFT process involves calculating affinity scores of experts to task-specific data and choosing a subset of the most relevant experts. These selected experts are then fine-tuned while the rest of the model remains unchanged. This strategy notably trims the computational expenses tied to fine-tuning. For instance, it can cut storage needs by up to 90% and training time by up to 30% compared to full-parameter fine-tuning, without sacrificing the model’s overall performance.
In diverse downstream tasks, ESFT often outperformed traditional full-parameter fine-tuning techniques. Particularly in tasks such as math and code, ESFT achieved notable performance advances while retaining high specialization. ESFT’s effectiveness is demonstrated through its capacity to efficiently fine-tune expert subsets, selected based on task-relevance. ESFT maintained general task performance better than other PEFT methods like LoRA, making it a useful tool for LLM customization.
In summary, the research presents ESFT as a solution for resource-intensive fine-tuning in large language models. By selectively fine-tuning relevant experts, ESFT optimizes both performance and effectiveness. This method utilizes the specific architecture of sparse-architecture LLMs to deliver superior outcomes with reduced computational costs. The research indicates that ESFT can significantly improve training efficiency, lower storage and training time, and maintain high performance across diverse tasks, making it a promising avenue for future developments in customizing large language models.
