Large Language Models (LLMs) have become essential tools in various industries due to their superior ability to understand and generate human language. However, training LLMs is notably resource-intensive, demanding sizeable memory allocations to manage the multitude of parameters. For instance, the training of the LLaMA 7B model from scratch calls for approximately 58 GB of memory. This high memory requirement presents substantial access difficulties for many researchers and developers who lack the necessary advanced hardware.
To overcome this issue, a multitude of techniques have been developed such as creating smaller LLMs, utilizing efficient scaling techniques, and integrating sparsity into training methodologies. Among these, GaLore has been identified as a prime contender, permitting full-parameter training of LLMs through Singular Value Decomposition (SVD) for low-rank gradient updates. GaLore reduces memory usage by up to 63.3%, enabling a 7B model to be trained using 24GB of memory only. Despite this, GaLore still requires more memory than many commonly used devices have available.
Researchers from the University of Texas at Austin, the University of Surrey, the University of Oxford, the California Institute of Technology, and Meta AI have put forth a new solution called Q-GaLore to further reduce memory consumption and make LLM training more accessible. Q-GaLore combines quantization and low-rank projection to improve memory efficiency significantly. It operates by adaptively updating the gradient subspace based on convergence statistics while lessening the number of SVD operations.
In practice, Q-GaLore has performed excellently in pre-training and fine-tuning scenarios. Remarkably, Q-GaLore allowed for the training of an LLaMA-7B model from scratch on a single NVIDIA RTX 4060 Ti using just 16GB of memory. This is significant as it highlights the method’s remarkable memory efficiency and practicality. Q-GaLore also lessened memory consumption by up to 50% as compared to other techniques like LoRA and GaLore, while consistently outperforming QLoRA by up to 5.19 on MMLU benchmarks at the same memory cost.
The performance of Q-GaLore was assessed across various model sizes, from 60 million to 7 billion parameters. For a model with 1 billion parameters, Q-GaLore achieved comparable pre-training performance with a less than 0.84 increase in perplexity compared to the original GaLore, whilst saving 29.68% memory compared to GaLore. Notably, Q-GaLore facilitated the pre-training of a 7B model within a 16GB memory restriction, maintaining a perplexity difference of less than one compared to baseline models.
In conclusion, Q-GaLore provides a practical means to the memory constraints traditionally accompanying LLMs. By merging quantization and low-rank projection, Q-GaLore delivers competitive performance and broadens the reach of powerful language models. The method is a testament to the potential of optimizing large-scale models for more common hardware configurations, making innovative language processing technologies more available to a broader audience.
