Researchers from the College of Computer Science at Sichuan University and the Engineering Research Center of Machine Learning and Industry Intelligence in Chengdu, China have developed a method for quickly adapting dense retrieval models, known as DREditor. These models are crucial for industries such as enterprise search (ES), where service providers use personalized search engines to assist customer inquiries based on uploaded business documents. Providers must achieve speedy searching customization to meet scalability requirements. Any delays can result in poor customer experiences and potential business losses.
Existing methods such as the long-term fine-tuning of retrieval models can be too time-consuming and may not yield optimal results. Extensive training times drain computational resources, which increases infrastructure and energy costs. It also slows down development and experimentation cycles, which are needed to adapt models to changing requirements. This led to the search for a fresh solution.
DREditor uses efficient linear mapping to calibrate output embeddings. It approaches the issue differently, tackling a specifically constructed least squares problem with a custom-made edit operator. Compared with traditional fine-tuning, DREditor is 100–300 times more time-efficient across various datasets, sources, models, and devices, while maintaining or even enhancing retrieval performance.
The method also includes a post-processing step that applies a computation-efficient linear transformation via the derived edit operator. DREditor significantly outperforms older rule modification techniques, indicating its applicability to various scenarios. The researchers highlight their method’s potential for efficient development of domain-specific dense retrieval models.
In conclusion, the researchers introduced DREditor as a time-efficient method for customizing domain-specific dense retrieval models. It boosts the speed and scalability needed by ES providers to meet time-sensitive demands, and integrates the emerging focus on embedding calibration into retrieval tasks. Finally, the researchers extend the model’s applicability to zero-shot domain-specific scenarios, highlighting its potential for efficient and cost-effective development.
If you find this research interesting, follow us on social media and join our online communities. Don’t forget to check out our paper on Github. Credit goes to the original researchers for their work. Join us for more updates on Twitter, Reddit, Facebook, Discord, and LinkedIn. If you enjoy our content, you’ll love our newsletter. Our Telegram Channel always keeps you in the loop.
