Large Language Models (LLMs) have significantly contributed to the enhancement of conversational systems today, generating increasingly natural and high-quality responses. But with their matured growth have come certain challenges, particularly the need for up-to-date knowledge, a proclivity for generating non-factual orhallucinated content, and restricted domain adaptability. These limitations have motivated researchers to integrate LLMs with external knowledge to improve the accuracy, reliability and versatility of their conversational responses. This paper looks at the need for each system response turn to be augmented with external expertise and proposes an adaptive solution (RAGate).
Several techniques have been explored to enhance conversational responses, including knowledge retrieval and joint optimization of retriever and generator components. Dense passage retrieval methods and public search options, for example, fetch relevant information for conversational responses, reducing hallucination rates and enhancing conversational ability and domain generalizability. Nonetheless, current retrieval-augmented generation (RAG) techniques assume all conversations require external knowledge, which can lead to unnecessary and irrelevant information being included in responses.
RAGate, a gating model proposed by the authors, controls input and memory in a similar manner to long-short term memory (LSTM) models. Using a binary knowledge gate mechanism, RAGate manipulates external knowledge for conversational systems, predicting the need for RAG based on the context of conversations, and evaluating relevant inputs. The authors examine three RAGate variants: RAGate-Prompt (utilizing a pre-trained language model and devised prompts to adapt to new tasks), RAGate-PEFT (using fine-tuning methods such as QLoRA to train models) and RAGate-MHA (introducing a neural encoder alongside attention techniques to model context and evaluate need for augmentation).
Conducted on an annotated Task-Oriented Dialogue dataset (KETOD), experiments proved successful — RAGate efficiently used external knowledge to produce high-quality responses. The constant inclusion of external knowledge was found to increase the risk of hallucination, to which the model was effective in reducing, controlling conversation systems to create confident and informative responses. The paper also found a correlation between the confidence score and relevance of augmented knowledge. Therefore, dynamically determining the need for RAG augmentation based on confidence levels could yield more accurate and relevant results, enhancing the overall user experience.
Overall, RAGate provides an effective solution to the ongoing challenge of when to use external knowledge augmentation in conversational systems. The adaptation of human judgments and advanced language models contribute greatly to the efficiency and performance of RAG techniques and look set to revolutionize the future of proficient conversational systems.
