Artificial intelligence models, in particular large language models (LLMs), have made significant strides in generating coherent and contextually appropriate language. However, they sometimes create content that seems correct but is actually inaccurate or irrelevant, a problem often referred to as “hallucination”. This can pose a considerable issue in areas where high factual accuracy is critical, such as in medical or financial fields. As such, there is a pressing need for effective ways to detect and manage these inaccuracies to maintain the reliability of AI-produced information.
Previous methods to tackle this problem have included internal consistency checks where AI responses are matched against each other to spot contradictions. Some researchers have also explored the AI’s hidden states or output probabilities to identify potential errors. However, these solutions often rely exclusively on the AI’s internal data, which can be limited and not always up-to-date. A more recent approach involved fact-checking after the fact (post-hoc), utilizing external data sources to improve accuracy.
Researchers from the University of Illinois Urbana-Champaign, UChicago, and UC Berkeley have now developed a novel technique dubbed KnowHalu. The method incorporates a two-phase process to detect errors in AI-generated text to enhance accuracy. It begins by checking for “non-fabrication hallucinations”, accurate responses that do not aptly address the query. The second phase employs a more detailed strategy using structured and unstructured external knowledge sources for an in-depth factual analysis.
KnowHalu’s strategy involves a multi-step process starting with breaking down the original query into simpler sub-queries which enable targeted retrieval of applicable information from different knowledge bases. Each data point is then evaluated through a comprehensive judging mechanism considering various forms of knowledge, including semantic sentences and knowledge triplets. This multi-form knowledge analysis offers detailed factual validation, considerably enhancing the AI’s reasoning capabilities and accuracy.
KnowHalu’s effectiveness is illustrated through rigorous testing across different tasks such as question-answering and text summarization. The results demonstrate significant improvements in detecting inaccuracies, outperforming the current state-of-the-art methods by considerable margins. Specifically, KnowHalu achieved a 15.65% improvement in question-answering accuracy and a 5.50% increase in text summarization accuracy compared to previous techniques.
In summary, KnowHalu represents a substantial advancement in AI, effectively dealing with the issue of hallucinations in text generated by LLMs. It boosts the accuracy and reliability of AI applications, expanding their potential use in critical and information-sensitive sectors. With its innovative approach and demonstrated effectivity, KnowHalu sets a new standard in verifying and trusting AI-generated content, paving the future for safer AI interactions across various domains.
