Creating comprehensive and detailed outlines for long-form articles such as those found on Wikipedia is a considerable challenge due to issues in capturing the full depth of the topic, thus leading to shallow or poorly structured articles. This pivotal problem originates from systems’ inability to ask the correct queries and source information from a variety of perspectives, thus failing to create a thoroughly-rounded article.
Existing solutions such as retrieval-augmented generation (RAG) models attempt to solve this issue by uniting external information retrieval and language model competencies. However, these solutions can struggle with creating various queries and organizing the sourced information in a comprehensible manner. The RAG models may create overly generic questions and overlook crucial specifics or fail to take into account different perspectives, thus leading to the creation of insufficiently comprehensive articles.
To address these issues, researchers at Stanford have developed a new artificial intelligence system. This system, STORM (Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking), considerably enhances the research abilities of large language models. It allows them to generate detailed and thorough summaries for long articles. STORM functions on two central principles: more diverse perspectives lead to varied queries, and detailed queries necessitate iterative research. Leaning on these principles, STORM can create richer and more insightful questions, which ultimately lead to better structured and more detailed articles.
STORM’s method comprises several vital stages including perspective discovery, query generation, and creating a structured outline. The system accomplishes perspective discovery by retrieving and examining related Wikipedia topics to unearth diverse views. It then creates questions by adopting these specific views, which results in a broad range of possible inquiries. These newly generated queries are then perfected through multi-turn dialogues, when the system simulates conversation by referring to the information sourced from the internet. After completing these steps, STORM generates a structured framework reliant on collected information and the inherent knowledge of the language model.
To assess its effectiveness, the researchers used the FreshWiki dataset, which is a compilation of recent, high-quality Wikipedia articles. The evaluation metrics concentrated on the quality of the outline, its breadth, organization, and relevance vis-à-vis articles written by humans. Both automatic and human evaluations demonstrated that STORM outclassed traditional RAG models, particularly in terms of the breadth and organization of the articles. Results show the ability of STORM to generate comprehensive and highly detailed outlines.
While these improvements are significant, STORM faces challenges like bias in sources and the over-association of unrelated facts. Addressing these issues will be crucial in boosting the system’s performance. Despite these issues, STORM epitomizes a robust system for automating the creation of long articles, especially the pre-writing stage. It underscores the importance of multi-perspective research and iterative process in producing comprehensive and well-structured article outlines, thereby setting a new yardstick for grounded long-form content creation.
