Large Language Model

In recent years, there has been increasing attention paid to the development of Small Language Models (SLMs) as a more efficient and cost-effective alternative to Large Language Models (LLMs), which are resource-heavy and present operational challenges. In this context, researchers from the Department of Computer Science and Technology at Tsinghua University and Modelbest Inc. have…

Researchers from Meta/FAIR Labs and Mohamed bin Zayed University of AI have carried out a detailed exploration into the scaling laws for large language models (LLMs). These laws delineate the relationship between factors such as a model's size, the time it takes to train, and its overall performance. While it’s commonly held that larger models…

The field of Natural Language Processing (NLP) has witnessed a radical transformation following the advent of Large Language Models (LLMs). However, the prevalent Transformer architecture used in these models suffers from quadratic complexity issues. While techniques such as sparse attention have been developed to lower this complexity, a new generation of models is making headway…

Causal learning plays a pivotal role in the effective operation of artificial intelligence (AI), helping improve AI models' ability to rationalize decisions, adapt to new data, and visualize hypothetical scenarios. However, the evaluation of large language models' (LLM) proficiency in processing causality, such as GPT-3 and its variants, remains a challenge due to the need…

Large language models (LLMs), crucial for various applications such as automated dialog systems and data analysis, often struggle in tasks necessitating deep cognitive processes and dynamic decision-making. A primary issue lies in their limited capability to engage in significant reasoning without human intervention. Most LLMs function on fixed input-output cycles, not permitting mid-process revisions based…

Large language models (LLMs) paired with tree-search methodologies have been leading advancements in the field of artificial intelligence (AI), particularly for complex reasoning and planning tasks. These models are revolutionizing decision-making capabilities across various applications. However, a notable imperfection lies in their inability to learn from prior mistakes and frequent error repetition during problem-solving. Improving the…

Large Language Models (LLMs) have taken center stage in many intelligent agent tasks due to their cognitive abilities and quick responses. Even so, existing models often fail to meet demands when negotiating and navigating the multitude of complexities on webpages. Factors such as versatility of actions, HTML text-processing constraints, and the intricacy of on-the-spot decision-making…

Natural Language Processing (NLP) has traditionally centered around English language models, thereby excluding a significant portion of the global population. However, this status quo is being challenged by the Chinese Tiny LLM (CT-LLM), a groundbreaking development aimed at a more inclusive era of language models. CT-LLM, innovatively trained on the Chinese language, one of the…