The rapid improvement of large language models and their role in natural language processing has led to challenges in incorporating less commonly spoken languages. Embedding the majority of artificial intelligence (AI) systems in well-known languages inevitably forces a technological divide across linguistic communities that remains mostly unaddressed.
This paper introduces the SambaLingo system, a novel…
Digital agents, or software designed to streamline interactions between humans and digital platforms, are becoming increasingly popular due to their potential to automate routine tasks. However, a consistent challenge with these agents is their frequent misunderstanding of user commands or inability to adapt to new or unique environments—problems that can lead to errors and inefficiency.…
Memory is a crucial component of intelligence, facilitating the recall and application of past experiences to current situations. However, both traditional Transformer models and Transformer-based Large Language Models (LLMs) have limitations related to context-dependent memory due to the workings of their attention mechanisms. This primarily concerns the memory consumption and computation time of these attention…
Recent research in Artificial Intelligence (AI) has shown a growing interest in the capabilities of large language models (LLMs) due to their versatility and adaptability. These models, traditionally used for tasks in natural language processing, are now being explored for potential use in computational tasks, such as regression analysis. The idea behind this exploration is…
Chain-of-thought (CoT) prompting, an instruction method for language models (LMs), seeks to improve a model's performance across arithmetic, commonsense, and symbolic reasoning tasks. However, it falls short in larger models (with over 100 billion parameters) due to its repetitive rationale and propensity to produce unaligned rationales and answers.
Researchers from Penn State University and Amazon AGI…
Large Language Models (LLMs), known for their key role in advancing natural language processing tasks, continue to be polished to better comprehend and execute complex instructions across a range of applications. However, a standing issue is the tendency for LLMs to only partially follow given instructions, a shortcoming that results in inefficiencies when the models…
Advancements in multimodal architectures are transforming how systems process and interpret complex data. These technologies enable concurrent analyses of different data types such as text and images, enhancing AI capabilities to resemble human cognitive functions more precisely. Despite the progress, there are still difficulties in efficiently and effectively merging textual and visual information within AI…
Large Language Models (LLMs) have surpassed previous generations of language models on various tasks, sometimes even equating or surpassing human performance. However, it's challenging to evaluate their true capabilities due to potential contamination in testing datasets or a lack of datasets that accurately assess their abilities.
Most studies assessing LLMs have focused primarily on the English…
Large Language Models (LLMs) have become a crucial tool in artificial intelligence, capable of handling a variety of tasks, from natural language processing to complex decision-making. However, these models face significant challenges, especially regarding data memorization, which is pivotal in generalizing different types of data, particularly tabular data.
LLMs such as GPT-3.5 and GPT-4 are effective…
Neural network models are dominant in the areas of natural language processing and computer vision. However, the initialization and learning rates of these models often depend on heuristic methods, which can lead to inconsistencies across different studies and model sizes. The µ-Parameterization (µP) seeks to address this issue by proposing scaling rules for model parameters…
Artificial intelligence continues to transform scientific research and engineering design, presenting a faster and cost-effective alternative to physical experiments. Researchers from NVIDIA and Caltech are at the forefront, devising a new method that upends traditional numerical simulations using neural operators, providing enhanced efficiency in modeling complex systems. This innovative approach aids in addressing some of…