Nexusflow has recently launched Athene-Llama3-70B, a high-performance open-weight chat model that's been fine-tuned from Meta AI's earlier model, Llama-3-70B. The improvement in terms of performance is quite significant with the new model achieving an impressive Arena-Hard-Auto score of 77.8%, surpassing models like GPT-4o and Claude-3.5-Sonnet. This is a substantial improvement from Llama-3-70B-Instruct, the predecessor which…
The article introduces a benchmark known as ZebraLogic, which assesses the logical reasoning capabilities of large language models (LLMs). Using Logic Grid Puzzles, the benchmark measures how well LLMs can deduce unique value assignments for a set of features given specific clues. The unique value assignment task mirrors those that are often found in assessments…
Language Learning Models (LLMs) that are capable of interpreting natural language instructions to complete tasks are an exciting area of artificial intelligence research with direct implications for healthcare. Still, theypresent challenges as well. Researchers from Northeastern University and Codametrix conducted a study to evaluate the sensitivity of various LLMs to different natural language instructions specifically…
ChatGPT, an AI system by OpenAI, is making waves in the artificial intelligence field with its advanced language capabilities. Capable of performing tasks such as drafting emails, conducting research, and providing detailed information, such tools are transforming the way office tasks are conducted. They contribute to more efficient and productive workplaces. As with any technological…
Large language models (LLMs) are exceptional at generating content and solving complex problems across various domains. Nevertheless, they struggle with multi-step deductive reasoning — a process requiring coherent and logical thinking over extended interactions. The existing training methodologies for LLMs, based on next-token prediction, do not equip them to apply logical rules effectively or maintain…
The evaluation of large language models (LLMs) has always been a daunting task due to the complexity and versatility of these models. However, researchers from Google DeepMind, Google, and UMass Amherst have introduced FLAMe, a new family of evaluation models developed to assess the reliability and accuracy of LLMs. FLAMe stands for Foundational Large Autorater…
Language models (LMs), used in applications such as autocomplete and language translation, are trained on a vast amount of text data. Yet, these models also face significant challenges in relation to privacy and copyright concerns. In some cases, the inadvertent inclusion of private and copyrighted content in training datasets can lead to legal and ethical…
Researchers at the University of Texas (UT) in Austin have introduced a new benchmark designed to evaluate the effectiveness of artificial intelligence in solving complex mathematical problems. PUTNAMBENCH is aimed at solving a key issue facing the sector as current benchmarks are not sufficiently rigorous and mainly focus on high-school level mathematics.
Automating mathematical reasoning…
DeepSeek has announced the launch of its advanced open-source AI model, DeepSeek-V2-Chat-0628, on Hugging Face. The update represents a significant advancement in AI text generation and chatbot technology. This new version secures the overall ranking of #11 according to the LMSYS Chatbot Arena Leaderboard, outperforming all other existing open-source models. It is an upgrade on…
Large Language Models (LLMs) are vital for tasks in natural language processing but they encounter issues when it comes to deployment. This is due to their substantial computational and memory requirements during inference. Current research studies are focused on boosting LLM efficiency by applying methods such as quantization, pruning, distillation, and improved decoding. One of…
Generative artificial intelligence (AI) technologies, like Large Language Models (LLMs), are showing promise in areas like programming processes, customer service productivity, and collaborative storytelling. However, their impact on human creativity, a cornerstone of our behavior, is still somewhat unknown. To investigate this, a research team from the University College London and the University of Exeter…
Large language models (LLMs) are being extensively used in multiple applications. However, they have a significant limitation: they struggle to process long-context tasks due to the constraints of transformer-based architectures. Researchers have explored various approaches to boost LLMs' capabilities in processing extended contexts, including improving softmax attention, reducing computational costs and refining positional encodings. Techniques…