Large language models (LLMs) have gained significant popularity recently, but evaluating them can be quite challenging, particularly for highly specialised client tasks requiring domain-specific knowledge. Therefore, Amazon researchers have developed a new evaluation approach for Retrieval-Augmented Generation (RAG) systems, focusing on such systems' factual accuracy, defined as their ability to retrieve and apply correct information…
DVC.ai has introduced DataChain, a pioneering open-source Python library fashioned to manage and curate massive-scale, unstructured data. By integrating advanced AI and machine learning abilities, DataChain aims to enhance the data processing workflow—making it an essential tool for data scientists and developers.
DataChain's chief features encompass AI-driven data curation, but it also employs local machine learning…
Reinforcement Learning from Human Feedback (RLHF) plays a pivotal role in ensuring the quality and safety of Large Language Models (LLMs), such as Gemini and GPT-4. However, RLHF poses significant challenges, including the risk of forgetting pre-trained knowledge and reward hacking. Existing practices to improve text quality involve choosing the best output from N-generated possibilities,…
Large Language Models (LLMs) have improved significantly, but challenges persist, particularly in the prefilling stage. This is because the cost of computing attention increases with the number of tokens in the prompts, leading to a slow time-to-first-token (TTFT). As such, optimizing TTFT is crucial for efficient LLM inference.
Various methods have been proposed to improve…
Before the development of PILOT (PIecewise Linear Organic Tree), linear model trees were slow to fit and susceptible to overfitting, notably with large datasets. The traditional regression trees faced challenges capturing linear relationships efficiently. Linear model trees also encountered problems with interpretability when integrating linear models in leaf nodes. The research points out the need…
Multi-target multi-camera tracking (MTMCT) has become indispensable in intelligent transportation systems, yet real-world applications are complex due to a shortage of publicly available data and laborious manual annotation. MTMCT involves tracking vehicles across multiple camera lenses, detecting objects, carrying out multi-object tracking, and finally clustering trajectories to generate a comprehensive image of vehicle movement. MTMCT…
In the domain of visual question answering (VQA), the Multi-Image Visual Question Answering (MIQA) remains a major hurdle. It entails generating pertinent and grounded responses to natural language prompts founded on a vast assortment of images. While large multimodal models (LMMs) have proven competent in single-image VQA, they falter when dealing with queries involving an…
