Large language models (LLMs) have gained significant popularity recently, but evaluating them can be quite challenging, particularly for highly specialised client tasks requiring domain-specific knowledge. Therefore, Amazon researchers have developed a new evaluation approach for Retrieval-Augmented Generation (RAG) systems, focusing on such systems' factual accuracy, defined as their ability to retrieve and apply correct information…
DVC.ai has introduced DataChain, a pioneering open-source Python library fashioned to manage and curate massive-scale, unstructured data. By integrating advanced AI and machine learning abilities, DataChain aims to enhance the data processing workflow—making it an essential tool for data scientists and developers.
DataChain's chief features encompass AI-driven data curation, but it also employs local machine learning…
Reinforcement Learning from Human Feedback (RLHF) plays a pivotal role in ensuring the quality and safety of Large Language Models (LLMs), such as Gemini and GPT-4. However, RLHF poses significant challenges, including the risk of forgetting pre-trained knowledge and reward hacking. Existing practices to improve text quality involve choosing the best output from N-generated possibilities,…
The United Nations' Sustainable Development Goals (SDGs) aim to eradicate poverty, protect the environment, combat climate change, and bolster global peace and prosperity by the year 2030. Despite extensive research, additional work is required to accurately forecast SDG scores, which measure progress towards these objectives. By employing ARIMAX and Linear Regression machine learning models which…
Large Language Models (LLMs) have improved significantly, but challenges persist, particularly in the prefilling stage. This is because the cost of computing attention increases with the number of tokens in the prompts, leading to a slow time-to-first-token (TTFT). As such, optimizing TTFT is crucial for efficient LLM inference.
Various methods have been proposed to improve…
Before the development of PILOT (PIecewise Linear Organic Tree), linear model trees were slow to fit and susceptible to overfitting, notably with large datasets. The traditional regression trees faced challenges capturing linear relationships efficiently. Linear model trees also encountered problems with interpretability when integrating linear models in leaf nodes. The research points out the need…
Multi-target multi-camera tracking (MTMCT) has become indispensable in intelligent transportation systems, yet real-world applications are complex due to a shortage of publicly available data and laborious manual annotation. MTMCT involves tracking vehicles across multiple camera lenses, detecting objects, carrying out multi-object tracking, and finally clustering trajectories to generate a comprehensive image of vehicle movement. MTMCT…