Computer vision

Researchers have noted gaps in the evaluation methods for Large Vision Language Models (LVLMs). Primarily, they note that evaluations overlook the potential of visual content being unnecessary for many samples, as well as the risk of unintentional data leakage during training. They also indicate the limitations of single-task benchmarks for accurately assessing the multi-modal capabilities…

The ability of large Multimodal Language Models (MLLMs) to tackle visual math problems is currently the subject of intense interest. While MLLMs have performed remarkably well in visual scenarios, the extent to which they can fully understand and solve visual math problems remains unclear. To address these challenges, frameworks such as GeoQA and MathVista have…

Researchers from the Max Planck Institute for Intelligent Systems, Adobe, and the University of California have introduced a diffusion image-to-video (I2V) framework for what they call training-free bounded generation. The approach aims to create detailed video simulations based on start and end frames without assuming any specific motion direction, a process known as bounded generation,…

The exponential advancement of Multimodal Large Language Models (MLLMs) has triggered a transformation in numerous domains. Models like ChatGPT- that are predominantly constructed on Transformer networks billow with potential but are hindered by quadratic computational complexity which affects their efficiency. On the other hand, Language-Only Models (LLMs) lack adaptability due to their sole dependence on…

Researchers from Alibaba Group and the Renmin University of China have developed an advanced version of MultiModal Large Language Models (MLLMs) to better understand and interpret images rich in text content. Named DocOwl 1.5, this innovative model uses Unified Structure Learning to enhance the efficiency of MLLMs across five distinct domains: document, webpage, table, chart,…

Large language models like GPT-4, while powerful, often struggle with basic visual perception tasks such as counting objects in an image. This can be due to the way these models process high-resolution images. Current AI systems can mainly perceive images at a fixed low resolution, leading to distortion, blurriness, and loss of detail when the…

In the ever-evolving fields of computer vision and artificial intelligence, traditional methodologies favor larger models for advanced visual understanding. The assumption underlying this approach is that larger models can extract more powerful representations, prompting the construction of enormous vision models. However, a recent study challenges this wisdom, with a closer look at the practice of…