Computer vision

Computer vision is a rapidly growing field that enables machines to interpret and understand visual data. This technology involves various tasks like image classification, object detection, and more, which require balancing local and global visual contexts for effective processing. Conventional models often struggle with this aspect; Convolutional Neural Networks (CNNs) manage local spatial relationships but…

Text-to-image generation models, such as DALLE-3 and Stable Diffusion, are increasingly being used to generate detailed and contextually accurate images from text prompts, thanks to advancements in AI technology. However, these models face challenges like misalignment, hallucination, bias, and the creation of unsafe or low-quality content. Misalignment refers to the discrepancy between the image produced…

DeepMind researchers have unveiled a new model, PaliGemma, pushing forward the evolution of vision-language models. The new model successfully integrates the strengths of both the PaLI vision-language model series and the Gemma family of language models. PaliGemma is an example of a sub-3B vision-language model that uses a 400M SigLIP vision model along with a…

DeepMind researchers have developed an open vision-language model called PaliGemma, blending the strengths of the PaLI vision-language model series with Gemma family of language models. This model merges a 400 million SigLIP vision model with a 2 billion Gemma language model, creating a compact vision-language model that can compete with larger predecessors such as PaLI-X,…

Vision-Language Models (VLMs) offer immense potential for transforming various applications, including visual assistance for visually impaired individuals. However, their efficacy is often marred by complexities such as multi-object scenarios and diverse cultural contexts. Recent research highlights these issues in two separate studies focused on multi-object hallucination and cultural inclusivity. Hallucination in vision-language models occurs when objects…

Text-to-3D generation technology is becoming increasingly influential across various fields such as video games, augmented reality, and virtual reality. The process creates detailed 3D content from text descriptions, which was traditionally a laborious and expensive task requiring a significant amount of effort from skilled artists. By automating this process with AI technology, it becomes a…

The computer vision sector is currently dominated by large-scale models that offer remarkable performance but demand high computational resources, making them impractical for real-world applications. To address this, the Google Research Team has opted to reduce these models into smaller, more efficient architectures via model pruning and knowledge distillation. The team's focus is on knowledge…

Researchers from Shanghai Jiaotong University, Shanghai AI Laboratory, and Nanyang Technological University's S-Lab have developed an advanced multi-modal large language model (MLLM) called MG-LLaVA. This new model aims to overcome the limitations of current MLLMs when interpreting low-resolution images. The main challenge with existing MLLMs has been their reliance on low-resolution inputs which compromises their…

Deep learning models such as Convolutional Neural Networks (CNNs) and Vision Transformers have seen vast success in visual tasks like image classification, object detection, and semantic segmentation. However, their ability to accommodate different data changes, particularly in security-critical applications, is a significant concern. Many studies have assessed the robustness of CNNs and Transformers against common…

The field of research that aims to enhance large multimodal models (LMMs) to effectively interpret long video sequences faces challenges stemming from the extensive amount of visual tokens vision encoders generate. These visual tokens pile up, particularly with LLaVA-1.6 model, which generates between 576 and 2880 visual tokens for one image, a number that significantly…