Open-source large multimodal models (LMMs), such as LLaVA, CogVLM, and DreamLLM, which primarily handle multimodal understanding without generation capabilities, currently face significant limitations. They often lack the native integration required to align visual representations with pre-trained language models, leading to complexity and inefficiency in both training and inference time. Moreover, many are either restricted to…
