Omost is an innovative project aimed at improving the image generation capabilities of Large Language Models (LLMs). The technology essentially converts the programming ability of an LLM into advanced image composition skills. The concept behind Omost’s name is two-fold; firstly, after its use, the produced image should be ‘almost’ perfect. Secondly, ‘O’ stands for ‘omni,’ meaning multi-modal, while ‘most’ signifies the extraction of maximum potential from the technology.
Omost tool enables LLMs to write code that compounds visual content on a virtual Canvas agent. This Canvas can then be made visible using specific image generators’ implementations to produce actual images. As of now, Omost furnishes three pretrained LLM models such as omost-llama-3-8b, omost-dolphin-2.9-llama3-8b, and omost-phi-3-mini-128k, which are trained using versatile datasets, including annotated data from various sources, data from automatic image annotation, reinforcement learning through Direct Preference Optimization (DPO), to list some.
To use Omost, one can either visit the official HuggingFace space or deploy it locally. The Canvas agent is pivotal for Omost’s image composition, providing functions to set global and local image descriptions. Image composition parameters include descriptions that describe elements separately, setting location and offset for bounding image elements, specifying distance to viewer, and defining color using HTML web color names.
The platform also provides a foundational renderer based on attention manipulation, offering methods for region-guided diffusion. These methods include Multi-Diffusion, Attention Decomposition, Attention Score Manipulation, Gradient Optimization, and External Control Models. Moreover, Omost has some experimental features like Prompt Prefix Tree, which enhances prompt understanding by merging sub-prompts into coherent descriptions. Experimental parameters like tags, atmosphere, style, and quality meta are also there to better the overall quality and atmosphere of the created image.
In conclusion, Omost’s introduction is a monumental leap in leveraging LLMs for complex image composition. It combines powerful coding capabilities with advanced rendering techniques, permitting users to generate high-quality images with detailed descriptions and exact control over visual elements. Omost opens up opportunities for users, whether via the official HuggingFace space or local deployment, to employ a powerful set of tools to create captivating visual content.
