Behold the power of AI and Machine Learning! Researchers from the Institute for Intelligent Computing and Alibaba Group have come together to revolutionize 3D character creation with their invention of ‘Mach’ (Make-A-Character). This innovative system simplifies the creation of detailed and realistic 3D avatars, making it easier to generate personalized avatars that match the user’s envisioned persona. Leveraging advanced language and vision models, the user can transform basic text descriptions into detailed 3D avatars effortlessly. With the help of Triplane, geometry generation has been made easier and camera parameters have been optimized, in addition to dense facial landmarks and a reference image. Furthermore, their data diversity is augmented with Individual Typology Angle (ITA) and High Dynamic Range (HDR) lights, which cover a wide range of natural lighting conditions.
The 2D face parsing and 3D generation modules enable the mesh and textures of the target face, along with additional matched accessories, to be generated easily. The process also employs differentiable rendering and enhancement methods to extract and perfect the diffuse texture using a reference image, as well as strand-based hair generation guided by hairstyle images generated using SD models. Additionally, accessories like garments, glasses, eyelashes, and irises are sourced from a tagged 3D asset library, and their semantic attributes are extracted before they are assembled to create a comprehensive 3D figure.
The results of the generated 3D avatars exhibit realistic textures and geometry. The facial rig control further showcased expressive animations, while Stable Diffusion Model, LLM, and ControlNet enabled the generation of detailed facial attributes. Ultimately, the research highlights the importance of dense facial landmarks for accurately reconstructing the face and head structure. Mach utilizes synthetic images for training data, and establishes a multi-view capturing and processing pipeline to produce uniformly topological head scans.
In summary, this amazing study has put forward a method for generating detailed 3D avatars with realistic textures and geometry, guided by text prompts and dense facial landmarks. Be sure to join our 35k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, LinkedIn Group, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more. Don’t forget to check out this incredible paper and subscribe to our newsletter to stay updated with all the latest AI developments!
