The process began with an attempt to train AnimateDiff Motion LoRAs with the AnimateDiff-MotionDirector repository. However, the installation process proved difficult with numerous errors, including a daunting 40GB clone from huggingface.co, with unclear file requirements leading to potential duplicate downloads. After spending hours on Runpod and unsuccessfully trying to accomplish the tasks on a local computer, the repository was abandoned in disappointment.
After an online search, a helpful video detailed an easy process using ComfyUI. After downloading a workflow from the video description and importing it into ComfyUI, the process became more manageable. Any missing nodes could be installed via the ComfyUI Manager. The workflow instructions were understandable, and the default settings worked well. Necessary files were moved into relevant folders, and “Refresh” ensured ComfyUI registered these updates.
The AnimateDiff video was then completed. A video was uploaded onto the Load Video node and the appropriate models were selected. A prompt was written to describe the video and another to direct AnimateDiff to generate a resulting video with the Motion LoRA. The Motion LoRA was named accordingly.
The workflow was queued and left to ComfyUI. Different training frames were attempted, with a maximum of 32 frames before GPU memory limits were reached. The training duration varied depending on frame number, taking 20 minutes for 16 frames and 25 minutes for 24 frames.
Once the process was completed, the Motion LoRA was found in the AnimateDiff Motion LoRA folder under sections labelled with the date, time and LoRA name. The workflow also produced a comparison video. The default settings resulted in LoRA files of 128MB each.
This alternative process using ComfyUI proved efficient and provided high quality results, evident in the completed animation. Sample results of the successful Motion LoRAs using this process can also be found on personal blog and store links. The initial daunting task turned into an efficient and rewarding process. Thanks to this easy and effective method, new Motion LoRAs can be created swiftly and efficiently with impressive results.
