This post discusses an error encountered during the operation of the RunPod.io platform—which is used to learn and train LoRA building infrared sensors—specifically while using the Kohya_ss script to check latents.
The error occurred immediately after a cache validity check. This was completed successfully, but following that the program attempted to cache latents again, and this is when the error occurred. The error message indicated that when unpacking 4 bytes of data, a buffer of at least 4 bytes is required. However, the actual buffer size at this instance was reportedly zero.
Upon further investigation, it was found that the error was linked to an image file being truncated. An attempt to convert the image to bytes led to a confirmation that the image was indeed truncated. Additional checks performed through the software’s inbuilt tools such as the accelerate command supported this conclusion and no solution was presented within the error message itself.
The error persisted through various iterations and instances of Kohya_SS on RunPod.io, including in pre-configured pod Stable Diffusion Kohya_ss ComfyUI, leading to a consistent problem that needed to be resolved for the platform to be usable.
It was noted that the cause of the problem was the SDXL_train_network.py file, located in the root of the Kohya_ss installation. The solution suggested was to edit this file and add two lines at the beginning: “from PIL import ImageFile” and “ImageFile.LOAD_TRUNCATED_IMAGES = True”.
This essentially allows for images that were truncated to be processed as normal, enabling the operator to continue with the Kohya_ss Training script without further interruption. Following these changes, running the Kohya_ss Training script or command line should proceed successfully, remedying the error.
Overall, this post provides a technical account of a common error faced by users of RunPod.io when checking latents with the Kohya_ss script, and presents a solution allowing users to carry on with their operations without further obstacles.
