(October 29th, 2023)Development of a Customized Lora: A Deep Dive into Personalized AI-Generated Imagery

 Development of a Customized Lora: A Deep Dive into Personalized AI-Generated Imagery

October 29th, 2023

Objective:

The primary objective of this project was to create a professional-grade photograph using a custom AI model, termed "Lora." This initiative was inspired by Aragon (https://www.aragon.ai/), a commercial AI tool available at Aragon AI, which offers similar capabilities but at a cost.

Methodology:

The project began with the compilation of two image datasets. The initial dataset lacked diversity, consisting of overly similar images, which necessitated the creation of a second, more varied dataset. Both the standard definition (SD) and extra-large (XL) versions of Lora were trained using Kohya's framework, renowned for its robustness and versatility in managing advanced AI models.

Challenges and Results:

The anticipated high-quality output from the XL Lora did not meet expectations initially, as the images suffered from pixelation and blurriness. Adjusting the sampler settings to their maximum helped improve the quality, though it remained below the expected standard. Nonetheless, the model successfully captured and replicated my distinct facial features and hairstyle, indicating its potential to learn and reproduce personal characteristics.

(2nd attempt) Training SDXL Lora using Kohya

https://github.com/bmaltais/kohya_ss 

Using the advanced capabilities of Kohya's framework (detailed on GitHub), the second attempt involved training an enhanced version, SDXL Lora. Despite hopes for superior output, the images remained blurry and pixelated at standard sampler settings. Increasing the sampler intensity only marginally improved the image quality.

Image Dataset

Result: 

I made an XL lora. I wasn’t happy with the result because the quality of the images were bad. They were pixelated, blurry even though I made an XL lora. I thought the quality would be significantly better. I had to put the sampler all the way up to get a photo with this quality:

Otherwise, I would get something with this quality with the sampler at ~ 25:

One thing that I did like about the images was that it created various pictures of me that looked similar to me. It did a good job at generating images that had my facial features and hair that I trained with. Below you can see all of the images I generated:

(1st attempt) Training SD Lora using Kohya

The first training session aimed to generate a diverse range of facial expressions, backgrounds, and attire. However, the model predominantly produced variations of the existing dataset without significant diversity in expressions or settings, highlighting the importance of a varied training set for achieving more dynamic results.

Image Dataset

Result

What I didn’t like about the result is that it mainly generated variations of my image dataset. I was hoping it would be able to create various facial expressions, backgrounds and clothes but it had a hard time with that. When providing an image dataset, it is important to provide various backgrounds and clothes in addition to various facial expressions.