- This topic has 104 replies, 7 voices, and was last updated 2 years, 1 month ago by
.
-
Posts
-
Hi gts6, what's your opinion regarding LoRa training vs Dreambooth model fine-tuning in Kohya? I ask because the best (IMO) result I've achieved so far was fine-tuned using EpicPhotogasm. I've heard some say that quality is significantly better using Dreambooth (fine tuning) vs training a lora. I'm aware of the cons... large file size, VRAM use, etc.
What has been your experience with these two methods?
I believe that extracting a LoRA from custom Dreambooth models in Kohya is no longer supported.@frank22 I've never actually done anything with Dreambooth. I pretty much use RealisticVisionV60B1
checkpoint with the above config and strategy. I've always had pretty good results that i've been happy with, so never really needed to experiment further. I did forget to mention what checkpoint I was using earlier. So if you haven't gotten satisfactory results, you can try that checkpoint. When rendering aftwards, I use DPM++ 2M SDE sampling method, 50 sampling steps, hires fix 4x Ultrasharp (sometimes I use scunet or ersgan 4x+ for a different look), Hires steps 10, denoising strength .16, upscale by 1.2thats the "usual", sometimes I use completely different settings, but generally I like this for the "real" renders after creating the lora
RealisticVisionV60B1 looks like it produces images very similar to epiCPhotoGasm3.1. I find that CyberRealistic has become outdated and not so good at realistic or cinematic lighting. IMO 4x_NMKD-Superscale-SP178000_G also produces good upscaling but 4x Ultrasharp is still my favourite.
The rest of your settings and workflow are pretty much the same as mine.
All of my LoRas so far have been of known people/celebrities because without that you can't really judge how good the results of your settings are. I guess we all want the best LoRas without waiting 2-3 hours for them to process.
Question - you said before that "Epochs do not affect the result AT ALL". But doesn't each epoch work from the preceding one and use it to refine and improve upon it?Yes the epochs effect the result in small steps along the way, but it doesnt effect it in any different way if you hadnt used them at all, if that makes sense. IT probably doesnt lol. You can achieve the same result either way, epochs gradually. Its very hard to explain, but epochs are periods of time, basically.
When I say an epoch doesnt affect the result, I mean it as it wont affect it the same way like a CFG or sample steps or denoising. With epoch you will achieve the same result with or without, when the times are the same.
Yep, I've seen Youtube tutes where 1 epoch is suggested, but as you say that would leave no room to maneuver if the LoRa is over/under trained.
Up to using your config I was using one that took 90-120 minutes to process in Kohya. The "basic settings" (not low VRAM) Aitrepreneur linked in his tute, yeh I know it old now. The config worked great but it took a long time if you increased epochs to 3-5.
I guess using 100 repeats/1 epoch is slower than 5 repeats/40 epochs, even though the steps are calculated on the same basis. Is there anything else in the config that would be slowing it down? I hate to keep pestering you but "I'm on a quest" LOL.
@frank22 Is your quest looking for quicker speed or different results?
As for speed, the samples slow it down if doing too many or too often (as in samples for every single epoch when theres many of them). The train process stops and does the samples renders. They are necessary though, but a ton of them is not.Im not sure what could be slowing it down on you, I usually can train a lora pretty quick on mine with these settings. Havent really timed them because I usually just let it run and go in the other room for a while, but id say usually half hour or less. It prob depends on your PC power a lot more than a config setting
I like the results I got with Aitrepreneur's settings but it's also quite slow, 60-90 minutes depending on the number of epochs.
From what I can see, for 30ish images he uses 100 repeats, batch size 2 and 1 epoch. If I use 3-4 epochs and 768 size images it's really slow. I guess it's to be expected with larger images and more epochs.
Is there something else in his config that could be changed to speed things up a bit without impacting on the LoRa quality? I suspect it's just how Kohya works and I'll have to live with it. I see there's people quite happy with 4+ hours LoRa training so maybe I shouldn't complain.
As I mentioned before I'm on a 13700K, and 12GB RTX4070.{
"pretrained_model_name_or_path": "runwayml/stable-diffusion-v1-5",
"v2": false,
"v_parameterization": false,
"logging_dir": "C:/Users/ohmni/AI/SAMPLE IMAGES/ADAMS/LORA OUTPUT/log",
"train_data_dir": "C:/Users/ohmni/AI/SAMPLE IMAGES/ADAMS/LORA OUTPUT/img",
"reg_data_dir": "",
"output_dir": "C:/Users/ohmni/AI/SAMPLE IMAGES/ADAMS/LORA OUTPUT/model",
"max_resolution": "512,512",
"learning_rate": "0.0001",
"lr_scheduler": "constant",
"lr_warmup": "0",
"train_batch_size": 2,
"epoch": "1",
"save_every_n_epochs": "1",
"mixed_precision": "bf16",
"save_precision": "bf16",
"seed": "1234",
"num_cpu_threads_per_process": 2,
"cache_latents": true,
"caption_extension": ".txt",
"enable_bucket": false,
"gradient_checkpointing": false,
"full_fp16": false,
"no_token_padding": false,
"stop_text_encoder_training": 0,
"use_8bit_adam": true,
"xformers": true,
"save_model_as": "safetensors",
"shuffle_caption": false,
"save_state": false,
"resume": "",
"prior_loss_weight": 1.0,
"text_encoder_lr": "5e-5",
"unet_lr": "0.0001",
"network_dim": 128,
"lora_network_weights": "",
"color_aug": false,
"flip_aug": false,
"clip_skip": 2,
"gradient_accumulation_steps": 1.0,
"mem_eff_attn": false,
"output_name": "Addams",
"model_list": "runwayml/stable-diffusion-v1-5",
"max_token_length": "75",
"max_train_epochs": "",
"max_data_loader_n_workers": "1",
"network_alpha": 128,
"training_comment": "",
"keep_tokens": "0",
"lr_scheduler_num_cycles": "",
"lr_scheduler_power": ""
}the main reason its slower is because its a shit-ton more steps. If you do the calculation, if you are using 4 epochs with 30 images with 100 repeats.
If you like the quality better with the more steps approach, theres nothing wrong with that. Its never one-size-fits-all. The source images matter greatly. Sometimes more steps are required. "Most" of the time I get best results 1500-3000 total steps. Could be whatever youre doing requires more steps if you like that result better.
EDIT: but with using the calculation with the epochs, its easier to find the sweet spot, because going high number of steps, you risk going too far and overtraining. If epoch'd you can go backwards to a previous spot before it became cooked.
Yeh Aitrepreneur's 1 epoch thing is pretty dumb IMO. Unless you've got some epochs to work with you've got no room to maneuver if it doesn't turn out perfect the first time. I think there's a way to recommence a training from where it left off but don't know anything about that.
You could always use both ways combined: use their config and just use the calculations I gave for determining the repeats/epochs/samples. Just let it run and go do other things if the speed is taking too long. At least you have a good likelihood of finding the lora that you are most happy with. Just make it come out with more total steps than 3000 if you think you require more. Divide by more than 7000, try 10000. And if need be because theres too many epochs (maxed out), then increase the repeats SLIGHTLY.
@frank22 I tried some experimenting today with this. I was using realisticVision checkpoint while training, as I mentioned previous, because that was the checkpoint I was always rendering in and it worked fine. But now when I want to try some rendering in some other checkpoints, I know I will get better results always using the plain SD1.5 checkpoint for training, since all the other checkpoints are based on it. I should have just done it from sd1.5 always, but liked the results with the train run on the same checkpoint I would render in. So.. anyway, I remember you required more steps to get the results you liked than I needed for mine. And now I find after testing that the optimal number of steps is based upon the checkpoint you are training on.
Running with the realisticVision needed much less steps. The same exact images and captions, etc have been still sitting on my hard drive untouched. Everything is identical (except the number of repeats I needed to increase very slightly to add the needed extra steps required). Training on the SD1.5 base needed about 3x more steps than running on the RV checkpoint to get the same quality. (actually, I get BETTER results rendering using a different checkpoint altogether (cyberrealistic5.0) after training with sd1.5)
So I think thats the reason why you might have needed more steps, it all comes down to the checkpoint, I believe. Just figured I share this info, to use in your own trainings.
Thanks, that's interesting and useful information. I think most people train on the 1.5 base checkpoint because it makes loras much more compatible with all the other 1.5 models, which is good for sharing with other people. The two 1.5 base models are quite large (4.3GB and 7.7GB) so could that explain the difference in steps?
I've read that when training lora it's better to use v1-5-pruned.ckpt (ema+non-ema weights, 7.7GB) rather than the pruned-emaonly model. Others say the larger model is only useful for fine-tuning the ckpt, so I don't really know which to believe.
You may have already seen this. The conclusion is that you should train with the ckpt you intend to use for generating images, which makes complete sense to me.
Maybe we should clarify the terms better and quality. When I use them I'm referring to how well the lora realistically resembles the images in my dataset. Others use them to describe the overall aesthetics of the generated images, skin hue, over-fitting issues etc, for non-likeness subjects like generic elves/furries where likeness isn't the aim.
It would be interesting, if you had the time/inclination, to train a Dreambooth ckpt on your dataset to see how it compares to the loras you've made. I did it once (by accident) and the result was very accurate and stable. But training into a base model would obviously tie you to using only that model for rendering. You can apparently extract a lora from it but that would seem to defeat the purpose. IMO dataset quality probably overrides all the other training variables we're tinkering with. Sorry for rambling.
@frank22 I havent tested between the different 1.5 base models, nor the dreambooth. But I will do so soon. Im not sure the ideal number of steps for each checkpoint since i've now seen that number is not the same between them. So I guess ill just use very high steps and work backwards through lower epochs until I find the sweet spot
- You must be logged in to reply to this topic.
