Flux LoRA Specific Layer Training: Text Encoder, Resolution,Learning Rate and Alpha

When using a specific layer training method to train a LoRA for Flux,I encountered issues where the model failed to converge and generated images unrelated to my dataset. After a series of troubleshooting and experiments, I discovered some key factors behind the training failure. This chart summarizes my findings, and I hope it helps others avoid the same mistakes.

• t5xxl_fp8_e4m3fn_sclaed is not a good choice for training and should be avoided,and the speedup it provides is negligible.

• When training on specific layers, you'll need to increase the learning rate and network alpha values.

• If you also increase the training resolution, you'll have to raise them even more.