K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs
Our method supports advanced generative models such as Flux and Stable
Diffusion, etc.
K-LoRA generally achieves a seamless integration of objects and styles, effectively preserving fidelity and preventing distortion.
Results on Flux
Results on Flux
Results on Stable Diffusion
Results on Stable Diffusion
_page-0001.jpg)
Recent studies have explored combining different LoRAs to jointly generate learned style and content. However, existing methods either fail to effectively preserve both the original subject and style simultaneously or require additional training. In this paper, we argue that the intrinsic properties of LoRA can effectively guide diffusion models in merging learned subject and style. Building on this insight, we propose K-LoRA, a simple yet effective training-free LoRA fusion approach. In each attention layer, K-LoRA compares the Top-K elements in each LoRA to be fused, determining which LoRA to select for optimal fusion. This selection mechanism ensures that the most representative features of both subject and style are retained during the fusion process, effectively balancing their contributions. Experimental results demonstrate that the proposed method effectively integrates the subject and style information learned by the original LoRAs, outperforming state-of-the-art training-based approaches in both qualitative and quantitative results.
We utilizes the Top-K function to select the important LoRA weights in each forward layer based on the sum of matrix elements.
Compared to other methods, K-LoRA We provide comparisons with Direct Merge, Joint Training, B-LoRA and ZipLoRA.
We conducted experiments to evaluate whether our method can modify the object's actions, the surrounding environment, or introduce new contents through prompt adjustments.
