AI-Toolkit Gains Content Generation Features

Developer Ostris has updated AI-Toolkit, adding content generation functionality to the LoRA training interface. Now users can work with popular open-source models directly within a single application.

Supported Models

• Image Generation: Krea 2, Flux 1/2, Qwen Image 2512, Qwen Image 2.1, SDXL, Z-Image Turbo.

• Video Creation: MiniMax H3, LTXV 2.3/2.5, Wan 2.2.

• Music Generation: Ace-Step 1.5XL, MiniMax Music 3 (in beta), Ye2.

Various quantization options are available for the transformer and text encoder: bf16, int-8 convrot, fp8, qfp8, nvfp4. Additionally, the ability to merge LoRA with the base model has been implemented.

The advantage of this solution lies in combining LoRA training and content generation processes without the need to move checkpoints to ComfyUI. However, the flexibility and ecosystem of the node-based ComfyUI interface remain unmatched.

At this stage, the alpha version has minimal settings and speed issues. For example, generating a 1024x1024 image via fp8 Krea 2 takes 13 seconds on an RTX 4090, whereas the same process in ComfyUI on bf16 completes in 5 seconds. Creating high-resolution images (2048x2048) may take over a minute or cause hangs.

AI-Toolkit installation and updates are performed via the AI-Toolkit-Easy-Install script.