Qwen-Image 2.1 gets eight controls in one file with ControlNet Union

Alibaba's PAI team has released ControlNet Union for Qwen-Image 2.1, Qwen's image generation model, on Hugging Face. ControlNet is an add-on that lets you steer the structure of a text-to-image result with a reference image. Give it a pose skeleton, for example, and the model draws the person you describe in exactly that pose.

What it brings

Until now, each type of control usually needed its own ControlNet file. According to the model card, ControlNet Union packs eight kinds of control into a single file:

• Edges and lines: Canny, HED, Lineart and MLSD edge and line maps
• Depth: a depth map estimated from a single photo
• Pose: a human skeleton (DWPose)
• Scribble: rough hand-drawn sketches
• Grayscale: the structure of a black-and-white image

The same file can also redraw a masked part of a photo from a prompt (inpainting), and both can be combined, so a redrawn person follows a given pose. According to the model card, it runs with a single pass per step to keep generation fast.

How to use it

ControlNet Union does not run on its own: the roughly 7 GB file is loaded on top of the base Qwen-Image 2.1 model. Instructions and sample code are in the PAI team's VideoX-Fun repository, and there is a browser demo on Hugging Face.

Having everything in one file makes design and product image work that switches between control types easier. However, ControlNet Union is released under Qwen's research licence, which allows research and evaluation use only; commercial use requires a separate licence from Qwen.

Sources: Hugging Face: Qwen-Image-2.1-Fun-Controlnet-Union model card, GitHub: VideoX-Fun, Qwen Research licence.