Google's R4T compiles query fan-out into a single diffusion pass

Google Research has introduced Retrieve-for-Train (R4T), a framework for query fan-out in search and recommendation. It runs reinforcement learning (RL) once, offline, to learn good fan-out behaviour, then distills that behaviour into a small diffusion model that produces all retrieval directions in a single pass.

Query fan-out splits one broad prompt into several sub-queries. The team points to two failures when a generic LLM does this at inference time. The first is paraphrastic collapse: for "Bohemian festival style", zero-shot Qwen3-4B produced "bohemian festival fashion" and "festival bohemian clothes", near-synonyms that return a homogeneous slate. The second is latency, since autoregressive generation plus repeated retrieval calls is slow and Best-of-N sampling multiplies inference cost.

Reward and training

For open-ended abstract retrieval (OAR), the reward combines three weighted terms. The ablation shows why all three matter: with groundedness alone, Gemma3-4B converged to strings such as "line ending line ending line ending"; adding alignment accelerated the collapse as the policy repeated paraphrases; adding diversity closed both shortcuts. Training uses GRPO with soft PPO regularization, adding forward and reverse KL penalties, with group size 8, learning rate 1×10⁻⁷ and global batch size 512.

Results

Experiments used the Polyvore fashion dataset with a CLIP-based matryoshka encoder at 128 dimensions, and a proprietary expert-playlist music dataset with MuLan embeddings. Every method produced k = 10 sub-queries; Best-of-N used N = 5. Scored by an LLM judge on 5-point Likert scales, Gemma3-4B R4T-FOLM averaged 49.1 on Polyvore against 40.9 for Best-of-N and 38.5 zero-shot. Diversity rose from 56.0 to 76.8, with the diffusion variant holding 74.3. On music, R4T-FOLM averaged 58.1 versus 49.2.

On weakly supervised compositional retrieval, R4T-FOLM (Qwen) reached 20.9 Recall@5K and 64.6 Hit@5K against 15.7 and 52.1 for Gemini-2.5-Flash, but its Vendi Score fell to 27.5, which the authors link to reduced output entropy under strong RL optimization.

At batch size 8, autoregressive fan-out took about 1.46 seconds versus 0.07 for diffusion; at batch size 1024, nearly 50 seconds versus 4.21. The authors report a consistent 12x to 20x speedup. Details are in the MarkTechPost report.