100 GRPO steps lift a 350M model from 22.6% to 29.7% on IFStruct
A Hugging Face guide fine-tunes LFM2.5-350M with GRPO using roughly 500 samples and 100 steps, raising its IFStruct score from 22.6% to 29.7%.
A guide published on the Hugging Face blog walks through a cheap, fully public recipe for making a 350 million parameter model more reliable at structured output. The author fine-tunes LFM2.5-350M with Group Relative Policy Optimization (GRPO) via the TRL library and measures the result on the IFStruct benchmark: the score moves from 22.6% to 29.7%.
The run is deliberately small: about 500 samples and 100 training steps, sized for a free-tier Colab or Kaggle GPU with 16 GB of memory. The code is published on GitHub.
Baseline and reward design
To keep the comparison inside one serving stack, the model was served locally on a MacBook with llama.cpp using the BF16 GGUF build. The IFStruct release blog reports 21.1% for LFM2.5-350M; the local measurement came out at 22.6%, and that number was used as the baseline. The benchmark was run with 2000 samples.
Training data came from nvidia/Nemotron-RL-instruction_following-structured_outputs, which pairs each prompt with a target JSON Schema and an expected field count. Since the Nemotron distribution differs from the IFStruct evaluation, prompts were augmented to close two gaps between them.
A LoRA adapter was attached to the base model. Because LFM2.5 uses a hybrid attention and convolution architecture, the setup targets LFM-specific module names. Three reward functions, each on a [0, 1] scale, score every completion on structural correctness and are combined as a weighted sum with weights 1.0, 0.5 and 2.0. Training used 8 generations per prompt group. All three reward components climbed during the run, KL from the reference model lifted off zero after warmup, and the truncated-completion fraction stayed near zero.
Where the gain landed
The improvement shows up exactly where training aimed: the JSON pass rate rises from 18.0% to 31.9%, while YAML stays mostly flat. The guide notes the result is still below the 33.15% reported for Qwen3.5-2B, but argues that light task-specific tuning brings a small model close to a much larger one.
The full pipeline and notebook are in the original Hugging Face post.
