FLAIR · project
RVM: Scaling Reinforcement Learning for Diffusion Models via Velocity Matching
Reward feedback can improve a generative model beyond its original training objective, but applying reinforcement learning to a long denoising process can be expensive. RVM studies reward fine-tuning through the velocity field that drives generation.
Reward feedback through velocity matching
The method generates samples, evaluates their rewards, and constructs noisy training states from those samples. Reward-weighted regression then adjusts the velocity field toward preferred outputs, while an anchor term controls how far the model moves during an update. Training does not require storing or scoring a complete denoising trajectory.
Connecting methods and applications
This formulation explains relationships between several velocity-based fine-tuning objectives and provides a practical route to reward optimization for diffusion and flow models. The project presents video-generation comparisons and evaluations of quality, temporal consistency, and training cost. RVM is a research preprint; the paper and accompanying implementation describe the objective and experimental settings.
Publication
Scaling Reinforcement Learning for Diffusion Models via Velocity Matching
Jaemoo Choi, Wei Guo, Yuchen Zhu, Arash Vahdat, Molei Tao, Julius Berner, and Yongxin Chen
Preprint · 2026