FLAIRGEORGIA TECH

FLAIR · project

RVM: Scaling Reinforcement Learning for Diffusion Models via Velocity Matching

Reward feedback can improve a generative model beyond its original training objective, but applying reinforcement learning to a long denoising process can be expensive. RVM studies reward fine-tuning through the velocity field that drives generation.

Reward feedback through velocity matching

The method generates samples, evaluates their rewards, and constructs noisy training states from those samples. Reward-weighted regression then adjusts the velocity field toward preferred outputs, while an anchor term controls how far the model moves during an update. Training does not require storing or scoring a complete denoising trajectory.

Connecting methods and applications

This formulation explains relationships between several velocity-based fine-tuning objectives and provides a practical route to reward optimization for diffusion and flow models. The project presents video-generation comparisons and evaluations of quality, temporal consistency, and training cost. RVM is a research preprint; the paper and accompanying implementation describe the objective and experimental settings.

Publication

Scaling Reinforcement Learning for Diffusion Models via Velocity Matching

Jaemoo Choi, Wei Guo, Yuchen Zhu, Arash Vahdat, Molei Tao, Julius Berner, and Yongxin Chen

Preprint · 2026