Post

Modern models excel at knowledge representation but struggle with physical reasoning. Humans solve novel physical problems easily via accurate mental simulation, a capacity current generative models lack.

To bridge this gap, Allen introduced 3DSPA: a 3D Semantic Point Autoencoder. It predicts 3D point trajectories to measure physical realism in generated video, detecting physical rule violations just like a human infant would.

The outcome? Used as a reward signal with reinforcement learning, 3DSPA measurably improves physical realism in generative video models. Gravity works properly and objects stop passing through solid barriers.