Okay, I cranked the tripping balls setting a bit too hard, but it's clear that adding a motion vector helps. This is SDXL-Turbo btw.
affine@refinement.systemsnah, naively blending the previous previous processed frame with a fresh one is too dumbed down, looks like crap when running it on a video.