text_both_base_0728Checkpoint epoch017-global_step71000, on the first 20
held-out val.jsonl clips. Every clip has generated audio as well as
video — unmute to judge the full result.
a video with {object};
the model must put the object and its sound back.a video without {object}; the model must take it out.--prepend-ref, stripped again before decoding);
no first frame forces it off (--no-prepend-ref).
See the caveat below on which mode matches training.add
direction was found to overshoot at --cfg-scale 5.0 — the latents leave
the VAE's range and decode to a flat pink wash, monotonically in cfg and across seeds,
while remove survives it. This matrix is generated at cfg 1.0.with first frame is out-of-distribution for this checkpoint.
Training used ref_prepend_warmup_steps=5000 with
p_ref = max(0, 1 - step/5000), so from step 5,000 onward the anchor was
stripped from both target and source latents. At step 56,000 this checkpoint trained
anchor-free for 66,000 of its 71,000 steps — no first frame is the mode
that matches training; the with-first-frame cell (generated for add only) is shown for comparison, and any blowout
confined to the first ~9 pixel frames is the expected OOD symptom.Mean of (L1 → target − L1 → input) over the 20 clips of each cell. Negative is good: the generation sits closer to the ground-truth target than to the clip it was given.