Long video to vertical clips: 0 of 96 cuts inside a word by Whisper’s own times, but two ways of grading the picks disagree

The honest verdict

The parts done in code behaved as designed. None of the 96 model-picked cut points fell inside a word by Whisper’s own word times, which the snapper also starts from (0 of 102 for the keyword-rule baseline too, so that zero does not tell the model from the rules). The second check transcribed the 48 rendered clips again and found the first and last word intact in 47 of 48; the two misses are recognition differences (a name heard differently, a hallucinated “bye” after a silent ending), not clipped words. The cuts were not all ideal: 3 were within 30 ms of a word, 7 were on audio louder than -12 dB relative to speech, and lint warnings hit 23% of clips. The free model is good at picking moments, with 20 of 22 clips on the synthetic talks landing on a planted highlight against 12 of 22 for the rules, but read that with care: it is one run on 22 clips, the planted highlights were written to stand alone (the other turns refer back to earlier ones), so they are easy targets, the scripts were drafted by the same model family that picks the clips, and real NASA speech has no highlight gold. Recall is only 20 of 43, because it was asked for 2-3 clips per 7-minute talk and every talk has 5 planted highlights. Whether the clips stand on their own depends on who judges. The blind LLM judge said 47 of 48 do (and 31 of 51 rule-picked clips, 61%), but it is lenient and noisy: it also said 60% of random windows do and gave different verdicts on the same random windows in different batches (53% vs 35% on 3 videos), and every verdict came from a fallback model, dots-3-note-preview, not the gemma model that was requested. A 20-clip sample (10 synthetic, 10 NASA, drawn on the first day from the 9 videos done by then, so the 4 talks processed on day two are not in it) scored with a rubric by Claude, the AI that built Recut and not an independent human, gave 16 of 20 fully standalone and 15 of 20 passing the whole rubric; the 4 that failed open with “that signal”, “the gentler approach”, “The Welcome series” or a garbled sentence. The NASA clips are standalone (9 of 10) but weaker as social hooks (1.1 on a 0-2 scale). Chapters are weaker than they look: 15 of 20 predicted boundaries were within ±15 s (mean error 1.9 s), but 4 of 20 were more than 30 s from any real topic change, and a plain lexical baseline did as well at ±30 s. Face tracking kept the face inside the 9:16 window in 96.8% of sampled frames against 88.9% for a centre crop (25 NASA clips); a centre crop is already fairly good for people at a press-conference table, and 23 slide or B-roll clips fell back to the full frame over a blurred fill. Speech recognition made 1.6% word errors on synthetic voices and missed or got wrong 7.2% of NASA’s caption words on real speech (the raw 20.0% is inflated because NASA’s live captions skip whole passages), and speakers were only scored on the easy case of two clearly different synthetic voices (98.6% of words); expect worse on similar voices and overlapping speech. The run also exposed errors upstream of the model: the test voice read “$10” as “a dollar ten” and Whisper heard “$15 ... $400” as “$1.15 ... $1.400”, so the model wrote “$1.10” and “$1,400” in 12 posts, and the grounding check flagged all 12. It also flagged 4 long-form passages, 3 of them NASA show-note quotes stitched together from separate sentences with ellipses, so not verbatim. The free reasoning model sometimes spends its whole budget thinking and returns nothing, which cost 2 videos their posts and 1 its clips until retries were added, and the free tier’s daily request limit stopped the run after 9 of 13 videos. The test set is 13 videos (2 h 40 min): 9 synthetic ones and 4 public-domain NASA briefings, all processed with free models for $0.