Build ICASSP manuscript and experiment framework

manuscript/main/REVIEWER_CHECKLIST.md

@@ -0,0 +1,141 @@
+# ICASSP 2027 completion checklist
+
+This is an internal execution checklist. It is not part of the submitted PDF.
+
+## Reviewer verdict on the current draft
+
+The research question is potentially publishable: **at matched wearer-response
+recall, can correctly aligned person-level vision reduce false wearer entries
+during third-party conversation without degrading legitimate wearer responses?**
+
+The contribution is not “the first audio-visual multiparty turn model.” VAP,
+EgoSpeak, SICNet, and MuVAP already occupy adjacent territory. The defensible
+gap is continuous, causal floor-entry projection for an **invisible camera
+wearer**, conditioned on addressee evidence from a variable participant set.
+
+The current aggregate result (G+A+pooled-V minus G+A: +0.30 points at 0.2 s,
++0.29 mean mAP, one seed) motivates the question but does not establish social
+grounding. Acceptance depends on the event protocol and identity controls.
+
+## P0 — Freeze the task before training a new model
+
+- [ ] Define the exact 5 Hz causal cutoff and physical target interval for each
+  of the ten 0.2 s horizons.
+- [ ] Freeze addressed-response and third-party event definitions.
+- [ ] Add competing-risk rules: an addressed response ends at the first wearer
+  onset, unrelated-other onset, timeout, or invalid annotation boundary.
+- [ ] Specify overlap, minimum silence, maximum response delay, duplicate
+  detections, clip boundaries, group/none/multiple/unknown targets, and
+  offscreen speakers.
+- [ ] Produce train/test event-count tables, duration, class prevalence, and
+  visibility coverage before examining model scores.
+- [ ] Mark all already viewed Sessions 1–3 results as exploratory. From this
+  point forward, lock the event protocol and select thresholds, checkpoints,
+  and all hyperparameters using a training-side fold or cross-fitting over
+  Sessions 4–12. Run the fixed final model on Sessions 1–3 only after that
+  lock; do not describe the historically viewed split as untouched.
+
+**Gate:** two independent code paths or a manual sample audit must agree on at
+least 100 randomly selected events. Do not start the main model sweep before
+this gate passes.
+
+## P1 — Establish matched baselines
+
+Run three seeds with identical optimizer, history, parameter budget, and event
+evaluator:
+
+1. G: frozen streaming omni state.
+2. G+A: global state plus recent audio.
+3. G+A+V-pool: the current pooled visual baseline.
+4. Temporal control: G+A plus a parameter-matched non-person temporal branch.
+
+Report original ten-horizon AP/mAP and the event metrics. Confirm that changing
+batching, cached features, or track padding does not change baseline outputs.
+
+**Gate:** baseline standard deviations are known, and the event evaluator
+reproduces identical scores from saved predictions.
+
+## P2 — Implement the minimum person-set model
+
+- [ ] Shared temporal encoder for every face track.
+- [ ] Each person representation cross-attends to the same recent mixed-audio
+  token sequence; no separated/close-mic audio at inference.
+- [ ] Masked permutation-equivariant set encoder.
+- [ ] Learned OFFSCREEN/NULL element.
+- [ ] Fixed floor/address/third-party role queries.
+- [ ] Zero-initialized gated residual into the frozen global state.
+- [ ] Unit tests for variable `N`, empty visible set, missing frames, track-slot
+  permutation, padding invariance, and strict causal cutoff.
+
+**Gate:** with residual gate zero, logits are numerically identical to G+A; a
+random permutation of track slots leaves logits unchanged.
+
+## P3 — Train only the ablations that answer the claim
+
+Use the same three seeds:
+
+1. WearerSet with activity loss only.
+2. WearerSet + per-person active-speaker auxiliary loss.
+3. WearerSet + active-speaker + pure-target addressee auxiliary loss.
+
+Mask group/unknown/offscreen target annotations from addressee loss. They may
+remain in the activity loss. Do not use LLM pseudo-labels as primary gold.
+
+Stop after these stages unless a clear event-level gain appears. Broad Thinker
+LoRA, 7B, and ROMA experiments are diagnostics, not the core method, because
+the existing runs show negative transfer or no consistent 0.2 s benefit.
+
+## P4 — Run causal visual counterfactuals
+
+For the frozen best checkpoint, evaluate:
+
+- correct person tracks;
+- wrong-person tracks from the same scene;
+- time-shifted tracks with unchanged audio;
+- removed/zero visual features;
+- visibility-conditioned subsets.
+
+All controls must reuse the same model weights. Retraining each control would
+mix representation and optimization effects.
+
+**Acceptance gate:** at matched addressed-response recall, correct vision must
+reduce third-party false entries more than wrong/time-shifted/removed vision,
+without worse response latency or wearer-yield recall. Aggregate mAP alone is
+not sufficient.
+
+## P5 — Statistics and analysis
+
+- [ ] Three training seeds, mean ± sample standard deviation.
+- [ ] Paired cluster bootstrap by session/conversation block for 95% confidence
+  intervals; do not bootstrap adjacent 5 Hz frames independently.
+- [ ] Operating points at addressed recall 0.7, 0.8, and 0.9; headline at 0.8.
+- [ ] Error strata: target visibility, number of visible people, overlap,
+  backchannel/interruption proxy, and offscreen target.
+- [ ] P50/P95 decision latency, throughput, peak GPU memory, and trainable
+  parameters.
+- [ ] Ego4D only as secondary aggregate activity generalization; it cannot
+  validate addressee grounding without comparable target labels.
+
+## Paper assembly order
+
+1. Finalize the event protocol and insert counts in Experimental Setup.
+2. Produce the main matched-recall table before polishing prose.
+3. Produce the correct/wrong/time-shift/remove counterfactual table.
+4. Draw one compact method figure with causal cutoffs and the variable set.
+5. Collapse the preliminary aggregate table into the final main table if page
+   pressure is high.
+6. Add one short failure analysis, not a catalog of all earlier experiments.
+7. Replace the abstract and conclusion TODOs with exact numbers and confidence
+   intervals.
+8. Remove every visible `\todo{}` marker and verify the official ICASSP 2027
+   template/page limit once released.
+
+## Stop/go decision
+
+- **Go:** correct person-level vision provides a statistically credible
+  event-level advantage over G+A, pooled-V, temporal capacity control, and
+  wrong/time-shifted vision.
+- **Reframe:** all visual variants improve similarly. Claim better local
+  temporal context, not addressee grounding.
+- **Stop this method:** aggregate mAP rises but third-party false entries do
+  not fall at matched recall, or legitimate response latency/recall degrades.