# ICASSP 2027 completion checklist

This is an internal execution checklist. It is not part of the submitted PDF.

## Reviewer verdict on the current draft

The research question is potentially publishable: **at matched wearer-response
recall, can correctly aligned person-level vision reduce false wearer entries
during third-party conversation without degrading legitimate wearer responses?**

The contribution is not “the first audio-visual multiparty turn model.” VAP,
EgoSpeak, SICNet, and MuVAP already occupy adjacent territory. The defensible
gap is continuous, causal floor-entry projection for an **invisible camera
wearer**, conditioned on addressee evidence from a variable participant set.

The current aggregate result (G+A+pooled-V minus G+A: +0.30 points at 0.2 s,
+0.29 mean mAP, one seed) motivates the question but does not establish social
grounding. Acceptance depends on the event protocol and identity controls.

## P0 — Freeze the task before training a new model

- [ ] Define the exact 5 Hz causal cutoff and physical target interval for each
  of the ten 0.2 s horizons.
- [ ] Freeze addressed-response and third-party event definitions.
- [ ] Add competing-risk rules: an addressed response ends at the first wearer
  onset, unrelated-other onset, timeout, or invalid annotation boundary.
- [ ] Specify overlap, minimum silence, maximum response delay, duplicate
  detections, clip boundaries, group/none/multiple/unknown targets, and
  offscreen speakers.
- [ ] Produce train/test event-count tables, duration, class prevalence, and
  visibility coverage before examining model scores.
- [ ] Mark all already viewed Sessions 1–3 results as exploratory. From this
  point forward, lock the event protocol and select thresholds, checkpoints,
  and all hyperparameters using a training-side fold or cross-fitting over
  Sessions 4–12. Run the fixed final model on Sessions 1–3 only after that
  lock; do not describe the historically viewed split as untouched.

**Gate:** two independent code paths or a manual sample audit must agree on at
least 100 randomly selected events. Do not start the main model sweep before
this gate passes.

## P1 — Establish matched baselines

Run three seeds with identical optimizer, history, parameter budget, and event
evaluator:

1. G: frozen streaming omni state.
2. G+A: global state plus recent audio.
3. G+A+V-pool: the current pooled visual baseline.
4. Temporal control: G+A plus a parameter-matched non-person temporal branch.

Report original ten-horizon AP/mAP and the event metrics. Confirm that changing
batching, cached features, or track padding does not change baseline outputs.

**Gate:** baseline standard deviations are known, and the event evaluator
reproduces identical scores from saved predictions.

## P2 — Implement the minimum person-set model

- [ ] Shared temporal encoder for every face track.
- [ ] Each person representation cross-attends to the same recent mixed-audio
  token sequence; no separated/close-mic audio at inference.
- [ ] Masked permutation-equivariant set encoder.
- [ ] Learned OFFSCREEN/NULL element.
- [ ] Fixed floor/address/third-party role queries.
- [ ] Zero-initialized gated residual into the frozen global state.
- [ ] Unit tests for variable `N`, empty visible set, missing frames, track-slot
  permutation, padding invariance, and strict causal cutoff.

**Gate:** with residual gate zero, logits are numerically identical to G+A; a
random permutation of track slots leaves logits unchanged.

## P3 — Train only the ablations that answer the claim

Use the same three seeds:

1. WearerSet with activity loss only.
2. WearerSet + per-person active-speaker auxiliary loss.
3. WearerSet + active-speaker + pure-target addressee auxiliary loss.

Mask group/unknown/offscreen target annotations from addressee loss. They may
remain in the activity loss. Do not use LLM pseudo-labels as primary gold.

Stop after these stages unless a clear event-level gain appears. Broad Thinker
LoRA, 7B, and ROMA experiments are diagnostics, not the core method, because
the existing runs show negative transfer or no consistent 0.2 s benefit.

## P4 — Run causal visual counterfactuals

For the frozen best checkpoint, evaluate:

- correct person tracks;
- wrong-person tracks from the same scene;
- time-shifted tracks with unchanged audio;
- removed/zero visual features;
- visibility-conditioned subsets.

All controls must reuse the same model weights. Retraining each control would
mix representation and optimization effects.

**Acceptance gate:** at matched addressed-response recall, correct vision must
reduce third-party false entries more than wrong/time-shifted/removed vision,
without worse response latency or wearer-yield recall. Aggregate mAP alone is
not sufficient.

## P5 — Statistics and analysis

- [ ] Three training seeds, mean ± sample standard deviation.
- [ ] Paired cluster bootstrap by session/conversation block for 95% confidence
  intervals; do not bootstrap adjacent 5 Hz frames independently.
- [ ] Operating points at addressed recall 0.7, 0.8, and 0.9; headline at 0.8.
- [ ] Error strata: target visibility, number of visible people, overlap,
  backchannel/interruption proxy, and offscreen target.
- [ ] P50/P95 decision latency, throughput, peak GPU memory, and trainable
  parameters.
- [ ] Ego4D only as secondary aggregate activity generalization; it cannot
  validate addressee grounding without comparable target labels.

## Paper assembly order

1. Finalize the event protocol and insert counts in Experimental Setup.
2. Produce the main matched-recall table before polishing prose.
3. Produce the correct/wrong/time-shift/remove counterfactual table.
4. Draw one compact method figure with causal cutoffs and the variable set.
5. Collapse the preliminary aggregate table into the final main table if page
   pressure is high.
6. Add one short failure analysis, not a catalog of all earlier experiments.
7. Replace the abstract and conclusion TODOs with exact numbers and confidence
   intervals.
8. Remove every visible `\todo{}` marker and verify the official ICASSP 2027
   template/page limit once released.

## Stop/go decision

- **Go:** correct person-level vision provides a statistically credible
  event-level advantage over G+A, pooled-V, temporal capacity control, and
  wrong/time-shifted vision.
- **Reframe:** all visual variants improve similarly. Claim better local
  temporal context, not addressee grounding.
- **Stop this method:** aggregate mAP rises but third-party false entries do
  not fall at matched recall, or legitimate response latency/recall degrades.