Files
icassp2027-experiment/manuscript/targeted/REVIEWER_REPORT.md
T

101 lines
4.9 KiB
Markdown
Raw Normal View History

2026-09-23 07:54:36 +08:00
# Internal ICASSP-style review loop
## Review 1: reject in original method framing
**Main concerns**
1. The novelty claim was too broad. EgoSpeak already defines egocentric
when-to-speak prediction, while MuVAP, AV-CONV, and 3M-Transformer cover
adjacent multiparty audiovisual structure.
2. The learned visual method did not support the intended speaker-grounding
claim. Pooled vision gave only `+0.291` mean-mAP points over `G+A`, and the
current-frame person-set screen did not exceed the historical pooled row.
3. Dense mAP, raw endpoint AP, and semantic floor events were conflated.
4. Planned tail, context, transfer, and behavior experiments occupied too much
of a four-page argument despite being unmeasured.
**Revision**
The paper was reframed from a successful visual adapter to a causal
global--local evidence audit. The claim is now that recent audio and global
context are complementary, while generic visual access remains insufficient.
All semantic event numbers and planned cross-backbone claims were removed.
## Review 2: weak reject after reframing
**Strengths**
- Clear task asymmetry: visible interlocutors condition prediction of an
invisible wearer.
- Strict causal cutoff and fixed 5-Hz / ten-horizon contract.
- A coherent positive result for `G+A`, especially wearer-offset anticipation.
- An unusually informative negative result: visible-activity labels help, but
current generic visual features do not recover that signal.
- Oracle controls distinguish visibility, timeliness, aggregation, and
identity continuity.
**Remaining concerns**
1. The deployable main comparison is single seed; only the oracle has three
seeds.
2. Sessions 1--3 have been inspected repeatedly, so the current matrix is not
a clean confirmatory test.
3. `G+A` changes history, feature access, and trainable capacity relative to
`A` or `G`; a matched wider/temporal control is missing.
4. Raw onset/offset AP is not a one-event-one-vote behavioral metric.
5. Checkpoint hashes, common valid-pair counts, and original predictions are
not all archived on this host.
**Revision**
The abstract and results now identify single-seed versus three-seed evidence,
avoid significance language, separate endpoint diagnostics from events, and
state the capacity/history confound. The main paper no longer claims reduced
false entry, semantic yield, speaker identity, or real-time throughput.
## Review 3: current verdict
**Score: 5/10, borderline, structurally ready but not yet confirmatory.**
The manuscript is substantially clearer and more defensible than a paper that
claims a successful visual grounding method. Its strongest publishable insight
is the controlled mismatch between oracle information value and learned visual
access. However, writing quality cannot replace the missing confirmatory
evidence. Calling the current artifact "accepted-paper ready" would overstate
the science.
## Minimum gates for a credible accept recommendation
1. Re-run `A`, `G`, `G+A`, and the selected visual condition for seeds 0/1/2
under one frozen recipe and common valid pairs; archive predictions,
checkpoint/config hashes, per-session values, and trainable parameter counts.
2. Add a parameter- and history-matched control. The central `G+A` gain must
remain larger than seed variance and the matched-capacity alternative.
3. Run the corrected competing-risk evaluator with training-side calibration,
manually audit real timelines, and report matched-recall false-entry and
false-stop rates only if denominators and invalid counts agree.
4. If visual grounding remains a contribution, replace generic pooling with a
causal active-speaker/floor estimator and require correct-time evidence to
beat delayed, removed, and capacity-matched controls. Otherwise retain the
honest audio-local paper and remove visual-method language.
5. Recheck the four-page technical limit after inserting results. The current
build uses the 2027 kit and puts only references on page five; retain the
IEEE AI-assisted drafting disclosure.
2026-09-23 07:54:36 +08:00
Passing gates 1--3 would support a solid ICASSP empirical paper. Passing gate
4 with a meaningful matched-recall behavior gain would support the stronger
method paper originally envisioned.
## Review 4: expanded five-page working preview
The layout now uses four pages for technical content and a fifth page for
references only. The Introduction includes a causal task timeline; the
method figure separates deployable inputs from privileged audit labels. The
results presentation has measured main/oracle tables and a clearly labeled
six-group **illustrative scenario** with explicit falsifiers. This improves
readability and makes the intended experimental logic reviewable, but it does
not raise the scientific verdict above borderline: L/T/C/B/H-Q/S values and
full-pipeline efficiency remain unmeasured, and the scenario cannot be cited
as evidence. Re-review after gates 1--4 and replacement of the scenario table
with actual per-seed/per-session results.