102 lines
5.1 KiB
Markdown
102 lines
5.1 KiB
Markdown
# Targeted ICASSP 2027 manuscript
|
|
|
|
This directory is an independent acceptance-target manuscript. It does not
|
|
modify or replace `../main/`.
|
|
|
|
## Positioning
|
|
|
|
The paper now starts from the egocentric conversational agent's when-to-speak
|
|
problem, positions it against 27 cited works in turn-taking, first-person
|
|
audio-visual conversation, active-speaker modeling, and streaming Omni
|
|
interaction, then studies two cutoff-aligned interfaces:
|
|
|
|
- **short-window encoder evidence (SWE)**: direct recent audio and optional
|
|
person-crop visual encoder features alongside the frozen Omni readout;
|
|
- **partial-tail refresh (PTR)**: query only the observed prefix of the current
|
|
two-second block at each 200-ms cutoff, reusing completed payloads.
|
|
|
|
SWE names the short temporal support of encoder evidence, not a shallow
|
|
network layer. The two-second chunk refers to this Qwen extraction setup,
|
|
not a universal Omni-model setting.
|
|
|
|
The evidence remains a causal global--local **audit**, not a claim that all
|
|
Omni models lack local perception or that PTR's independent gain is proven:
|
|
|
|
- global Omni context and recent audio are complementary across all horizons;
|
|
- most measured benefit is dense forecasting and wearer-offset anticipation;
|
|
- pooled and current-frame person vision do not establish speaker grounding;
|
|
- a three-seed causal oracle shows that timely visible-participant activity is
|
|
informative, while identity continuity is not currently necessary.
|
|
|
|
The draft also reflects two seed-0 exploratory screens reported in
|
|
`../main/main.tex`: no-box full-frame F3 (Table 1) and a global-only
|
|
chunk-construction comparison (Table 4). Neither supplies
|
|
matched visual-motion evidence, endpoint gains, or the pending T/C factorial.
|
|
The result directories cited by the main draft for F3 and its G+A recheck
|
|
are absent from this checkout, so their underlying logs have not been
|
|
independently audited here.
|
|
|
|
The Results section also reports the archived 2026-08-24 sparse Thinker-LoRA
|
|
diagnostic as a separate historical screen. Its strict causal-valid mean mAP
|
|
declined under the recorded LoRA recipes, but the legacy feature/head contract
|
|
is not matched to the current G+A matrix, and the remote raw metrics and
|
|
checkpoints are unavailable here. It is not a Table 1 model ranking or proof
|
|
that LoRA cannot help this task.
|
|
|
|
No semantic ENTRY/YIELD, addressee, online-detector, or real-time-throughput
|
|
claim is made. The deployable matrix is explicitly single-seed and exploratory.
|
|
EasyCom has archived screening scores. The Ego4D manifest and extraction /
|
|
training workflow exist, but this checkout has no archived Ego4D test metrics;
|
|
the manuscript specifies its transfer protocol and explicitly leaves any
|
|
cross-dataset performance claim pending.
|
|
The six-group L/T/C/B/H-Q/S table remains in `main.tex` as an
|
|
**illustrative, unmeasured planning aid**, excluded from the page-limited
|
|
PDF. Its directional statements are hypotheses and falsifiers, not EasyCom
|
|
scores. Replace or remove every row after the corresponding experiment,
|
|
including results that contradict the working story.
|
|
|
|
## Build
|
|
|
|
```bash
|
|
cd manuscript/targeted
|
|
make
|
|
```
|
|
|
|
The editable sources are `main.tex`, `tables_compact.tex`, `references.bib`,
|
|
`figures/task_overview.tex`, and `figures/system_overview_onecolumn.tex`.
|
|
Generated LaTeX files are written to `build/`.
|
|
|
|
The preview targets four technical pages plus a reference-only fifth page. Both
|
|
the task diagram (Fig. 1) and the system diagram (Fig. 2) occupy one column.
|
|
The current PDF reserves seven numbered result tables. Table 1 spans two
|
|
columns and reports 0.2--2.0-s mAP for EasyCom and the pending Ego4D transfer
|
|
rows. Only three EasyCom models have archived complete horizon curves; other
|
|
intermediate cells remain unavailable. The old Table 1B oracle audit,
|
|
Table 2A T/C matrix, and Table 2B chunk screen are now standalone Tables
|
|
2--4. B (including an Ego4D transfer slot), H, and S occupy Tables 5--7.
|
|
Pending cells are explicitly unmeasured. Active table layouts are in
|
|
`tables_compact.tex`; the earlier full-precision tabulars
|
|
remain disabled in `main.tex` for source-level auditing.
|
|
Figure 3 is an H/Q case slot, not a selected or measured example. Its
|
|
cutoff-marked timelines must be inserted only after the slice protocol and
|
|
case-selection rule are frozen. The single-column system figure is generated
|
|
from figures/system_overview_onecolumn.tex; the earlier full-width source
|
|
is retained for comparison.
|
|
|
|
The build uses the repository's ICASSP 2027 LaTeX assets in
|
|
`../ICASSP2027_Paper_Templates/`, selected by `latexmkrc`. The supplied
|
|
`spconf.sty` and `IEEEbib.bst` match the 2026 copies.
|
|
|
|
## Evidence sources
|
|
|
|
- `../../results/easycom_encoder_only_1s_4ep_2026-08-28/`
|
|
- `../../results/easycom_egospeak_three_level_eval_2026-09-16/`
|
|
- `../../results/easycom_omni_floor_screen_2026-09-16/`
|
|
- `../../results/easycom_oracle_pooled_wearer_audit_2026-09-10/`
|
|
- `../../results/easycom_sparse_supervision_vs_lora_2026-08-24/`
|
|
- `../../results/easycom_omni_lora_sparse_epoch2_2026-08-24/`
|
|
|
|
The manuscript does not use superseded opportunity-conditioned ENTRY/INT
|
|
numbers. See `REVIEWER_REPORT.md` for the internal review loop and the remaining
|
|
scientific submission gates.
|