5.9 KiB
Targeted ICASSP 2027 manuscript
This directory is an independent acceptance-target manuscript. It does not
modify or replace ../main/.
Positioning
The paper now starts from the egocentric conversational agent's when-to-speak problem, positions it against 27 cited works in turn-taking, first-person audio-visual conversation, active-speaker modeling, and streaming Omni interaction, then studies two cutoff-aligned interfaces:
- short-window encoder evidence (SWE): direct recent audio and optional person-crop visual encoder features alongside the frozen Omni readout;
- partial-tail refresh (PTR): query only the observed prefix of the current two-second block at each 200-ms cutoff, reusing completed payloads.
SWE names the short temporal support of encoder evidence, not a shallow network layer. The two-second chunk refers to this Qwen extraction setup, not a universal Omni-model setting.
The evidence remains a causal global--local audit, not a claim that all Omni models lack local perception or that PTR's independent gain is proven:
- global Omni context and recent audio are complementary across all horizons;
- most measured benefit is dense forecasting and wearer-offset anticipation;
- pooled and current-frame person vision do not establish speaker grounding;
- a three-seed causal oracle shows that timely visible-participant activity is informative, while identity continuity is not currently necessary.
The draft retains the remote SVA/TSVA terminology and four-method EasyCom/Ego4D main table. The no-box TSVA and global-only chunk runs are seed-0 exploratory screens; neither supplies matched visual-motion or PTR evidence. Their saved records were checked in the L20 mounted repository.
The Results section also reports the archived 2026-08-24 sparse Thinker-LoRA diagnostic as a separate historical screen. Its strict causal-valid mean mAP declined under the recorded LoRA recipes, but the legacy feature/head contract is not matched to the current G+A matrix, and the remote raw metrics and checkpoints are unavailable here. It is not a Table 1 model ranking or proof that LoRA cannot help this task.
No semantic ENTRY/YIELD, addressee, online-detector, or real-time-throughput claim is made. The main EasyCom deployable matrix is single-seed and exploratory. Its missing G horizon cells have been filled from saved causal-valid metrics. Ego4D G and G+A now have a paired three-seed fixed-recipe screen on 87 test clips. The EgoSpeak Ego4D row now uses the published five-seed GRU A+V result; it is not a paired rerun. EasyCom SVA has a four-epoch seed-0 screen. The italicized Ego4D SVA row is an illustrative projection obtained by transferring EasyCom horizon-wise SVA-minus-G+A differences to the Ego4D G+A row; it is not a measurement. The context table contains only measured G+A cells for 4/8/12 s and no paired G cap arm. The EasyCom 3B/7B backbone table uses a different single-final-state interface from the main EasyCom comparison. Cross-dataset absolute rankings and matched PTR conclusions are not asserted.
The six-group L/T/C/B/H-Q/S table remains in main.tex as an
illustrative, unmeasured planning aid, excluded from the page-limited
PDF. Its directional statements are hypotheses and falsifiers, not EasyCom
scores. Replace or remove every row after the corresponding experiment,
including results that contradict the working story.
Build
cd manuscript/targeted
make
The editable sources are main.tex, tables_compact.tex, references.bib,
and figures/system_overview_onecolumn.tex. The earlier
figures/task_overview.tex is retained as unused source.
Generated LaTeX files are written to build/.
The preview targets four technical pages plus a reference-only fifth page. The single-column method diagram (Fig. 1) has SWE on top and PTR below; the earlier standalone task figure is not included. The current PDF contains five numbered result tables. Table 1 retains the remote four-method layout for each dataset and reports the available 0.2--2.0-s mAP curves. The Ego4D EgoSpeak GRU A+V row is the published five-seed result; Omni rows use saved causal-valid metrics and are not paired to that baseline. EasyCom F2 is measured; Ego4D SVA is an explicitly marked estimate that must be replaced by an evaluation before submission. Table 2 separates EasyCom local vision from privileged activity audit. Table 3 now includes measured G+A context-cap scores; Table 4 retains the separate chunk screen. Table 5 has the exploratory 3B/7B paired audio comparison. Active table layouts are in the tables_compact.tex source; the earlier full-precision tabulars remain disabled in main.tex.
Figure 2 uses figures/omni_case_selected_easycom_v2.pdf, copied from the
mounted exploratory EasyCom case report. It shows two outcome-selected
0.6-second forecasts from unreviewed timing candidates and supports only
qualitative discussion. The single-column system figure is generated
from figures/system_overview_onecolumn.tex; the earlier full-width source
is retained for comparison.
The build uses the repository's ICASSP 2027 LaTeX assets in
../ICASSP2027_Paper_Templates/, selected by latexmkrc. The supplied
spconf.sty and IEEEbib.bst match the 2026 copies.
Evidence sources
../../results/easycom_encoder_only_1s_4ep_2026-08-28/../../results/easycom_egospeak_three_level_eval_2026-09-16/../../results/easycom_omni_floor_screen_2026-09-16/../../results/easycom_oracle_pooled_wearer_audit_2026-09-10/../../results/easycom_sparse_supervision_vs_lora_2026-08-24/../../results/easycom_omni_lora_sparse_epoch2_2026-08-24/../../results/easycom_context_window_4_8_12_16_2026-09-24/../../results/easycom_omni_3b_7b_backbone_2026-09-24/../../results/ego4d_local_paired_audio_visual_2026-09-24/../../results/easycom_chunk200ms_vs_native2s_2026-09-24/
The manuscript does not use superseded opportunity-conditioned ENTRY/INT
numbers. See REVIEWER_REPORT.md for the internal review loop and the remaining
scientific submission gates.