Update manuscript
This commit is contained in:
@@ -1,8 +1,4 @@
|
|||||||
LATEXMK ?= latexmk
|
LATEXMK ?= latexmk
|
||||||
TEMPLATE_DIR := $(abspath ../ICASSP2026_Paper_Templates)
|
|
||||||
|
|
||||||
export TEXINPUTS := $(TEMPLATE_DIR):
|
|
||||||
export BSTINPUTS := $(TEMPLATE_DIR):
|
|
||||||
|
|
||||||
.PHONY: all clean
|
.PHONY: all clean
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
This is the editable paper directory. It uses the unmodified official ICASSP
|
This is the editable paper directory. It uses the unmodified official ICASSP
|
||||||
2026 LaTeX assets stored in `../ICASSP2026_Paper_Templates/`. When the ICASSP
|
2026 LaTeX assets stored in `../ICASSP2026_Paper_Templates/`. When the ICASSP
|
||||||
2027 author kit is released, replace or add the official template and update
|
2027 author kit is released, replace or add the official template and update
|
||||||
`TEMPLATE_DIR` in the Makefile only after checking the new instructions.
|
`template_dir` in `latexmkrc` only after checking the new instructions.
|
||||||
|
|
||||||
Compile from this directory:
|
Compile from this directory:
|
||||||
|
|
||||||
@@ -14,12 +14,19 @@ make
|
|||||||
The PDF and all intermediate files are written to `build/`; only source files
|
The PDF and all intermediate files are written to `build/`; only source files
|
||||||
belong in Git.
|
belong in Git.
|
||||||
|
|
||||||
|
VS Code's LaTeX Workshop extension is supported when the repository is opened
|
||||||
|
as the workspace. Open `main.tex` and run **LaTeX Workshop: Build LaTeX
|
||||||
|
project**. The workspace setting writes output to `build/`, while `latexmkrc`
|
||||||
|
provides the template and bibliography-style search paths to both LaTeX
|
||||||
|
Workshop and command-line builds.
|
||||||
|
|
||||||
Current source layout:
|
Current source layout:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
main/
|
main/
|
||||||
main.tex
|
main.tex
|
||||||
references.bib
|
references.bib
|
||||||
|
latexmkrc
|
||||||
Makefile
|
Makefile
|
||||||
README.md
|
README.md
|
||||||
build/ # generated, ignored
|
build/ # generated, ignored
|
||||||
|
|||||||
@@ -0,0 +1,9 @@
|
|||||||
|
my $path_separator = $^O eq 'MSWin32' ? ';' : ':';
|
||||||
|
my $template_dir = '../ICASSP2026_Paper_Templates';
|
||||||
|
|
||||||
|
$ENV{'TEXINPUTS'} =
|
||||||
|
$template_dir . $path_separator . ($ENV{'TEXINPUTS'} // '');
|
||||||
|
$ENV{'BSTINPUTS'} =
|
||||||
|
$template_dir . $path_separator . ($ENV{'BSTINPUTS'} // '');
|
||||||
|
|
||||||
|
$out_dir = 'build';
|
||||||
+38
-44
@@ -169,7 +169,7 @@ $\mathcal P_\tau=\{(\vect x_{\tau,i},\vect b_{\tau,i})\}_{i=1}^{N_\tau}$,
|
|||||||
a shared visual encoder and masked set operator produce
|
a shared visual encoder and masked set operator produce
|
||||||
\begin{equation}
|
\begin{equation}
|
||||||
\vect v_\tau=\mathrm{SetPool}
|
\vect v_\tau=\mathrm{SetPool}
|
||||||
\left(\{E_v(\vect x_{\tau,i},\vect b_{\tau,i})}_{i=1}^{N_\tau}
|
\left(\{E_v(\vect x_{\tau,i},\vect b_{\tau,i})\}_{i=1}^{N_\tau}
|
||||||
\cup\{\vect v_\varnothing\}\right),
|
\cup\{\vect v_\varnothing\}\right),
|
||||||
\label{eq:set}
|
\label{eq:set}
|
||||||
\end{equation}
|
\end{equation}
|
||||||
@@ -219,16 +219,44 @@ its label/input boundary is fully reported.}
|
|||||||
\section{Experiments}
|
\section{Experiments}
|
||||||
\label{sec:experiments}
|
\label{sec:experiments}
|
||||||
|
|
||||||
\textbf{Data and split.}
|
\subsection{Dataset and Data Preparation}
|
||||||
We follow the EgoSpeak EasyCom split: Sessions 4--12 for training-side model
|
|
||||||
development and Sessions 1--3 for final reporting \cite{donley2021easycom}.
|
|
||||||
Thresholds, windows, checkpoints, loss weights, and early stopping are selected
|
|
||||||
without using the official test sessions. Already observed test results are
|
|
||||||
treated as exploratory internal evidence and are not included in the paper.
|
|
||||||
\todo{Report duration, frame/event counts, class prevalence, visibility and
|
|
||||||
box coverage, source discontinuities, and the frozen calibration fold.}
|
|
||||||
|
|
||||||
\textbf{Baselines.}
|
We evaluate on EasyCom following the EgoSpeak split, using Sessions 4--12 for training and development and Sessions 1--3 for final evaluation \cite{donley2021easycom}. All hyperparameters, event windows, operating thresholds, and checkpoints are selected without access to the test sessions. The input stream is converted to strictly causal examples at 5~Hz. Each decision point is paired with ten future 0.2-s activity bins for background, camera wearer, and aggregate non-wearer speech. Provided person boxes are used only to construct the visible-person inputs; participant identity, close-mic audio, transcripts, and future observations are not available to the model. \todo{Report dataset duration, number of training/test examples and events, and the frozen development/calibration split.}
|
||||||
|
|
||||||
|
\subsection{Model Configuration and Training}
|
||||||
|
|
||||||
|
Our main backbone is Qwen2.5-Omni-3B. We compare a frozen global Omni readout, matched PEFT/LoRA adaptation, global features with recent local audio, a parameter- and history-matched temporal local branch, and the proposed causal local audio--visual adapter. Qwen2.5-Omni-7B is used only as a capacity-scaling control. All learned variants use the same causal context, prediction head, training examples, and class-balanced binary cross-entropy objective in Eq.~\eqref{eq:loss}. Unless otherwise stated, model selection is performed on the development data and final numbers are averaged over three random seeds. \todo{Freeze context length, local window/rates, optimizer, epochs, LoRA configuration, trainable parameter counts, and inference latency.}
|
||||||
|
|
||||||
|
\subsection{Evaluation Metrics}
|
||||||
|
|
||||||
|
We evaluate both dense future-activity prediction and event-level floor
|
||||||
|
control. For comparability with EgoSpeak, we report per-frame AP at
|
||||||
|
$0.2$--$2.0$~s horizons and their mean mAP.
|
||||||
|
|
||||||
|
For event-level evaluation, a \emph{wearer entry} is a transition from wearer
|
||||||
|
silence to speech, while a \emph{wearer yield} is a transition from speech to
|
||||||
|
silence. An entry prediction is considered correct if it is committed within
|
||||||
|
the predefined response window of a ground-truth wearer onset. We report entry
|
||||||
|
recall, false entries per minute of eligible wearer-silent time, and signed
|
||||||
|
onset latency. Yield events are evaluated analogously using recall and latency.
|
||||||
|
|
||||||
|
Multiparty interaction introduces a competing-risk condition. After another
|
||||||
|
speaker releases the floor, the current wearer-entry opportunity is terminated
|
||||||
|
if another non-wearer participant starts speaking before the wearer does.
|
||||||
|
Predictions after this competing onset are therefore not matched to the
|
||||||
|
previous opportunity. This prevents an other-to-other handover from being
|
||||||
|
treated as an open floor for the wearer.
|
||||||
|
|
||||||
|
Operating thresholds are selected on the development split and frozen for test
|
||||||
|
evaluation. Our primary behavioral metric is false wearer entries per minute at
|
||||||
|
a matched wearer-entry recall of $0.8$; dense mAP and wearer-yield performance
|
||||||
|
are reported as complementary measures.
|
||||||
|
|
||||||
|
|
||||||
|
\section{Results and Analysis}
|
||||||
|
\label{sec:results}
|
||||||
|
|
||||||
|
\subsection{Baselines}
|
||||||
The confirmatory comparison contains: an EgoSpeak reproduction; the frozen
|
The confirmatory comparison contains: an EgoSpeak reproduction; the frozen
|
||||||
Qwen2.5-Omni-3B global state $G$; matched LoRA/PEFT; $G$ plus recent local
|
Qwen2.5-Omni-3B global state $G$; matched LoRA/PEFT; $G$ plus recent local
|
||||||
audio; a parameter- and history-matched temporal local branch; and the proposed
|
audio; a parameter- and history-matched temporal local branch; and the proposed
|
||||||
@@ -237,40 +265,6 @@ not a second paper storyline. \todo{Freeze all baseline implementations and
|
|||||||
ensure the EgoSpeak comparison uses identical targets, split, causal validity,
|
ensure the EgoSpeak comparison uses identical targets, split, causal validity,
|
||||||
and evaluation.}
|
and evaluation.}
|
||||||
|
|
||||||
\textbf{Event protocol.}
|
|
||||||
Dense ten-horizon AP/mAP is retained for EgoSpeak comparability. The implemented
|
|
||||||
competing-risk evaluator counts each eligible aggregate-other boundary once,
|
|
||||||
ends a response window before the next other onset, and ends a yield window
|
|
||||||
before the next other offset. Pending the remaining P0 real-data audit and
|
|
||||||
scenario freeze, the intended primary behavior metric is false wearer entries per
|
|
||||||
minute of multi-other opportunity at a training-side selected wearer-onset
|
|
||||||
recall, with $R=.8$ as the headline operating point. The intended analysis also
|
|
||||||
reports onset latency and wearer-yield recall/latency, following the
|
|
||||||
event-oriented emphasis on recall, false alarms, and latency
|
|
||||||
\cite{jiang2026turnbench}. The protocol will censor a response window after an
|
|
||||||
other offset at the next other onset, and a yield window after an other onset
|
|
||||||
at the next other offset.
|
|
||||||
|
|
||||||
\todo{P0 blocker: run the tested competing-risk evaluator on real predictions,
|
|
||||||
manually audit sampled timelines, and freeze pairing before treating any event
|
|
||||||
score as a paper result. Specify overlap,
|
|
||||||
minimum duration, timeouts, duplicate detections, source boundaries, offscreen
|
|
||||||
cases, and event counts. Until this is complete, call existing values
|
|
||||||
timing-derived proxies rather than final event results.}
|
|
||||||
|
|
||||||
\textbf{Controls and statistics.}
|
|
||||||
The main intervention compares correctly aligned local evidence with
|
|
||||||
time-shifted evidence, removed vision, global-frame visual pooling, and the
|
|
||||||
capacity-matched temporal branch. A same-rate control is required for a
|
|
||||||
dual-rate claim. Track/no-ID/broken-continuity controls are added only if a
|
|
||||||
track-aware variant first wins on training-side data. We report three seeds,
|
|
||||||
mean$\pm$sample standard deviation, and paired cluster-bootstrap intervals over
|
|
||||||
sessions or conversation blocks. \todo{Freeze seeds, bootstrap unit/count,
|
|
||||||
optimizer, epochs, hardware, throughput, P50/P95 latency, and trainable
|
|
||||||
parameters.}
|
|
||||||
|
|
||||||
\section{Results and Analysis}
|
|
||||||
\label{sec:results}
|
|
||||||
|
|
||||||
\begin{table}[t]
|
\begin{table}[t]
|
||||||
\caption{Confirmatory EasyCom results. Lower multi-other false entries are
|
\caption{Confirmatory EasyCom results. Lower multi-other false entries are
|
||||||
|
|||||||
Reference in New Issue
Block a user