wx44wx/icassp2027-experiment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add wide evaluation table and qualitative placeholder

Browse Source
This commit is contained in:
Xin Wang
2026-09-16 17:12:31 +08:00
parent d294bd6ede
commit 45facc8bf4
1 changed files with 83 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
manuscript/main/main.tex
+83 -37
View File
@@ -339,9 +339,12 @@ non-wearer release. It resolves positively when the wearer enters, or
negatively when the same non-wearer resumes or a different non-wearer takes the negatively when the same non-wearer resumes or a different non-wearer takes the
next floor. From the first causal 5-Hz state that observes the release, the next floor. From the first causal 5-Hz state that observes the release, the
entry score is the maximum projected wearer probability before that competing entry score is the maximum projected wearer probability before that competing
resolution, capped at 2.0~s. Positive entries without a reviewer-selected resolution, capped at 2.0~s. A selected-release positive is excluded if another
release are reported as unanchored and excluded from this matched-opportunity non-wearer onset occurs before or exactly at the wearer entry, or if wearer
analysis; they remain available to the separate raw-onset analysis. activity already occurred after the selected release. Positive entries without
a reviewer-selected release are reported as unanchored and excluded from this
matched-opportunity analysis; they remain available to the separate raw-onset
analysis.
An INT opportunity begins when a non-wearer starts while the wearer holds the An INT opportunity begins when a non-wearer starts while the wearer holds the
floor. It resolves positively when the reviewed behavior is an INT-yield, and floor. It resolves positively when the reviewed behavior is an INT-yield, and
@@ -352,8 +355,9 @@ strictly at the competing non-wearer offset. All prediction values are read
diagonally from one causal source row of the target-aligned forecast tensor. diagonally from one causal source row of the target-aligned forecast tensor.
No input after the observed action outcome is used. For current INT-positive No input after the observed action outcome is used. For current INT-positive
annotations, which do not select a competing onset explicitly, the annotations, which do not select a competing onset explicitly, the
implementation uses the latest supplied non-wearer interval crossing the implementation evaluates only a unique supplied non-wearer interval crossing
wearer offset and surfaces cases with multiple eligible intervals for audit. the wearer offset; cases with multiple eligible intervals are excluded and
counted for audit rather than assigned an arbitrary onset.
The headline entry hard negatives are same-speaker mid-turn pauses and other- The headline entry hard negatives are same-speaker mid-turn pauses and other-
to-other transfers. Wearer backchannels are not pooled into headline false- to-other transfers. Wearer backchannels are not pooled into headline false-
@@ -394,42 +398,57 @@ latency claim.
\subsection{Main Comparison} \subsection{Main Comparison}
Table~\ref{tab:main} compares the proposed causal local audiovisual adapter Table~\ref{tab:main} places the released EgoSpeak checkpoint beside the
with an EgoSpeak reproduction, the frozen Qwen2.5-Omni-3B global state $G$, currently available Omni screening runs and reserves a row for the final causal
matched PEFT, $G$ plus recent local audio, and a parameter- and history-matched local audiovisual model. Historical unmatched runs are identified explicitly
temporal local branch. Qwen2.5-Omni-7B is included only as a capacity-scaling and cannot support confirmatory attribution. The final comparison will use the
control. All methods use the same split, causal inputs, targets, and evaluation same split, causal inputs, targets, training/selection protocol, and three
protocol. Event thresholds are selected on training-side calibration data and random seeds; event thresholds will be selected on training-side calibration
frozen before test evaluation. \todo{Freeze all baseline implementations and data and frozen before test evaluation. \todo{Freeze all baseline
verify identical targets, causal validity, and evaluation.} implementations and verify identical targets, causal validity, and evaluation.}
\begin{table}[t] \begin{table*}[t]
\caption{Confirmatory EasyCom results. Entry and INT false-positive rates use \caption{EasyCom Sessions~1--3 dense future-activity and raw-boundary
the opportunity-conditioned activity score at matched recall $R=0.8$; E/I AP screening. All values are percentages. ``Avg.'' averages dense mAP over the ten
is the corresponding opportunity AP. Event results will be populated only horizons; On/Off are mean raw wearer onset/offset AP and are timing diagnostics,
after protocol-v2 reruns and human adjudication.} not semantic ENTRY/INT scores. $\dagger$ denotes historical seed-0 runs with an
unmatched training/selection protocol; $\ddagger$ denotes a three-seed screen.
Missing horizon curves and the final matched comparison will be filled after
the matched dense/boundary reruns.}
\label{tab:main} \label{tab:main}
\centering \centering
\small \scriptsize
\resizebox{\columnwidth}{!}{% \setlength{\tabcolsep}{2.6pt}
\begin{tabular}{lccccc} \resizebox{\textwidth}{!}{%
\begin{tabular}{@{}lccccccccccccc@{}}
\hline \hline
Method & 0.2-s mAP $\uparrow$ & Mean mAP $\uparrow$ & \multicolumn{10}{c}{Dense mAP at prediction horizon (s) $\uparrow$}
& Entry FPR@$R=.8$ $\downarrow$ & INT FPR@$R=.8$ $\downarrow$ & E/I AP $\uparrow$\\ & & \multicolumn{2}{c}{Raw boundary AP $\uparrow$}\\
\cline{2-11}\cline{13-14}
Method & .2 & .4 & .6 & .8 & 1.0 & 1.2 & 1.4 & 1.6 & 1.8 & 2.0
& Avg. & On & Off\\
\hline \hline
EgoSpeak MiniROADA A+V & 74.12 & 61.16 & \todo{--} & \todo{--} & \todo{--/--}\\ EgoSpeak MiniROADA A+V
Omni-3B frozen $G$ & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\ & 74.20 & 68.35 & 64.31 & 61.13 & 59.77 & 58.81 & 57.50 & 57.09 & 55.95 & 54.82
Omni-3B matched PEFT & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\ & 61.19 & 24.49 & 38.02\\
$G+$local audio & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\ Omni $G{+}A^{\dagger}$
$G+$matched temporal local & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\ & 81.72 & -- & -- & -- & -- & -- & -- & -- & -- & --
$G+$causal local AV (ours) & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\ & 65.11 & 25.96 & 45.25\\
Omni-7B scaling control & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\ Omni $G{+}A{+}$pooled-$V^{\dagger}$
& 82.03 & -- & -- & -- & -- & -- & -- & -- & -- & --
& 65.40 & 26.02 & 45.37\\
Current-frame F1$^{\ddagger}$
& $81.78{\pm}0.16$ & -- & -- & -- & -- & -- & -- & -- & -- & --
& $64.95{\pm}0.09$ & -- & --\\
$G+$causal local AV (ours)
& -- & -- & -- & -- & -- & -- & -- & -- & -- & --
& -- & -- & --\\
\hline \hline
\end{tabular}} \end{tabular}}
\end{table} \end{table*}
\textbf{Dense forecasting.} \textbf{Dense forecasting.}
The EgoSpeak baseline reaches 74.12\% 0.2-s mAP and 61.16\% mean mAP; its raw The EgoSpeak baseline reaches 74.20\% 0.2-s mAP and 61.19\% mean mAP; its raw
wearer-onset/offset mean APs are 24.49/38.02\%. Opportunity-conditioned values wearer-onset/offset mean APs are 24.49/38.02\%. Opportunity-conditioned values
remain pending a protocol-v2 rerun and human review; continuous wearer-score remain pending a protocol-v2 rerun and human review; continuous wearer-score
plots do not alone establish semantic floor-yield selectivity. plots do not alone establish semantic floor-yield selectivity.
@@ -459,6 +478,34 @@ beyond additional temporal capacity}.
\todo{If this difference is not statistically supported, state instead that \todo{If this difference is not statistically supported, state instead that
the local temporal branch accounts for most of the gain.} the local temporal branch accounts for most of the gain.}
\subsection{Qualitative Floor-Event Analysis}
We will compare EgoSpeak with the final matched model on human-confirmed
examples selected by pre-declared event subtype rather than post-hoc score
extremes. Each case will align participant-resolved activity, minimal transcript
context, the opportunity anchor and competing-risk censor, and the two models'
causally emitted wearer forecasts. The comparison will include a genuine wearer
entry, an other-to-other false-entry opportunity, and an INT-yield paired with a
non-takeover overlap; a shared failure case will be retained when space permits.
\begin{figure*}[t]
\centering
\fbox{\parbox[c][0.115\textheight][c]{0.965\textwidth}{
\centering
\textbf{Qualitative comparison placeholder}\\[0.7em]
\begin{tabular}{ccc}
Case A: wearer entry & Case B: other$\rightarrow$other HN
& Case C: INT-yield / failed takeover\\
participant timeline + transcript & opportunity anchor + censor
& EgoSpeak vs. proposed forecast\\
\end{tabular}
}}
\caption{Planned qualitative floor-event comparison. Final panels will use
human-confirmed protocol-v2 cases and show the same causal source time for both
models; the figure is a layout placeholder and contains no result claim.}
\label{fig:qualitative_floor_events}
\end{figure*}
\subsection{Information Opportunity} \subsection{Information Opportunity}
Before designing the deployable adapter, we measured whether local floor Before designing the deployable adapter, we measured whether local floor
@@ -569,7 +616,7 @@ future-media intervention test. Report within-backbone 5-Hz versus 1-Hz
deltas plus cache/no-cache throughput and memory; do not interpret unmatched deltas plus cache/no-cache throughput and memory; do not interpret unmatched
Qwen--MiniCPM raw scores as a pure backbone effect.} Qwen--MiniCPM raw scores as a pure backbone effect.}
\subsection{Limitations and Qualitative Analysis} \subsection{Limitations}
The results support \todo{precise supported claim}, but several limitations The results support \todo{precise supported claim}, but several limitations
remain. remain.
@@ -579,10 +626,9 @@ tracking assumption}.
Third, \todo{failure condition: occlusion, overlapping speakers, rapid floor Third, \todo{failure condition: occlusion, overlapping speakers, rapid floor
exchange, etc.} remains challenging. exchange, etc.} remains challenging.
\todo{Add one compact, pre-selected qualitative timeline only if space permits: \todo{Replace Fig.~\ref{fig:qualitative_floor_events} with compact, pre-selected
one multi-other false-entry improvement paired with one preserved real wearer human-confirmed timelines. Examples illustrate aggregate rates and never
onset or one failure. Examples illustrate aggregate rates and never establish establish the claim by themselves.}
the claim by themselves.}
\section{Conclusion} \section{Conclusion}
\label{sec:conclusion} \label{sec:conclusion}