Add EgoSpeak dense event proxy evaluation
This commit is contained in:
+37
-22
@@ -328,32 +328,47 @@ configuration, trainable parameter counts, and inference latency.}
|
||||
We first retain dense future-activity evaluation for direct comparison with
|
||||
EgoSpeak, reporting AP at the 0.2--2.0~s horizons and their mean mAP.
|
||||
|
||||
Behavioral evaluation uses fixed-horizon event classification. For both
|
||||
wearer-entry and INT-yield, we align the dense forecast to each adjudicated
|
||||
item at each of the ten nominal horizons. A positive remains a single corrected
|
||||
5-Hz behavior-anchor bin: its entry score is wearer activity probability and
|
||||
its INT-yield score is one minus wearer activity probability. We do not expand
|
||||
the positive timestamp into a tolerance window. A hard-negative opportunity is
|
||||
one sample whose score is the maximum erroneous-action probability over its
|
||||
labeled half-open region at the same fixed horizon. Thus, each positive event
|
||||
or hard-negative opportunity contributes exactly one score per horizon.
|
||||
For dense-only models, behavioral evaluation begins with an explicitly
|
||||
activity-derived event proxy rather than treating wearer voice activity as a
|
||||
semantic event detector. Following the prediction-region principle used by
|
||||
VAP~\cite{ekstedt2022vap}, every adjudicated item contributes one score from a
|
||||
fixed 0.6-s (three-frame) causal decision region. A positive uses the three
|
||||
source frames immediately preceding its corrected behavior anchor. An
|
||||
independently mined hard negative uses the first three source frames after its
|
||||
opportunity begins; a raw wearer boundary relabeled as a hard negative instead
|
||||
uses the same pre-boundary region as a positive. Items without a complete
|
||||
three-frame region are excluded and counted. The longer interval supplied to
|
||||
the annotator is evidence context only and is never a model-scoring window.
|
||||
|
||||
At each decision frame, we follow the target-aligned forecast tensor
|
||||
diagonally to recover the wearer probabilities emitted at that same causal
|
||||
source time. We average the 0.6--2.0-s horizons to obtain a sustained wearer-
|
||||
activity score. This mean is the entry proxy, and one minus the mean is the
|
||||
INT-yield proxy. Max-pooling the three decision-frame scores gives exactly one
|
||||
score per positive event or hard-negative opportunity. This construction can
|
||||
capture sustained activity versus a brief vocalization, but it cannot by
|
||||
itself establish semantic floor intent; in particular, wearer backchannels and
|
||||
within-turn pauses remain deliberately difficult negatives. A later linear
|
||||
event probe, if used, is fit only on training-side annotations and compared at
|
||||
matched capacity across representations, following the separated downstream-
|
||||
probe logic of MuVAP~\cite{qi2026muvap}.
|
||||
|
||||
\textbf{False-positive scoring rule.}
|
||||
A false positive is a hard-negative opportunity whose region-level score
|
||||
A false positive is a hard-negative opportunity whose fixed-region proxy score
|
||||
exceeds the frozen threshold, regardless of whether the wearer subsequently
|
||||
exhibits an onset or offset. Region max-pooling means that multiple high-scoring
|
||||
frames still count as one false-positive opportunity. Thus, false-positive
|
||||
exhibits an onset or offset. Max-pooling means that multiple high-scoring
|
||||
decision frames still count as one false-positive opportunity. Thus, false-positive
|
||||
evaluation includes conversational situations in which the correct wearer
|
||||
behavior is to remain silent or retain the floor.
|
||||
|
||||
For every horizon we report event AP and the recall--false-positive trade-off.
|
||||
The headline behavioral operating point uses the first, nominal 0.2-s forecast
|
||||
bin at matched recall $R=0.8$; thresholds are selected independently for entry
|
||||
and INT from training-side out-of-fold predictions. We additionally report the
|
||||
complete 0.2--2.0-s curves and the other-to-other hard-negative subset to test
|
||||
We report threshold-free proxy AP for entry and INT. The headline behavioral
|
||||
operating point additionally reports false-positive rate at matched recall
|
||||
$R=0.8$; thresholds are selected independently for entry and INT from training-
|
||||
side out-of-fold predictions. The other-to-other hard-negative subset tests
|
||||
whether local visual evidence specifically reduces erroneous wearer action
|
||||
scores during multiparty floor transfers. \todo{Freeze the hard-negative
|
||||
region endpoints and verify identical event denominators across methods.}
|
||||
scores during multiparty floor transfers. \todo{Freeze the 0.6-s decision
|
||||
window and 0.6--2.0-s aggregation range using training-side data, run window
|
||||
sensitivity analysis, and verify identical event denominators across methods.}
|
||||
|
||||
A deployment-style committed-event analysis with one-to-one matching and
|
||||
signed latency is secondary and is omitted unless a threshold, persistence,
|
||||
@@ -378,8 +393,8 @@ verify identical targets, causal validity, and evaluation.}
|
||||
|
||||
\begin{table}[t]
|
||||
\caption{Confirmatory EasyCom results. Entry and INT false-positive rates use
|
||||
the nominal 0.2-s forecast bin at matched recall $R=0.8$; E/I AP averages each
|
||||
event family over all ten fixed horizons.}
|
||||
the fixed-region activity-derived proxy at matched recall $R=0.8$; E/I AP is
|
||||
the corresponding event-balanced proxy AP.}
|
||||
\label{tab:main}
|
||||
\centering
|
||||
\small
|
||||
@@ -411,7 +426,7 @@ evidence primarily improves \todo{the interpretation supported by the data}.
|
||||
\textbf{Annotated behavior.}
|
||||
At matched wearer-entry recall $R=0.8$, the proposed adapter reduces entry FPR
|
||||
from \todo{$X$} to \todo{$Y$}; at matched INT-yield recall, it changes INT FPR
|
||||
from \todo{$A$} to \todo{$B$}. Its ten-horizon mean entry/INT event APs are
|
||||
from \todo{$A$} to \todo{$B$}. Its entry/INT activity-proxy APs are
|
||||
\todo{$E/I$}. These recall-matched opportunity error rates test whether lower
|
||||
scores on confusing floor situations are obtained without suppressing
|
||||
legitimate entry or yield events.
|
||||
|
||||
Reference in New Issue
Block a user