Record EgoSpeak three-level evaluation
This commit is contained in:
@@ -328,6 +328,9 @@ configuration, trainable parameter counts, and inference latency.}
|
||||
|
||||
We first retain dense future-activity evaluation for direct comparison with
|
||||
EgoSpeak, reporting AP at the 0.2--2.0~s horizons and their mean mAP.
|
||||
As a threshold-free diagnostic, we also report wearer OFF$\rightarrow$ON and
|
||||
ON$\rightarrow$OFF anticipation AP; these raw boundaries are timing proxies,
|
||||
not semantic EOT or INT labels.
|
||||
|
||||
For dense-only models, behavioral evaluation is opportunity-conditioned and
|
||||
uses the model's original wearer-activity forecasts rather than treating them
|
||||
@@ -403,7 +406,8 @@ verify identical targets, causal validity, and evaluation.}
|
||||
\begin{table}[t]
|
||||
\caption{Confirmatory EasyCom results. Entry and INT false-positive rates use
|
||||
the opportunity-conditioned activity score at matched recall $R=0.8$; E/I AP
|
||||
is the corresponding event-balanced opportunity AP.}
|
||||
is the corresponding opportunity AP. $\dagger$ denotes provisional
|
||||
LLM-assisted labels pending human review.}
|
||||
\label{tab:main}
|
||||
\centering
|
||||
\small
|
||||
@@ -413,7 +417,7 @@ is the corresponding event-balanced opportunity AP.}
|
||||
Method & 0.2-s mAP $\uparrow$ & Mean mAP $\uparrow$
|
||||
& Entry FPR@$R=.8$ $\downarrow$ & INT FPR@$R=.8$ $\downarrow$ & E/I AP $\uparrow$\\
|
||||
\hline
|
||||
EgoSpeak reproduction & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\
|
||||
EgoSpeak MiniROADA A+V & 74.12 & 61.16 & \todo{--} & \todo{--} & $52.49/20.13^\dagger$\\
|
||||
Omni-3B frozen $G$ & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\
|
||||
Omni-3B matched PEFT & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\
|
||||
$G+$local audio & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--/--}\\
|
||||
@@ -425,6 +429,11 @@ Omni-7B scaling control & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \t
|
||||
\end{table}
|
||||
|
||||
\textbf{Dense forecasting.}
|
||||
The EgoSpeak baseline reaches 74.12\% 0.2-s mAP and 61.16\% mean mAP; its raw
|
||||
wearer-onset/offset mean APs are 24.49/38.02\%. On provisional reviewed
|
||||
opportunities, ENTRY AP is 52.49\% versus 32.73\% prevalence, whereas INT AP is
|
||||
20.13\% versus 20.44\%; continuous wearer-score plots therefore do not alone
|
||||
establish semantic floor-yield selectivity.
|
||||
The proposed local AV adapter improves 0.2-s mAP from
|
||||
\todo{$X$ to $Y$} over the strongest matched baseline and improves
|
||||
mean mAP by \todo{$Z$ points}.
|
||||
|
||||
Reference in New Issue
Block a user