docs: add streaming cache ablation to manuscript
This commit is contained in:
@@ -157,8 +157,9 @@ is not interpreted as a general limitation of LoRA or all Omni models.
|
||||
\todo{Freeze the exact Qwen2.5-Omni-3B revision, processor, input rates,
|
||||
streaming/cache semantics, readout layer, context, dtype, LoRA targets/rank,
|
||||
trainable parameter budget, and cutoff audit. Add Qwen2.5-Omni-7B only as a
|
||||
capacity-scaling control. Do not add another backbone family unless it is
|
||||
needed to support and experimentally earns a cross-family claim.}
|
||||
capacity-scaling control. MiniCPM-o 4.5 is restricted to the diagnostic
|
||||
streaming/cache ablation below and does not enter the core method or support a
|
||||
cross-family superiority claim.}
|
||||
|
||||
\subsection{Causal local floor adapter}
|
||||
|
||||
@@ -267,6 +268,17 @@ control. All learned variants use identical causal examples, prediction
|
||||
horizons, prediction heads, and the class-balanced binary cross-entropy
|
||||
objective in Eq.~\eqref{eq:loss}.
|
||||
|
||||
The diagnostic streaming ablation additionally uses MiniCPM-o 4.5
|
||||
\cite{cui2026minicpmo45}. Its native cached stream is evaluated at its native
|
||||
one-second update interval: a state becomes available to the 200-ms row whose
|
||||
cutoff equals the chunk end and is then causally held until the next update. A
|
||||
Qwen row subjected to the same 1-Hz update-and-hold schedule separates
|
||||
refresh-rate effects from backbone effects. A MiniCPM-o
|
||||
5-Hz cache row is admissible only if a cached branch at every cutoff agrees
|
||||
with full causal-prefix recomputation within a frozen numerical tolerance and
|
||||
passes future-input intervention tests. Thus the native 1-Hz condition is a
|
||||
strictly causal held-state baseline, not a claim of native 5-Hz refresh.
|
||||
|
||||
Model selection is performed using training-side data only. Event operating
|
||||
points are selected from session-level out-of-fold predictions over
|
||||
Sessions~4--12; the final model is then retrained on all nine sessions before
|
||||
@@ -392,27 +404,40 @@ numbers from the four-page version.}
|
||||
\subsection{Mechanism Analysis}
|
||||
|
||||
To determine whether the gain arises from correctly aligned visual floor
|
||||
evidence rather than additional parameters or generic visual context, we
|
||||
evaluate the controls in Table~\ref{tab:ablation}.
|
||||
evidence rather than additional parameters or generic visual context, and to
|
||||
separate streaming refresh from representation family, we evaluate the
|
||||
controls in Table~\ref{tab:ablation}.
|
||||
|
||||
\begin{table}[t]
|
||||
\caption{Mechanism controls for the selected Omni-3B local adapter. All
|
||||
variants use the same prediction head and comparable temporal capacity; FPRs
|
||||
are measured at matched recall $R=0.8$.}
|
||||
\caption{Mechanism and streaming/cache controls. The streaming panel uses
|
||||
frozen global states and capacity-matched downstream heads; all rows share
|
||||
targets and evaluation. FPRs are measured at matched recall $R=0.8$.
|
||||
Cross-family raw scores are diagnostic.}
|
||||
\label{tab:ablation}
|
||||
\centering
|
||||
\small
|
||||
\resizebox{\columnwidth}{!}{%
|
||||
\begin{tabular}{lcccc}
|
||||
\hline
|
||||
Local condition & Mean mAP $\uparrow$ & EOT FPR $\downarrow$
|
||||
Condition & Mean mAP $\uparrow$ & EOT FPR $\downarrow$
|
||||
& O$\rightarrow$O FPR $\downarrow$ & INT FPR $\downarrow$\\
|
||||
\hline
|
||||
\multicolumn{5}{l}{\emph{Local-evidence controls (Qwen2.5-Omni-3B)}}\\
|
||||
Remove vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Time-shift vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Full-frame vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Correct local AV & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
\hline
|
||||
\multicolumn{5}{l}{\emph{Streaming/cache controls}}\\
|
||||
Qwen, full causal prefix at 5 Hz (no cache)
|
||||
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Qwen, cached 5-Hz update & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Qwen, cached 1-Hz + causal hold & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
MiniCPM-o, native cached 1-Hz + causal hold
|
||||
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
MiniCPM-o, audited cached 5-Hz branch
|
||||
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
\hline
|
||||
\end{tabular}}
|
||||
\end{table}
|
||||
|
||||
@@ -430,7 +455,7 @@ evidence-routing claim. If dense mAP improves but event behavior does not, do
|
||||
not claim improved floor control. If legitimate onset or yield degrades, do
|
||||
not describe lower false entry as success.}
|
||||
|
||||
\subsection{Scaling Control}
|
||||
\subsection{Scaling and Streaming Controls}
|
||||
|
||||
Increasing the frozen Omni backbone from 3B to 7B yields
|
||||
\todo{$X$ points} in 0.2-s mAP and \todo{EOT/INT FPR of $Y/Z$} at matched
|
||||
@@ -443,6 +468,25 @@ provides a complementary benefit to backbone scaling}.
|
||||
Because both backbones belong to the Qwen2.5-Omni family, we do not claim that
|
||||
this conclusion generalizes to all omni-modal models.
|
||||
|
||||
The streaming rows answer two different questions. First, full causal-prefix
|
||||
recomputation and streaming cache reuse at the same 5-Hz Qwen cutoffs test
|
||||
whether cache/chunk semantics change the representation; equivalent outputs
|
||||
make this an efficiency control rather than a distinct predictive method.
|
||||
Second, the matched Qwen 5-Hz and 1-Hz cached rows test whether an advantage is
|
||||
explained by more frequent causal state refresh rather than the proposed local
|
||||
evidence route. For native 1-Hz streams, the state emitted at the end of each
|
||||
one-second chunk is never backfilled into earlier 200-ms rows; source gaps and
|
||||
session boundaries reset all modality and language-model caches. MiniCPM-o is
|
||||
a secondary cross-family diagnostic under the same 1-Hz schedule, not a
|
||||
same-frequency architecture ranking. Its 5-Hz cached-branch row is omitted if
|
||||
cached/full-prefix equivalence, cutoff invariance, or reset audits fail.
|
||||
|
||||
\todo{Freeze native chunk boundaries, initialization of rows before the first
|
||||
complete chunk, cache-reset rules, the cached/full-prefix tolerance, and the
|
||||
future-media intervention test. Report within-backbone 5-Hz versus 1-Hz
|
||||
deltas plus cache/no-cache throughput and memory; do not interpret unmatched
|
||||
Qwen--MiniCPM raw scores as a pure backbone effect.}
|
||||
|
||||
\subsection{Limitations and Qualitative Analysis}
|
||||
|
||||
The results support \todo{precise supported claim}, but several limitations
|
||||
|
||||
Reference in New Issue
Block a user