docs: add streaming cache ablation to manuscript
This commit is contained in:
@@ -47,7 +47,10 @@ Its single candidate contribution is a causal local floor-evidence adapter:
|
||||
- the primary local method uses an anonymous, no-ID visible-person set;
|
||||
- track continuity is conditional and must defeat no-ID/shuffled/capacity
|
||||
controls before entering the paper;
|
||||
- MiniCPM-o and a cross-family sweep are removed from the core plan;
|
||||
- MiniCPM-o remains outside the core method and main comparison; it appears
|
||||
only as a scoped streaming/cache diagnostic with a matched Qwen 1-Hz
|
||||
causal-hold control, and its 5-Hz cache row requires equivalence and leakage
|
||||
audits before reporting;
|
||||
- the oracle is analysis-only motivation, not a deployable method or headline;
|
||||
- the publication gate is lower competing-risk-censored multi-other false
|
||||
entry at matched wearer-onset recall without worse valid onset or yield.
|
||||
|
||||
@@ -17,7 +17,9 @@ behavior-aligned multi-other event protocol.
|
||||
Current Qwen2.5-Omni-3B results are exploratory motivation only. They neither
|
||||
prove a limitation shared by Omni models nor establish track-consistent fusion
|
||||
as the final method. Qwen2.5-Omni-3B is primary; Qwen2.5-Omni-7B is an optional
|
||||
capacity control. MiniCPM-o and a cross-family sweep are outside the core paper.
|
||||
capacity control. MiniCPM-o remains outside the core method and main comparison;
|
||||
it is permitted only as a diagnostic streaming/cache ablation with a matched
|
||||
Qwen refresh-rate control.
|
||||
|
||||
The four-page paper has only three contributions: temporal--role adaptation
|
||||
mismatch, one causal local floor adapter, and causal behavioral evidence.
|
||||
@@ -63,6 +65,26 @@ a model cannot expose a comparable causal state, report that protocol as
|
||||
unsupported. Raw scores are not a pure architecture ranking; emphasize
|
||||
within-backbone deltas.
|
||||
|
||||
For the diagnostic streaming/cache ablation:
|
||||
|
||||
- [ ] compare Qwen full causal-prefix recomputation and cache reuse at the
|
||||
same 5-Hz cutoffs, recording numerical agreement, throughput, and memory;
|
||||
- [ ] map the native MiniCPM-o one-second output to 5-Hz rows by causal
|
||||
forward-hold only, never by backfilling the state into earlier rows;
|
||||
- [ ] run a Qwen 1-Hz update-and-hold control with the same physical chunk
|
||||
boundaries and report the within-Qwen 5-Hz versus 1-Hz delta;
|
||||
- [ ] freeze initialization before the first completed chunk and reset every
|
||||
modality/language-model cache at session and source-gap boundaries;
|
||||
- [ ] before admitting a MiniCPM-o 5-Hz cached-branch row, compare it against
|
||||
full causal-prefix recomputation at sampled cutoffs under a frozen cosine and
|
||||
maximum-absolute-error tolerance;
|
||||
- [ ] run future-media intervention tests at every tested cutoff and record
|
||||
throughput, peak memory, feature dimensionality, and exact code revision.
|
||||
|
||||
If the 5-Hz cached branch fails equivalence or leakage tests, omit that row and
|
||||
report only the native 1-Hz causal-hold diagnostic. Do not present unmatched
|
||||
Qwen--MiniCPM raw scores as a pure backbone comparison.
|
||||
|
||||
## P2 — Establish the compact adaptation matrix
|
||||
|
||||
Run three seeds for the primary backbone with the same data, physical cutoffs,
|
||||
@@ -144,17 +166,23 @@ Examples explain rates; they never replace them.
|
||||
2. Audit the Qwen2.5-Omni-3B readout and matched PEFT baseline.
|
||||
3. Complete the frozen/PEFT/audio/matched-temporal/local-AV matrix.
|
||||
4. Run correct/time-shift/remove-vision controls and the optional 7B scaling row.
|
||||
5. Consider track continuity only if the no-ID method already passes.
|
||||
6. Produce one confirmatory event table and one mechanism-ablation table.
|
||||
7. Draw one compact final system figure; remove rejected methods from the main
|
||||
5. Run the Qwen refresh-rate control and the scoped MiniCPM-o streaming/cache
|
||||
diagnostic; admit the 5-Hz MiniCPM-o row only after equivalence and leakage
|
||||
audits.
|
||||
6. Consider track continuity only if the no-ID method already passes.
|
||||
7. Produce one confirmatory event table and one mechanism-ablation table.
|
||||
8. Draw one compact final system figure; remove rejected methods from the main
|
||||
narrative.
|
||||
8. Replace every visible TODO and verify the ICASSP 2027 kit/page limit.
|
||||
9. Replace every visible TODO and verify the ICASSP 2027 kit/page limit.
|
||||
|
||||
## Stop/go decisions
|
||||
|
||||
- **Claim boundary:** with Qwen-only evidence, describe a Qwen2.5-Omni case
|
||||
study and the tested adaptation mechanism; do not generalize to all Omni
|
||||
models.
|
||||
- **Streaming boundary:** call native MiniCPM-o 1-Hz plus causal hold a held-state
|
||||
baseline, not native 5-Hz inference; a 5-Hz cache result exists only after
|
||||
cached/full-prefix equivalence and future-input invariance pass.
|
||||
- **Audio-local reframe:** local audio improves but all visual variants fail;
|
||||
claim a fine-grained acoustic readout bottleneck, not social visual reasoning.
|
||||
- **Track-aware method:** correct continuity beats pooled vision, temporal
|
||||
|
||||
@@ -157,8 +157,9 @@ is not interpreted as a general limitation of LoRA or all Omni models.
|
||||
\todo{Freeze the exact Qwen2.5-Omni-3B revision, processor, input rates,
|
||||
streaming/cache semantics, readout layer, context, dtype, LoRA targets/rank,
|
||||
trainable parameter budget, and cutoff audit. Add Qwen2.5-Omni-7B only as a
|
||||
capacity-scaling control. Do not add another backbone family unless it is
|
||||
needed to support and experimentally earns a cross-family claim.}
|
||||
capacity-scaling control. MiniCPM-o 4.5 is restricted to the diagnostic
|
||||
streaming/cache ablation below and does not enter the core method or support a
|
||||
cross-family superiority claim.}
|
||||
|
||||
\subsection{Causal local floor adapter}
|
||||
|
||||
@@ -267,6 +268,17 @@ control. All learned variants use identical causal examples, prediction
|
||||
horizons, prediction heads, and the class-balanced binary cross-entropy
|
||||
objective in Eq.~\eqref{eq:loss}.
|
||||
|
||||
The diagnostic streaming ablation additionally uses MiniCPM-o 4.5
|
||||
\cite{cui2026minicpmo45}. Its native cached stream is evaluated at its native
|
||||
one-second update interval: a state becomes available to the 200-ms row whose
|
||||
cutoff equals the chunk end and is then causally held until the next update. A
|
||||
Qwen row subjected to the same 1-Hz update-and-hold schedule separates
|
||||
refresh-rate effects from backbone effects. A MiniCPM-o
|
||||
5-Hz cache row is admissible only if a cached branch at every cutoff agrees
|
||||
with full causal-prefix recomputation within a frozen numerical tolerance and
|
||||
passes future-input intervention tests. Thus the native 1-Hz condition is a
|
||||
strictly causal held-state baseline, not a claim of native 5-Hz refresh.
|
||||
|
||||
Model selection is performed using training-side data only. Event operating
|
||||
points are selected from session-level out-of-fold predictions over
|
||||
Sessions~4--12; the final model is then retrained on all nine sessions before
|
||||
@@ -392,27 +404,40 @@ numbers from the four-page version.}
|
||||
\subsection{Mechanism Analysis}
|
||||
|
||||
To determine whether the gain arises from correctly aligned visual floor
|
||||
evidence rather than additional parameters or generic visual context, we
|
||||
evaluate the controls in Table~\ref{tab:ablation}.
|
||||
evidence rather than additional parameters or generic visual context, and to
|
||||
separate streaming refresh from representation family, we evaluate the
|
||||
controls in Table~\ref{tab:ablation}.
|
||||
|
||||
\begin{table}[t]
|
||||
\caption{Mechanism controls for the selected Omni-3B local adapter. All
|
||||
variants use the same prediction head and comparable temporal capacity; FPRs
|
||||
are measured at matched recall $R=0.8$.}
|
||||
\caption{Mechanism and streaming/cache controls. The streaming panel uses
|
||||
frozen global states and capacity-matched downstream heads; all rows share
|
||||
targets and evaluation. FPRs are measured at matched recall $R=0.8$.
|
||||
Cross-family raw scores are diagnostic.}
|
||||
\label{tab:ablation}
|
||||
\centering
|
||||
\small
|
||||
\resizebox{\columnwidth}{!}{%
|
||||
\begin{tabular}{lcccc}
|
||||
\hline
|
||||
Local condition & Mean mAP $\uparrow$ & EOT FPR $\downarrow$
|
||||
Condition & Mean mAP $\uparrow$ & EOT FPR $\downarrow$
|
||||
& O$\rightarrow$O FPR $\downarrow$ & INT FPR $\downarrow$\\
|
||||
\hline
|
||||
\multicolumn{5}{l}{\emph{Local-evidence controls (Qwen2.5-Omni-3B)}}\\
|
||||
Remove vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Time-shift vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Full-frame vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Correct local AV & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
\hline
|
||||
\multicolumn{5}{l}{\emph{Streaming/cache controls}}\\
|
||||
Qwen, full causal prefix at 5 Hz (no cache)
|
||||
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Qwen, cached 5-Hz update & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
Qwen, cached 1-Hz + causal hold & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
MiniCPM-o, native cached 1-Hz + causal hold
|
||||
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
MiniCPM-o, audited cached 5-Hz branch
|
||||
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||
\hline
|
||||
\end{tabular}}
|
||||
\end{table}
|
||||
|
||||
@@ -430,7 +455,7 @@ evidence-routing claim. If dense mAP improves but event behavior does not, do
|
||||
not claim improved floor control. If legitimate onset or yield degrades, do
|
||||
not describe lower false entry as success.}
|
||||
|
||||
\subsection{Scaling Control}
|
||||
\subsection{Scaling and Streaming Controls}
|
||||
|
||||
Increasing the frozen Omni backbone from 3B to 7B yields
|
||||
\todo{$X$ points} in 0.2-s mAP and \todo{EOT/INT FPR of $Y/Z$} at matched
|
||||
@@ -443,6 +468,25 @@ provides a complementary benefit to backbone scaling}.
|
||||
Because both backbones belong to the Qwen2.5-Omni family, we do not claim that
|
||||
this conclusion generalizes to all omni-modal models.
|
||||
|
||||
The streaming rows answer two different questions. First, full causal-prefix
|
||||
recomputation and streaming cache reuse at the same 5-Hz Qwen cutoffs test
|
||||
whether cache/chunk semantics change the representation; equivalent outputs
|
||||
make this an efficiency control rather than a distinct predictive method.
|
||||
Second, the matched Qwen 5-Hz and 1-Hz cached rows test whether an advantage is
|
||||
explained by more frequent causal state refresh rather than the proposed local
|
||||
evidence route. For native 1-Hz streams, the state emitted at the end of each
|
||||
one-second chunk is never backfilled into earlier 200-ms rows; source gaps and
|
||||
session boundaries reset all modality and language-model caches. MiniCPM-o is
|
||||
a secondary cross-family diagnostic under the same 1-Hz schedule, not a
|
||||
same-frequency architecture ranking. Its 5-Hz cached-branch row is omitted if
|
||||
cached/full-prefix equivalence, cutoff invariance, or reset audits fail.
|
||||
|
||||
\todo{Freeze native chunk boundaries, initialization of rows before the first
|
||||
complete chunk, cache-reset rules, the cached/full-prefix tolerance, and the
|
||||
future-media intervention test. Report within-backbone 5-Hz versus 1-Hz
|
||||
deltas plus cache/no-cache throughput and memory; do not interpret unmatched
|
||||
Qwen--MiniCPM raw scores as a pure backbone effect.}
|
||||
|
||||
\subsection{Limitations and Qualitative Analysis}
|
||||
|
||||
The results support \todo{precise supported claim}, but several limitations
|
||||
|
||||
Reference in New Issue
Block a user