docs: add streaming cache ablation to manuscript
This commit is contained in:
@@ -47,7 +47,10 @@ Its single candidate contribution is a causal local floor-evidence adapter:
|
|||||||
- the primary local method uses an anonymous, no-ID visible-person set;
|
- the primary local method uses an anonymous, no-ID visible-person set;
|
||||||
- track continuity is conditional and must defeat no-ID/shuffled/capacity
|
- track continuity is conditional and must defeat no-ID/shuffled/capacity
|
||||||
controls before entering the paper;
|
controls before entering the paper;
|
||||||
- MiniCPM-o and a cross-family sweep are removed from the core plan;
|
- MiniCPM-o remains outside the core method and main comparison; it appears
|
||||||
|
only as a scoped streaming/cache diagnostic with a matched Qwen 1-Hz
|
||||||
|
causal-hold control, and its 5-Hz cache row requires equivalence and leakage
|
||||||
|
audits before reporting;
|
||||||
- the oracle is analysis-only motivation, not a deployable method or headline;
|
- the oracle is analysis-only motivation, not a deployable method or headline;
|
||||||
- the publication gate is lower competing-risk-censored multi-other false
|
- the publication gate is lower competing-risk-censored multi-other false
|
||||||
entry at matched wearer-onset recall without worse valid onset or yield.
|
entry at matched wearer-onset recall without worse valid onset or yield.
|
||||||
|
|||||||
@@ -17,7 +17,9 @@ behavior-aligned multi-other event protocol.
|
|||||||
Current Qwen2.5-Omni-3B results are exploratory motivation only. They neither
|
Current Qwen2.5-Omni-3B results are exploratory motivation only. They neither
|
||||||
prove a limitation shared by Omni models nor establish track-consistent fusion
|
prove a limitation shared by Omni models nor establish track-consistent fusion
|
||||||
as the final method. Qwen2.5-Omni-3B is primary; Qwen2.5-Omni-7B is an optional
|
as the final method. Qwen2.5-Omni-3B is primary; Qwen2.5-Omni-7B is an optional
|
||||||
capacity control. MiniCPM-o and a cross-family sweep are outside the core paper.
|
capacity control. MiniCPM-o remains outside the core method and main comparison;
|
||||||
|
it is permitted only as a diagnostic streaming/cache ablation with a matched
|
||||||
|
Qwen refresh-rate control.
|
||||||
|
|
||||||
The four-page paper has only three contributions: temporal--role adaptation
|
The four-page paper has only three contributions: temporal--role adaptation
|
||||||
mismatch, one causal local floor adapter, and causal behavioral evidence.
|
mismatch, one causal local floor adapter, and causal behavioral evidence.
|
||||||
@@ -63,6 +65,26 @@ a model cannot expose a comparable causal state, report that protocol as
|
|||||||
unsupported. Raw scores are not a pure architecture ranking; emphasize
|
unsupported. Raw scores are not a pure architecture ranking; emphasize
|
||||||
within-backbone deltas.
|
within-backbone deltas.
|
||||||
|
|
||||||
|
For the diagnostic streaming/cache ablation:
|
||||||
|
|
||||||
|
- [ ] compare Qwen full causal-prefix recomputation and cache reuse at the
|
||||||
|
same 5-Hz cutoffs, recording numerical agreement, throughput, and memory;
|
||||||
|
- [ ] map the native MiniCPM-o one-second output to 5-Hz rows by causal
|
||||||
|
forward-hold only, never by backfilling the state into earlier rows;
|
||||||
|
- [ ] run a Qwen 1-Hz update-and-hold control with the same physical chunk
|
||||||
|
boundaries and report the within-Qwen 5-Hz versus 1-Hz delta;
|
||||||
|
- [ ] freeze initialization before the first completed chunk and reset every
|
||||||
|
modality/language-model cache at session and source-gap boundaries;
|
||||||
|
- [ ] before admitting a MiniCPM-o 5-Hz cached-branch row, compare it against
|
||||||
|
full causal-prefix recomputation at sampled cutoffs under a frozen cosine and
|
||||||
|
maximum-absolute-error tolerance;
|
||||||
|
- [ ] run future-media intervention tests at every tested cutoff and record
|
||||||
|
throughput, peak memory, feature dimensionality, and exact code revision.
|
||||||
|
|
||||||
|
If the 5-Hz cached branch fails equivalence or leakage tests, omit that row and
|
||||||
|
report only the native 1-Hz causal-hold diagnostic. Do not present unmatched
|
||||||
|
Qwen--MiniCPM raw scores as a pure backbone comparison.
|
||||||
|
|
||||||
## P2 — Establish the compact adaptation matrix
|
## P2 — Establish the compact adaptation matrix
|
||||||
|
|
||||||
Run three seeds for the primary backbone with the same data, physical cutoffs,
|
Run three seeds for the primary backbone with the same data, physical cutoffs,
|
||||||
@@ -144,17 +166,23 @@ Examples explain rates; they never replace them.
|
|||||||
2. Audit the Qwen2.5-Omni-3B readout and matched PEFT baseline.
|
2. Audit the Qwen2.5-Omni-3B readout and matched PEFT baseline.
|
||||||
3. Complete the frozen/PEFT/audio/matched-temporal/local-AV matrix.
|
3. Complete the frozen/PEFT/audio/matched-temporal/local-AV matrix.
|
||||||
4. Run correct/time-shift/remove-vision controls and the optional 7B scaling row.
|
4. Run correct/time-shift/remove-vision controls and the optional 7B scaling row.
|
||||||
5. Consider track continuity only if the no-ID method already passes.
|
5. Run the Qwen refresh-rate control and the scoped MiniCPM-o streaming/cache
|
||||||
6. Produce one confirmatory event table and one mechanism-ablation table.
|
diagnostic; admit the 5-Hz MiniCPM-o row only after equivalence and leakage
|
||||||
7. Draw one compact final system figure; remove rejected methods from the main
|
audits.
|
||||||
|
6. Consider track continuity only if the no-ID method already passes.
|
||||||
|
7. Produce one confirmatory event table and one mechanism-ablation table.
|
||||||
|
8. Draw one compact final system figure; remove rejected methods from the main
|
||||||
narrative.
|
narrative.
|
||||||
8. Replace every visible TODO and verify the ICASSP 2027 kit/page limit.
|
9. Replace every visible TODO and verify the ICASSP 2027 kit/page limit.
|
||||||
|
|
||||||
## Stop/go decisions
|
## Stop/go decisions
|
||||||
|
|
||||||
- **Claim boundary:** with Qwen-only evidence, describe a Qwen2.5-Omni case
|
- **Claim boundary:** with Qwen-only evidence, describe a Qwen2.5-Omni case
|
||||||
study and the tested adaptation mechanism; do not generalize to all Omni
|
study and the tested adaptation mechanism; do not generalize to all Omni
|
||||||
models.
|
models.
|
||||||
|
- **Streaming boundary:** call native MiniCPM-o 1-Hz plus causal hold a held-state
|
||||||
|
baseline, not native 5-Hz inference; a 5-Hz cache result exists only after
|
||||||
|
cached/full-prefix equivalence and future-input invariance pass.
|
||||||
- **Audio-local reframe:** local audio improves but all visual variants fail;
|
- **Audio-local reframe:** local audio improves but all visual variants fail;
|
||||||
claim a fine-grained acoustic readout bottleneck, not social visual reasoning.
|
claim a fine-grained acoustic readout bottleneck, not social visual reasoning.
|
||||||
- **Track-aware method:** correct continuity beats pooled vision, temporal
|
- **Track-aware method:** correct continuity beats pooled vision, temporal
|
||||||
|
|||||||
@@ -157,8 +157,9 @@ is not interpreted as a general limitation of LoRA or all Omni models.
|
|||||||
\todo{Freeze the exact Qwen2.5-Omni-3B revision, processor, input rates,
|
\todo{Freeze the exact Qwen2.5-Omni-3B revision, processor, input rates,
|
||||||
streaming/cache semantics, readout layer, context, dtype, LoRA targets/rank,
|
streaming/cache semantics, readout layer, context, dtype, LoRA targets/rank,
|
||||||
trainable parameter budget, and cutoff audit. Add Qwen2.5-Omni-7B only as a
|
trainable parameter budget, and cutoff audit. Add Qwen2.5-Omni-7B only as a
|
||||||
capacity-scaling control. Do not add another backbone family unless it is
|
capacity-scaling control. MiniCPM-o 4.5 is restricted to the diagnostic
|
||||||
needed to support and experimentally earns a cross-family claim.}
|
streaming/cache ablation below and does not enter the core method or support a
|
||||||
|
cross-family superiority claim.}
|
||||||
|
|
||||||
\subsection{Causal local floor adapter}
|
\subsection{Causal local floor adapter}
|
||||||
|
|
||||||
@@ -267,6 +268,17 @@ control. All learned variants use identical causal examples, prediction
|
|||||||
horizons, prediction heads, and the class-balanced binary cross-entropy
|
horizons, prediction heads, and the class-balanced binary cross-entropy
|
||||||
objective in Eq.~\eqref{eq:loss}.
|
objective in Eq.~\eqref{eq:loss}.
|
||||||
|
|
||||||
|
The diagnostic streaming ablation additionally uses MiniCPM-o 4.5
|
||||||
|
\cite{cui2026minicpmo45}. Its native cached stream is evaluated at its native
|
||||||
|
one-second update interval: a state becomes available to the 200-ms row whose
|
||||||
|
cutoff equals the chunk end and is then causally held until the next update. A
|
||||||
|
Qwen row subjected to the same 1-Hz update-and-hold schedule separates
|
||||||
|
refresh-rate effects from backbone effects. A MiniCPM-o
|
||||||
|
5-Hz cache row is admissible only if a cached branch at every cutoff agrees
|
||||||
|
with full causal-prefix recomputation within a frozen numerical tolerance and
|
||||||
|
passes future-input intervention tests. Thus the native 1-Hz condition is a
|
||||||
|
strictly causal held-state baseline, not a claim of native 5-Hz refresh.
|
||||||
|
|
||||||
Model selection is performed using training-side data only. Event operating
|
Model selection is performed using training-side data only. Event operating
|
||||||
points are selected from session-level out-of-fold predictions over
|
points are selected from session-level out-of-fold predictions over
|
||||||
Sessions~4--12; the final model is then retrained on all nine sessions before
|
Sessions~4--12; the final model is then retrained on all nine sessions before
|
||||||
@@ -392,27 +404,40 @@ numbers from the four-page version.}
|
|||||||
\subsection{Mechanism Analysis}
|
\subsection{Mechanism Analysis}
|
||||||
|
|
||||||
To determine whether the gain arises from correctly aligned visual floor
|
To determine whether the gain arises from correctly aligned visual floor
|
||||||
evidence rather than additional parameters or generic visual context, we
|
evidence rather than additional parameters or generic visual context, and to
|
||||||
evaluate the controls in Table~\ref{tab:ablation}.
|
separate streaming refresh from representation family, we evaluate the
|
||||||
|
controls in Table~\ref{tab:ablation}.
|
||||||
|
|
||||||
\begin{table}[t]
|
\begin{table}[t]
|
||||||
\caption{Mechanism controls for the selected Omni-3B local adapter. All
|
\caption{Mechanism and streaming/cache controls. The streaming panel uses
|
||||||
variants use the same prediction head and comparable temporal capacity; FPRs
|
frozen global states and capacity-matched downstream heads; all rows share
|
||||||
are measured at matched recall $R=0.8$.}
|
targets and evaluation. FPRs are measured at matched recall $R=0.8$.
|
||||||
|
Cross-family raw scores are diagnostic.}
|
||||||
\label{tab:ablation}
|
\label{tab:ablation}
|
||||||
\centering
|
\centering
|
||||||
\small
|
\small
|
||||||
\resizebox{\columnwidth}{!}{%
|
\resizebox{\columnwidth}{!}{%
|
||||||
\begin{tabular}{lcccc}
|
\begin{tabular}{lcccc}
|
||||||
\hline
|
\hline
|
||||||
Local condition & Mean mAP $\uparrow$ & EOT FPR $\downarrow$
|
Condition & Mean mAP $\uparrow$ & EOT FPR $\downarrow$
|
||||||
& O$\rightarrow$O FPR $\downarrow$ & INT FPR $\downarrow$\\
|
& O$\rightarrow$O FPR $\downarrow$ & INT FPR $\downarrow$\\
|
||||||
\hline
|
\hline
|
||||||
|
\multicolumn{5}{l}{\emph{Local-evidence controls (Qwen2.5-Omni-3B)}}\\
|
||||||
Remove vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
Remove vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
Time-shift vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
Time-shift vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
Full-frame vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
Full-frame vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
Correct local AV & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
Correct local AV & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
\hline
|
\hline
|
||||||
|
\multicolumn{5}{l}{\emph{Streaming/cache controls}}\\
|
||||||
|
Qwen, full causal prefix at 5 Hz (no cache)
|
||||||
|
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
|
Qwen, cached 5-Hz update & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
|
Qwen, cached 1-Hz + causal hold & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
|
MiniCPM-o, native cached 1-Hz + causal hold
|
||||||
|
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
|
MiniCPM-o, audited cached 5-Hz branch
|
||||||
|
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
||||||
|
\hline
|
||||||
\end{tabular}}
|
\end{tabular}}
|
||||||
\end{table}
|
\end{table}
|
||||||
|
|
||||||
@@ -430,7 +455,7 @@ evidence-routing claim. If dense mAP improves but event behavior does not, do
|
|||||||
not claim improved floor control. If legitimate onset or yield degrades, do
|
not claim improved floor control. If legitimate onset or yield degrades, do
|
||||||
not describe lower false entry as success.}
|
not describe lower false entry as success.}
|
||||||
|
|
||||||
\subsection{Scaling Control}
|
\subsection{Scaling and Streaming Controls}
|
||||||
|
|
||||||
Increasing the frozen Omni backbone from 3B to 7B yields
|
Increasing the frozen Omni backbone from 3B to 7B yields
|
||||||
\todo{$X$ points} in 0.2-s mAP and \todo{EOT/INT FPR of $Y/Z$} at matched
|
\todo{$X$ points} in 0.2-s mAP and \todo{EOT/INT FPR of $Y/Z$} at matched
|
||||||
@@ -443,6 +468,25 @@ provides a complementary benefit to backbone scaling}.
|
|||||||
Because both backbones belong to the Qwen2.5-Omni family, we do not claim that
|
Because both backbones belong to the Qwen2.5-Omni family, we do not claim that
|
||||||
this conclusion generalizes to all omni-modal models.
|
this conclusion generalizes to all omni-modal models.
|
||||||
|
|
||||||
|
The streaming rows answer two different questions. First, full causal-prefix
|
||||||
|
recomputation and streaming cache reuse at the same 5-Hz Qwen cutoffs test
|
||||||
|
whether cache/chunk semantics change the representation; equivalent outputs
|
||||||
|
make this an efficiency control rather than a distinct predictive method.
|
||||||
|
Second, the matched Qwen 5-Hz and 1-Hz cached rows test whether an advantage is
|
||||||
|
explained by more frequent causal state refresh rather than the proposed local
|
||||||
|
evidence route. For native 1-Hz streams, the state emitted at the end of each
|
||||||
|
one-second chunk is never backfilled into earlier 200-ms rows; source gaps and
|
||||||
|
session boundaries reset all modality and language-model caches. MiniCPM-o is
|
||||||
|
a secondary cross-family diagnostic under the same 1-Hz schedule, not a
|
||||||
|
same-frequency architecture ranking. Its 5-Hz cached-branch row is omitted if
|
||||||
|
cached/full-prefix equivalence, cutoff invariance, or reset audits fail.
|
||||||
|
|
||||||
|
\todo{Freeze native chunk boundaries, initialization of rows before the first
|
||||||
|
complete chunk, cache-reset rules, the cached/full-prefix tolerance, and the
|
||||||
|
future-media intervention test. Report within-backbone 5-Hz versus 1-Hz
|
||||||
|
deltas plus cache/no-cache throughput and memory; do not interpret unmatched
|
||||||
|
Qwen--MiniCPM raw scores as a pure backbone effect.}
|
||||||
|
|
||||||
\subsection{Limitations and Qualitative Analysis}
|
\subsection{Limitations and Qualitative Analysis}
|
||||||
|
|
||||||
The results support \todo{precise supported claim}, but several limitations
|
The results support \todo{precise supported claim}, but several limitations
|
||||||
|
|||||||
Reference in New Issue
Block a user