Refine targeted methods and freeze section structure

AGENTS.md

@@ -62,6 +62,30 @@ EasyCom 遵循官方 Sessions 4--12/1--3 划分；十个 0.2--2.0 秒 horizon
 raw onset/offset、局部音视频和 oracle/时序消融应区分实测、历史非匹配 screen、
 仅用于分析的 privileged oracle 与待测方案。raw endpoint AP 不等同语义 EOT/INT。
 
+## Targeted manuscript 的 Method 固定结构
+
+Method 按以下三个小节展开，除非用户明确要求，不再改变这个组织顺序；具体特征和
+训练状态以代码、结果记录与 [`manuscript/targeted/main.tex`](manuscript/targeted/main.tex)
+为准。
+
+1. **Task and Global--Local Fusion。** 定义 5 Hz 因果截止点、十个未来 horizon 和
+   background/wearer/others 输出；说明 Thinker 全局状态本身已含音视频，再定义 SWE
+   的近期音频及可选视觉旁路。区分 pooled 特征的拼接聚合与 F2/F3 保留视觉 token 的
+   query attention。训练范围分开写：主方案只训练 readout/局部分支/预测头；Thinker
+   LoRA、限制在上层的 LoRA、预测头冻结或联合训练是参数适配控制。最后四层
+   **readout** 与 LoRA 更新哪些层是两个独立维度。已归档 LoRA 结果为非匹配探索，
+   不能写成层数消融或优劣定论。
+2. **Causal Partial-Tail Refresh。** 完成块进入持久历史；每个 200 ms 截止点只临时查询
+   当前 2 秒块已观测的前缀，尾块不提交、不中途引入未来媒体。与 completed-only
+   在同一截止点和上下文上限下配对比较；未完成实验前只称因果访问规则。
+3. **Short-Window Feature Extraction。** 默认局部音频为截止点前一秒混合波形经冻结
+   Qwen 音频编码器；EgoSpeak 式 wav2vec2 卷积编码是待做的同窗口、同预测头对照，
+   不能冒充已测 G+A。视觉区分最近单帧、最近一秒多帧人物框 pooled 特征、无框的
+   F2 最新全画面空间 token，以及 F3 同网格位置两帧 token 经共享 GRU 后注意力聚合。
+   另列一种特殊的短窗信息审计：由转写标注生成、仅含截止点之前的可见非 wearer
+   活动历史，比较逐人、打乱、延迟和 OR 聚合等形式。它不是部署时可获得的视觉特征，
+   也不是性能上界。F3 不是人物跟踪，F3--F2 差异也不能单独归因于运动理解。
+
 ## 不可破坏的研究契约
 
 - 严格因果：任何输入音频、视频帧、人物框或归一化统计不得来自预测截止时间之后。

manuscript/targeted/main.tex

@@ -143,7 +143,7 @@ committed to persistent history.}
 \section{Method}
 \label{sec:method}
 
-\subsection{Task and Short-Window Encoder Evidence}
+\subsection{Task and Global--Local Fusion}
 
 At the 5-Hz cutoff $t_n=0.2n$~s, the model observes only media available by
 $t_n$ and predicts activity at $t_n+h$, where
@@ -152,32 +152,48 @@ horizon has three channels in fixed order: background, camera wearer, and
 aggregate others.  Wearer and others may overlap; no output identifies an
 individual speaker, addressee, or semantic end of turn.
 
-The global feature $\vect g_n$ concatenates the final four frozen
+The global representation $\vect g_n$ concatenates the final four frozen
 Qwen2.5-Omni-3B Thinker readouts and projects them to 2048 dimensions. It
-already contains audio and video context; $\G$ is not audio-free.  SWE
+already integrates audio and video over the available context; $\G$ is not
-provides a separate short-window path so the head need not recover all
+an audio-free baseline. Short-window encoder evidence (SWE) adds a direct,
-cutoff-local detail from the integrated state. Local audio $\vect a_n$ crops
+cutoff-aligned path for recent audio $\vect a_n$ and, when enabled, visual
-the latest one-second mixed
+evidence $\vect v_n$. This separates the question of what the global state
-waveform \emph{before} preprocessing, runs the frozen Qwen audio encoder, and
+encodes from whether a small prediction head can access transient cues.
-averages valid tokens.  Optional $\vect v_n$ averages frozen vision tokens
-inside provided person boxes, then across people and valid causal 2-FPS
-frames in the latest second. This pooling has no identity or mouth-motion trajectory;
-provided boxes do not imply an online detector.
 
-Each enabled local feature is layer-normalized, projected to 256 dimensions,
+In the pooled fusion, each available local vector is layer-normalized,
-and paired with a validity bit.  The trainable head is
+projected to 256 dimensions, and accompanied by a validity bit. The
+concatenated global and local features are passed to a layer-normalized MLP:
 \begin{equation}
- \widehat{\vect Y}_{n}=
+ \widehat{\vect Y}_{n,h,c}=\operatorname{softmax}_{c}
- \operatorname{softmax}_{c}F_{\theta}
+ F_{\theta}\!\left([\vect g_n;\widetilde{\vect a}_n;m_n^a;
- \bigl[\vect g_n;\widetilde{\vect a}_n;m_n^a;
+       \widetilde{\vect v}_n;m_n^v]\right)_{h,c},
-       \widetilde{\vect v}_n;m_n^v\bigr],
  \label{eq:fusion}
 \end{equation}
-where omitted modalities remove their terms and $F_\theta$ is a
+where a disabled branch is omitted and $F_\theta$ produces ten sets of
-layer-normalized MLP with a 30-logit output.  We train only the projections and
+three logits. For token-preserving visual variants, $\vect v_n$ denotes
-head with the EgoSpeak multi-positive cross-entropy over causal-valid
+an attention-pooled context: a query formed from $\vect g_n$ and valid
-source--horizon pairs.  The Omni and modality encoders remain frozen, so this
+$\vect a_n$ attends to the causal visual token set (and a learned null
-is frozen-backbone adaptation rather than end-to-end training.
+token), rather than averaging it beforehand. The resulting context and
+validity bit enter the same prediction head. We train the projections,
+visual aggregator when present, and head with the EgoSpeak multi-positive
+cross-entropy over causal-valid source--horizon pairs. The principal SWE
+configuration freezes all feature encoders and the Thinker: this is
+frozen-backbone adaptation, not task-level end-to-end training.
+
+We separate \emph{where} parameters are updated from \emph{which} Thinker
+layers are read. The primary comparison trains only the layer-combination
+projection, local-fusion modules, and 30-logit prediction head on cached
+frozen features. Parameter-adaptation controls instead attach low-rank
+adapters to Thinker attention projections, either across the configured
+Thinker or restricted to its upper layers; the prediction head may be
+updated jointly or held fixed to isolate representation changes. These
+controls require the same causal media, source cutoffs, supervision,
+initialization, and model-selection protocol as the frozen-head condition.
+Archived LoRA runs, including an upper-eight-layer setting, are
+exploratory and not a matched layer-scope ablation. Last-four-layer
+\emph{readout} fusion changes the head input, not the LoRA update scope;
+neither its gain nor an unmatched LoRA score establishes an adaptation
+advantage.
 
 \subsection{Causal Partial-Tail Refresh}
 
@@ -193,18 +209,42 @@ completed-only versus PTR readouts with $\A$ on/off at an identical 8-s cap;
 the separate 2--12-s cap sweep tests context length. Those comparisons are
 pending, so PTR is a causal access rule rather than a demonstrated gain.
 
-\subsection{Visible-Activity Information Audit}
+\subsection{Short-Window Feature Extraction}
 
-To distinguish missing information from ineffective learned vision, we build
+We define every local observation by its prediction cutoff, before applying
-analysis-only features from transcript-derived \emph{past} activity.  At each
+any encoder-specific preprocessing. The default audio branch encodes the
-cutoff, visible non-wearer participants receive two seconds of binary history
+mixed waveform in $[t_n-1\,\mathrm{s},t_n)$ with the frozen Qwen2.5-Omni
-sampled on the same 5-Hz grid.  The preparation fails if its aggregate
+audio encoder and averages valid output tokens. A wav2vec~2.0 convolutional
-activity disagrees with the released target.  We compare: (i) visibility with
+audio representation, as used by EgoSpeak \cite{kim2025egospeak}, is an
-activity zeroed, (ii) per-person histories, (iii) identities shuffled while
+encoder-choice control; it must use the same one-second input interval and
-preserving active-person counts at every lag, (iv) histories delayed by one
+forecasting head before differences can be attributed to the encoder. It is
-second, and (v) one OR-pooled aggregate-visible history.  These conditions
+not the source of the reported $\G+\A$ result.
-never expose future activity, but their speech labels make them privileged
+
-diagnostics, not deployable features or upper bounds.
+For vision, a single-frame variant uses the latest frame strictly before
+$t_n$; the pooled person-crop variant averages frozen Qwen vision tokens
+within provided boxes, across visible people, and over valid 2-FPS frames
+in the preceding second. These pooled features summarize appearance but
+discard person identity and explicit motion. Boxes are dataset-provided,
+not predictions from an evaluated online detector. Two no-box full-frame
+alternatives retain the post-merger spatial-token grid: F2 lets the
+global--audio query attend to the latest causal frame; F3 first encodes up
+to two observations at each matching grid location in chronological order
+with a shared GRU, then applies the same query to the resulting grid. F3
+links image coordinates, not people or tracks. Its difference from F2
+confounds additional history and temporal capacity, so it cannot alone
+establish a motion or speaker-grounding mechanism. Empty or unavailable
+local evidence is indicated by the validity masks, never filled with a
+future observation.
+
+As a distinct, privileged short-window feature, we also derive visible
+non-wearer speech activity from transcript annotations \emph{before} each
+cutoff. Each visible participant contributes a two-second binary history
+on the 5-Hz grid. We test visibility-only, per-person, identity-shuffled,
+one-second-delayed, and OR-pooled aggregate histories to ask which timely
+activity information a learned visual branch would need to recover.
+Although strictly past-only, these annotations are not available from raw
+media at inference; this is an information audit, not a deployable feature
+or a performance upper bound.
 
 \input{tables_compact}
 % Historical full-precision table layouts retained as source audit only.