icassp2027-experiment/docs/qwen_tail_chunk_ablation.md

Qwen partial-tail and chunk-size ablation

This runbook freezes the three-way comparison used to test whether causal 5 Hz adaptation needs an explicit partial tail. All modes use the same frozen Qwen2.5-Omni backbone, right-edged 5 Hz output grid, 2 FPS causal video grid, 8-second memory budget, downstream head, split, and optimization.

Mode	New media at output time t	5 Hz freshness	Interpretation
open-stream-replay	completed native 2 s blocks plus a temporary growing tail of 0.2--2.0 s	yes	proposed partial-tail representation
completed-block-hold	only completed native 2 s blocks; discard the incomplete tail and hold the last state	no	native-chunk stale control
micro-chunk-200ms	commit every new 0.2 s interval as an independent payload	yes	non-native micro-chunk stress control

The 2-second hold mode does not backfill future information. Its first nine rows (0.2--1.8 s) are zero initialization because no native block has yet completed; the first model state appears at 2.0 s. Saved initialization_mask, state_source_times, state_update_mask, payload_commit_mask, and native_boundary_mask make this auditable. Report a sensitivity analysis excluding initialization rows if they have a non-negligible effect.

The micro-chunk mode is deliberately not described as native Qwen streaming. It uses the model's physical-time TMRoPE rate to allocate five positions per 0.2-second step and refuses to run if ten micro steps do not tile one native 2-second/50-position chunk exactly. It tests whether freshness alone is sufficient when the model's native chunk construction is removed.

Do not infer compute cost only from chunk duration. For a 2-second source interval, partial-tail extraction prepares 0.2+...+2.0 = 11 seconds of media, the hold control prepares 2 seconds in one call, and the micro control prepares 2 seconds across ten calls. Wall time, peak memory, processor calls, and saved production_prepared_media_seconds should all be reported.

Required short GPU smoke tests

Run each mode on the same four seconds before launching complete sessions:

cd /data1/yfzx/wangx/Code/icassp2027-experiment

MODE=open-stream-replay \
SESSION_ID=4 GPU=0 END=4 SHORT_MEMORY_SECONDS=8 \
FEATURE_ROOT=features/qwen_chunk_ablation_smoke \
LOG_ROOT=logs/qwen_chunk_ablation_smoke \
bash run_smoke_easycom_direct.sh

MODE=completed-block-hold \
SESSION_ID=4 GPU=0 END=4 SHORT_MEMORY_SECONDS=8 \
FEATURE_ROOT=features/qwen_chunk_ablation_smoke \
LOG_ROOT=logs/qwen_chunk_ablation_smoke \
bash run_smoke_easycom_direct.sh

MODE=micro-chunk-200ms \
SESSION_ID=4 GPU=0 END=4 SHORT_MEMORY_SECONDS=8 \
FEATURE_ROOT=features/qwen_chunk_ablation_smoke \
LOG_ROOT=logs/qwen_chunk_ablation_smoke \
bash run_smoke_easycom_direct.sh

The expected feature shape for every mode is (20, D). Inspect the schedule and processing accounting:

python - <<'PY'
from pathlib import Path
import torch

root = Path("features/qwen_chunk_ablation_smoke")
for mode in (
    "open-stream-replay",
    "completed-block-hold",
    "micro-chunk-200ms",
):
    path = root / f"Session_4_{mode}_gpu0.pt"
    payload = torch.load(path, map_location="cpu", weights_only=False)
    diagnostics = payload["extraction_diagnostics"]
    print("\n", mode, tuple(payload["states"].shape))
    print("processor calls:", diagnostics["production_processor_calls"])
    print(
        "prepared media seconds:",
        diagnostics["production_prepared_media_seconds"],
    )
    print("source times:", diagnostics["state_source_times"].tolist())
    print("initialization rows:", int(diagnostics["initialization_mask"].sum()))
    print("state updates:", diagnostics["state_update_mask"].nonzero().flatten().tolist())
    print("payload commits:", diagnostics["payload_commit_mask"].nonzero().flatten().tolist())
    print("native boundaries:", diagnostics["native_boundary_mask"].nonzero().flatten().tolist())
    print("verification:", payload["verification"])
PY

For four seconds, expected production accounting is:

Mode	Processor calls	Prepared media seconds	State update rows	Payload commit rows
open-stream-replay	20	22.0	every row	9, 19
completed-block-hold	2	4.0	9, 19	9, 19
micro-chunk-200ms	20	4.0	every row	every row

All three modes should report native 2-second boundaries at rows 9 and 19; that timing fact is distinct from whether a mode updates or commits there.

cache_replay checks should be numerically exact or effectively exact. micro_chunk_vs_continuous_semantics is a diagnostic comparison, not an equivalence requirement: the micro representation intentionally changes input segmentation.

Extract all EasyCom sessions

The stable multi-session launcher already forwards MODE. Run the controls separately so their outputs and logs cannot overwrite each other. Adjust GPU IDs and roots to the remote host if needed.

cd /data1/yfzx/wangx/Code/icassp2027-experiment

PROJECT_ROOT=$PWD \
MODEL=/data1/yfzx/wangx/Models/Qwen2.5-Omni-3B \
VIDEO_ROOT=/data1/yfzx/wangx/Datasets/easycom/extracted_easycom/Main/Video_Compressed \
TARGET_ROOT=/data1/yfzx/wangx/Datasets/easycom/easycom_target_frame \
FEATURE_ROOT=/data1/yfzx/wangx/Datasets/easycom/omni_features/qwen_completed_block_hold_2fps_8s \
LOG_ROOT=$PWD/logs/qwen_completed_block_hold_2fps_8s \
MODE=completed-block-hold MEMORY_SECONDS=8 GPUS="0 1 2 3" \
bash run_extract_easycom_direct.sh

PROJECT_ROOT=$PWD \
MODEL=/data1/yfzx/wangx/Models/Qwen2.5-Omni-3B \
VIDEO_ROOT=/data1/yfzx/wangx/Datasets/easycom/extracted_easycom/Main/Video_Compressed \
TARGET_ROOT=/data1/yfzx/wangx/Datasets/easycom/easycom_target_frame \
FEATURE_ROOT=/data1/yfzx/wangx/Datasets/easycom/omni_features/qwen_micro_chunk_200ms_2fps_8s \
LOG_ROOT=$PWD/logs/qwen_micro_chunk_200ms_2fps_8s \
MODE=micro-chunk-200ms MEMORY_SECONDS=8 GPUS="0 1 2 3" \
bash run_extract_easycom_direct.sh

Each output is mode-tagged. Existing training code therefore fails closed if the wrong directory is paired with --expected-mode.

Train matched frozen heads

Use the same seeds and training hyperparameters as the partial-tail baseline. The commands below show one matched run per control:

python train_turn_head.py \
  --feature-root /data1/yfzx/wangx/Datasets/easycom/omni_features/qwen_completed_block_hold_2fps_8s \
  --target-root /data1/yfzx/wangx/Datasets/easycom/easycom_target_frame \
  --output checkpoints/easycom_qwen_completed_block_hold_8s.pt \
  --expected-mode completed-block-hold \
  --epochs 50 --batch-size 512 --device cuda:0

python train_turn_head.py \
  --feature-root /data1/yfzx/wangx/Datasets/easycom/omni_features/qwen_micro_chunk_200ms_2fps_8s \
  --target-root /data1/yfzx/wangx/Datasets/easycom/easycom_target_frame \
  --output checkpoints/easycom_qwen_micro_chunk_200ms_8s.pt \
  --expected-mode micro-chunk-200ms \
  --epochs 50 --batch-size 512 --device cuda:0

This ablation is interpretable only when the partial-tail baseline is re-extracted or otherwise confirmed to use the same 8-second memory, decoder, 2 FPS sampling, readout layers, split, seeds, and head training recipe.