icassp2027-experiment/results/easycom_transition_auxiliary_4ep_2026-08-27/README.md

EasyCom transition-aware auxiliary-head ablation

Recorded on 2026-08-27 from user-reported remote evaluation output. This is a preliminary two-seed snapshot: seed 2 has not been reported, so these numbers must not yet be used as a final multi-seed paper result.

Experiment contract

• Dataset: EasyCom.
• Train split: Sessions 4 through 12, with no development set.
• Test split: EgoSpeak official Sessions 3, 2, and 1.
• Backbone: frozen Qwen2.5-Omni-3B Thinker.
• Representation: 8-second open-stream replay, 2 FPS video, 5 Hz states, final four query layers stacked.
• Readout: concat-MLP followed by the activity MLP.
• Activity objective: LSTR-MCE.
• Training budget: 4 epochs, batch size 512, learning rate 7e-5, weight decay 5e-5, dropout 0.2, 1.6 warmup epochs, warmup factor 0.3.
• Completed seeds: 0 and 1. Missing seed: 2.
• Primary evaluation protocol: causal-valid-only activity AP/mAP.

A positive transition at future horizon h means that the complete three-label activity state changes between target rows t+h-1 and t+h.

• R1: activity loss + 0.2 times weighted transition BCE, with positive weight 3.
• R2: R1 loss - 0.1 times exact expected weighted transition correctness.

Individual results

All values below are percentage points.

Variant	Seed	0.2s mAP	BG AP	Wearer AP	Others AP	Mean mAP	Transition 0.2s AP	Transition 0.2s F1@0.5	Transition mean AP
R1	0	73.51590	56.84019	77.92448	85.78303	60.86360	15.19153	0.46172	13.04354
R1	1	73.78692	56.92102	78.63090	85.80884	60.82023	15.32492	1.14068	13.06174
R2	0	73.53288	56.89701	77.92732	85.77432	60.86528	15.37608	0.00000	13.11854
R2	1	73.82947	56.97474	78.69340	85.82027	60.84178	15.47356	0.84227	13.13820

Two-seed mean and sample standard deviation

Variant	0.2s mAP	Wearer AP	Others AP	Mean mAP	Transition 0.2s AP	Transition mean AP
R1	73.65141 +/- 0.19164	78.27769 +/- 0.49951	85.79594 +/- 0.01825	60.84192 +/- 0.03067	15.25823 +/- 0.09432	13.05264 +/- 0.01287
R2	73.68118 +/- 0.20972	78.31036 +/- 0.54169	85.79729 +/- 0.03249	60.85353 +/- 0.01662	15.42482 +/- 0.06893	13.12837 +/- 0.01390

Matched-seed comparison with LSTR-MCE

The baseline values are taken from the existing four-epoch loss-structure ablation. Using only the same completed seeds 0 and 1 avoids comparing a two-seed transition result with a three-seed baseline mean.

Comparison	Delta 0.2s mAP	Delta BG AP	Delta wearer AP	Delta others AP	Delta mean mAP
R1 minus LSTR-MCE	+0.04975	+0.02633	+0.05942	+0.06350	+0.04949
R2 minus LSTR-MCE	+0.07951	+0.08160	+0.09209	+0.06486	+0.06110
R2 minus R1	+0.02977	+0.05527	+0.03267	+0.00136	+0.01162

All activity changes are much smaller than the observed seed variation. The current evidence therefore does not establish a meaningful activity anticipation improvement.

Transition diagnostic

The 0.2-second transition prevalence is 11.21718%, which is also the expected AP of a random ranking. R1 reaches 15.19--15.32% AP, or 1.35--1.37 times the random baseline. R2 reaches 15.38--15.47%, or 1.37--1.38 times random. The auxiliary head therefore learns weak but non-random transition ranking.

F1 at threshold 0.5 is nearly zero because recall is only 0--0.58%. This fixed threshold is not a reliable model-quality measure under weighted BCE: positive weight 3 changes probability calibration and remains below the approximately 7.9 negative-to-positive ratio implied by the test prevalence. Threshold-free AP is the appropriate diagnostic here. Any deployable threshold must be selected on training-held-out data, not the test set.

R2 improves transition AP over R1 by only 0.16660 points at 0.2 seconds and 0.07573 points on the ten-horizon mean. Its activity mean-mAP improvement over R1 is only 0.01162 points. Exact expected correctness supplies almost the same gradient direction as weighted BCE, so this result is consistent with the two terms being largely redundant rather than R2 learning a new sequential policy.

Current interpretation

This is a preliminary null ablation:

Exact 200-ms transition supervision provides weak non-random boundary ranking but no meaningful improvement in the primary activity-anticipation metric. Adding exact expected correctness produces negligible further gain.

This experiment should not be described as interactive reinforcement learning. R2 computes the binary-action expectation analytically on fixed offline labels; it is a low-variance supervised reward control.

Required follow-up

1. Complete and report seed 2 before freezing the table.
2. Do not select a transition threshold or tune positive/reward weights on the test sessions.
3. If the transition line is continued, replace the sparse exact-boundary target with a pre-specified cumulative transition-hazard target: whether at least one transition occurs within the next h steps.
4. If that structured target also fails to improve activity mAP, treat transition auxiliary learning on frozen states as a rejected direction and move experiment budget to representation or temporal modeling.

Provenance

• Feature root: /data1/yfzx/wangx/Datasets/easycom/omni_features/open_stream_2fps_8s_last4_stack
• Target root: /data1/yfzx/wangx/Datasets/easycom/easycom_target_frame
• Remote result roots: results/easycom-transition-r1-4ep-seed-N and results/easycom-transition-r2-4ep-seed-N
• Source: user-reported summary and diagnostic console output.
• Raw remote metrics JSON archived locally: no.
• Full per-horizon activity and transition curves archived locally: no.

Machine-readable records are in summary.json and metrics.csv. The received console values are preserved in raw_console.txt.