# EasyCom four-epoch loss-structure ablation

Recorded on **2026-08-26** from user-reported remote
`train_turn_head.py`/`infer_turn_head.py` results. This experiment tests
whether the three-column EgoSpeak target is better modeled as competing
softmax classes, independent binary marginals, or four mutually exclusive
speech-activity states.

## Experiment contract

- Dataset: EasyCom.
- Train split: Sessions 4--12, all used for training.
- Development split: none.
- Test split: official Sessions 3, 2, and 1.
- Backbone: frozen Qwen2.5-Omni-3B Thinker.
- Streaming representation: 8-second open-stream replay, 2 FPS video, final
  four query-layer states stacked at each 5 Hz target step.
- Readout: `concat-mlp`, followed by the anticipation MLP.
- Training budget: exactly 4 epochs for every objective.
- Reported command schedule: Adam, batch size 512, learning rate `7e-5`,
  weight decay `5e-5`, dropout `0.2`, warmup factor `0.3`, and 1.6 warmup
  epochs (the same 40% warmup fraction as the earlier 10/25 protocol).
- Seeds: 0, 1, and 2.
- Evaluation: `causal_valid_only` AP/mAP over ten future horizons from 0.2 to
  2.0 seconds.

The objectives are:

1. `lstr-mce`: three softmax logits with the original multi-positive MCE.
2. `independent-bce`: three independent sigmoid/BCE marginal labels.
3. `four-state-ce`: softmax over silence, wearer-only, others-only, and
   overlap; evaluation marginalizes back to the original three labels.

## Three-seed summary

All values are percentage AP points, reported as mean +/- sample standard
deviation.

| Objective | 0.2s mAP | 0.2s wearer AP | 0.2s others AP | 10-horizon mean mAP |
|---|---:|---:|---:|---:|
| LSTR-MCE | 73.70581 +/- 0.23802 | 78.41206 +/- 0.48622 | 85.74830 +/- 0.03661 | 60.81597 +/- 0.06941 |
| Independent BCE | 73.70702 +/- 0.08593 | 78.58842 +/- 0.16884 | 86.01169 +/- 0.09991 | **60.91733 +/- 0.03540** |
| Four-state CE | **73.81030 +/- 0.05499** | **78.72040 +/- 0.14753** | **86.03151 +/- 0.06097** | 60.88488 +/- 0.05728 |

Relative to LSTR-MCE, four-state CE changes 0.2-second mAP by only
`+0.10449`, wearer AP by `+0.30834`, others AP by `+0.28321`, and mean mAP by
`+0.06891`. Independent BCE changes the same metrics by `+0.00121`,
`+0.17636`, `+0.26339`, and `+0.10136`, respectively.

## Individual runs

| Run | 0.2s mAP | Background AP | Wearer AP | Others AP | Mean mAP |
|---|---:|---:|---:|---:|---:|
| four-state-ce seed 0 | 73.74955 | 56.71375 | 78.57366 | 85.96124 | 60.83261 |
| four-state-ce seed 1 | 73.85667 | 56.63089 | 78.86871 | 86.07041 | 60.94612 |
| four-state-ce seed 2 | 73.82467 | 56.69228 | 78.71884 | 86.06288 | 60.87591 |
| independent-bce seed 0 | 73.62153 | 56.53017 | 78.40881 | 85.92561 | 60.87646 |
| independent-bce seed 1 | 73.79339 | 56.64809 | 78.74388 | 85.98820 | 60.93730 |
| independent-bce seed 2 | 73.70613 | 56.38458 | 78.61257 | 86.12125 | 60.93824 |
| lstr-mce seed 0 | 73.44637 | 56.76438 | 77.86650 | 85.70824 | 60.84861 |
| lstr-mce seed 1 | 73.75696 | 56.94418 | 78.57004 | 85.75664 | 60.73626 |
| lstr-mce seed 2 | 73.91410 | 57.16263 | 78.79964 | 85.78001 | 60.86305 |

## Seed-0 horizon comparison

| Horizon | LSTR-MCE mAP | Independent BCE mAP | Four-state CE mAP |
|---:|---:|---:|---:|
| 0.2s | 73.44637 | 73.62153 | 73.74955 |
| 0.4s | 68.63790 | 68.93990 | 68.82697 |
| 0.6s | 64.64750 | 64.79984 | 64.80523 |
| 0.8s | 61.50690 | 61.57880 | 61.52546 |
| 1.0s | 59.35000 | 59.53538 | 59.44807 |
| 1.2s | 58.08335 | 57.95764 | 57.96347 |
| 1.4s | 57.22382 | 56.94317 | 56.95413 |
| 1.6s | 56.15409 | 56.04943 | 55.84352 |
| 1.8s | 55.11616 | 55.09841 | 54.89116 |
| 2.0s | 54.32006 | 54.24044 | 54.31858 |
| **Mean** | **60.84861** | **60.87646** | **60.83261** |

The complete seed-0 class AP curves are stored in [`metrics.csv`](metrics.csv)
and [`summary.json`](summary.json).

## Interpretation

This is not a clear aggregate improvement. The two non-competing objectives
slightly improve wearer/others ranking, but generally trade away background
ranking. Four-state CE is best for the pre-specified short-horizon wearer
metric, while independent BCE is marginally best for ten-horizon mean mAP.
The differences are small relative to seed variation, and four-state CE does
not establish a decisive advantage over the simpler BCE formulation.

The defensible paper statement is therefore:

> Non-competing label formulations provide small speech-class gains under a
> fixed four-epoch budget, but no loss formulation significantly dominates
> aggregate anticipation mAP.

Before interpreting this as an overlap-modeling result, evaluate overlap AP
and per-session paired differences from the saved predictions. Because no dev
set was used, these results describe a fixed-compute ablation rather than each
objective's independently optimized performance. Do not select a primary loss
post hoc from these test results.

## Provenance

- Feature root:
  `/data1/yfzx/wangx/Datasets/easycom/omni_features/open_stream_2fps_8s_last4_stack`
- Target root:
  `/data1/yfzx/wangx/Datasets/easycom/easycom_target_frame`
- Remote checkpoint root: `checkpoints/easycom_loss_structure_4ep`
- Remote result root: `results/easycom_loss_structure_4ep`
- Full per-horizon output supplied for seed 0 only.
- Three-seed aggregate and individual-run summaries supplied for all losses.
- Raw remote checkpoint JSON and `metrics.json` files archived locally: no.

Machine-readable records: [`summary.json`](summary.json),
[`metrics.csv`](metrics.csv), and [`raw_console.txt`](raw_console.txt).