# EasyCom: open-stream replay versus causal-prefix

Recorded on **2026-08-19**. These are the current single-run results reported
from the remote EasyCom experiments. They are an experiment snapshot, not yet
the final multi-seed paper numbers.

## Experiment contract

- Dataset: EasyCom using EgoSpeak's official split.
- Train sessions: 7, 11, 10, 6, 9, 4, 8, 5, and 12.
- Test sessions: 3, 2, and 1.
- Model: frozen Qwen2.5-Omni-3B Thinker plus the trainable `10 x 3` head.
- Prediction rate: 5 Hz, one state every 0.2 seconds.
- Video sampling: 2 FPS.
- Causal context budget: 20 seconds.
- Targets: ten future horizons from 0.2 through 2.0 seconds.
- Metric: frame AP for all three classes, followed by per-horizon mAP and the
  mean over all ten horizons.

The two representations are trained and evaluated separately. A head trained
on one representation must not be reused on the other.

## Summary

All values below are percentages.

| Representation | EgoSpeak full-timeline mean mAP | Strict causal mean mAP | Difference from open stream |
|---|---:|---:|---:|
| Open-stream replay | 58.316 | 58.326 | — |
| Causal-prefix | 58.830 | 58.841 | +0.514 / +0.515 |

For context, the EgoSpeak Table 2 EasyCom reference values currently recorded
in the project README are 58.7 for Transformer A+V and 60.6 for GRU A+V.
Accordingly, causal-prefix is 0.130 points above the Transformer reference,
whereas open-stream replay is 0.384 points below it. Neither reaches the GRU
A+V result.

## Full-timeline result by horizon

| Horizon | Time | Open-stream mAP | Causal-prefix mAP | Causal minus open |
|---:|---:|---:|---:|---:|
| 1 | 0.2 s | 69.756 | 72.149 | +2.393 |
| 2 | 0.4 s | 65.163 | 66.904 | +1.741 |
| 3 | 0.6 s | 61.578 | 62.468 | +0.890 |
| 4 | 0.8 s | 58.783 | 59.214 | +0.431 |
| 5 | 1.0 s | 56.913 | 57.037 | +0.124 |
| 6 | 1.2 s | 55.727 | 55.789 | +0.062 |
| 7 | 1.4 s | 54.819 | 54.845 | +0.026 |
| 8 | 1.6 s | 53.957 | 53.905 | -0.052 |
| 9 | 1.8 s | 53.459 | 53.265 | -0.194 |
| 10 | 2.0 s | 53.002 | 52.723 | -0.279 |

The complete per-class AP values for both evaluation protocols are stored in
[`metrics.csv`](metrics.csv). [`summary.json`](summary.json) contains the
machine-readable aggregate comparison.

## Current interpretation

1. Causal-prefix is 0.514 mAP points better overall in the exact EgoSpeak
   full-timeline protocol.
2. Almost all of that advantage occurs at 0.2--0.6 seconds. At 1.2--1.4
   seconds the difference is negligible, and open-stream is slightly better
   at 1.6--2.0 seconds.
3. The full-timeline and strict-causal means differ by only 0.010--0.011
   points, so unavailable pre-session rows are not driving the comparison.
4. The short-horizon pattern is consistent with block-local audio processing,
   partial-block AV/TMRoPE organization, or other 2-second boundary effects
   weakening precise speech-onset/offset cues. It does not look like a general
   loss of longer-horizon conversational information.
5. Previous smoke tests showed exact open-stream cache replay but a lower
   cosine against the continuous-window representation. This supports a real
   representation-semantics difference rather than a cache-copy error.

## Required follow-up before publication

- Repeat head training with at least three identical seeds for both modes and
  report mean and standard deviation.
- Group feature cosine and AP/mAP by the ten 0.2-second phases inside each
  2-second block to test for a block-boundary effect.
- Compare samples before and after the 20-second memory budget fills to isolate
  input segmentation from rolling-memory behavior.
- Archive the original remote `metrics.json`, checkpoint configuration, git
  commit, Transformers version, and Qwen model revision alongside the final
  paper results. The numbers in this snapshot were transcribed from the
  reported console tables.