icassp2027-experiment/results/easycom_omni_floor_screen_2026-09-16/README.md

# EasyCom Omni floor-event screening

This record freezes the first comparison of EgoSpeak, Omni `G+A`, Omni
`G+A+pooled-V`, and the three-seed current-frame F1 Frame-set Attention screen
on EasyCom Sessions 1--3. Values were produced on the remote experiment host
and supplied by the user on 2026-09-16. Generated predictions and per-event
CSVs remain on that host and are not committed here.

## Evaluated layers

1. Dense future activity uses causal-valid framewise AP/mAP over ten horizons.
2. Raw wearer endpoint and aggregate-other-boundary AP are threshold-free
   timing diagnostics; they are not semantic EOT/INT labels.
3. Opportunity-conditioned ENTRY and INT/YIELD AP assigns one causally
   censored activity-derived score to each provisional LLM-assisted reviewed
   opportunity.

All opportunity runs use identical denominators: 163 ENTRY positives and 335
hard negatives, with 81 invalid/post-outcome cases; and 28 INT positives and
109 hard negatives, with 5 invalid/post-outcome cases. Positive prevalence is
32.731% for ENTRY and 20.438% for INT.

## Main results

| Model | 0.2-s mAP | Mean mAP | Boundary response AP | Boundary yield AP | ENTRY AP | INT/YIELD AP |
|---|---:|---:|---:|---:|---:|---:|
| EgoSpeak | 74.204 | 61.193 | 29.989 | 41.078 | 52.487 | 20.135 |
| Omni G+A, seed 0 | 81.723 | 65.109 | 28.840 | 47.873 | 43.886 | 21.535 |
| Omni G+A+pooled-V, seed 0 | 82.027 | 65.400 | 29.601 | 48.344 | 46.022 | 24.266 |
| F1, three-seed mean | 81.777 | 64.951 | 28.423 | 48.641 | 41.561 | 22.991 |

F1 standard deviations are 0.156 points for 0.2-s mAP, 0.095 for mean mAP,
0.480 for boundary response AP, 0.391 for boundary yield AP, 0.581 for ENTRY
AP, and 0.202 for INT/YIELD AP.

The raw endpoint diagnostic, available for the three seed-0 baselines, changes
from EgoSpeak to G+A to G+A+pooled-V as follows: onset AP
24.491/25.960/26.019 and offset AP 38.024/45.251/45.371. Thus most generic
offset/yield-timing gain comes from G+A; pooled visual input adds almost
nothing to raw transition timing.

## Interpretation

- G+A strongly improves dense forecasting and generic wearer-offset/yield
  timing over EgoSpeak, but not reviewed semantic opportunities. In particular,
  its INT AP remains close to prevalence and its ENTRY AP is lower than
  EgoSpeak's.
- Adding pooled-V over G+A gives only +0.304/+0.291 points in 0.2-s/mean mAP,
  +0.059/+0.120 in raw onset/offset AP, and +0.761/+0.471 in boundary
  response/yield AP. The larger but still preliminary changes are +2.135 ENTRY
  AP and +2.732 INT AP.
- F1 is stable across seeds but does not improve on the historical pooled-V
  run: its mean changes are -0.250/-0.449 points in 0.2-s/mean mAP, -1.178 in
  boundary response AP, +0.297 in boundary yield AP, -4.461 in ENTRY AP, and
  -1.275 in INT AP. The hoped-for pattern of unchanged dense/timing performance
  with improved semantic opportunity ranking is absent.
- Current F1 attends to a current anonymous visual person set using G+A. It has
  no per-person temporal history, person-conditioned audio, or verified
  identity continuity. This screen therefore does not support promoting F1 as
  the paper's main floor-reasoning method.

## Hard limitations

- The opportunity labels are provisional two-pass LLM-assisted annotations,
  not human-reviewed confirmatory labels.
- The old G+A and pooled-V runs are seed-0 fixed-four-epoch models trained on
  Sessions 4--12 without development selection. F1 trains on Sessions 4--11
  and uses Session 12 for checkpoint selection. Their comparison is historical,
  not a protocol-matched attribution.
- F1 needs a three-seed pooled-V control trained and selected with exactly the
  same train/dev protocol before a confirmatory conclusion.
- All matched-recall trigger/FPR values produced in these exploratory commands
  selected thresholds on Sessions 1--3 and are excluded from this record.
- Aggregate `others` activity cannot establish speaker identity, addressee, or
  semantic interruption intent.

Exact aggregate values are stored in [`summary.json`](summary.json) and
[`metrics.csv`](metrics.csv). The earlier EgoSpeak-only record is
[`easycom_egospeak_three_level_eval_2026-09-16`](../easycom_egospeak_three_level_eval_2026-09-16/README.md).