Fix opportunity competing-risk evaluation

This commit is contained in:
Xin Wang
2026-09-16 04:31:18 +08:00
parent 02e83c0c11
commit d4c900ab13
10 changed files with 310 additions and 88 deletions
@@ -1,5 +1,12 @@
# EasyCom Omni floor-event screening
> **Protocol erratum.** Dense and raw-boundary metrics remain valid. All
> opportunity-conditioned ENTRY/INT values below used the superseded v1 loader,
> which could cross an intervening ENTRY competitor and selected the latest
> onset in ambiguous multi-competitor INT cases. Retain them for provenance
> only; do not use them for model selection or manuscript claims before a v2
> rerun.
This record freezes the first comparison of EgoSpeak, Omni `G+A`, Omni
`G+A+pooled-V`, and the three-seed current-frame F1 Frame-set Attention screen
on EasyCom Sessions 1--3. Values were produced on the remote experiment host
@@ -11,9 +18,8 @@ CSVs remain on that host and are not committed here.
1. Dense future activity uses causal-valid framewise AP/mAP over ten horizons.
2. Raw wearer endpoint and aggregate-other-boundary AP are threshold-free
timing diagnostics; they are not semantic EOT/INT labels.
3. Opportunity-conditioned ENTRY and INT/YIELD AP assigns one causally
censored activity-derived score to each provisional LLM-assisted reviewed
opportunity.
3. Opportunity-conditioned ENTRY and INT/YIELD AP is a historical v1 screen
that requires rerunning with the corrected v2 competing-risk protocol.
All opportunity runs use identical denominators: 163 ENTRY positives and 335
hard negatives, with 81 invalid/post-outcome cases; and 28 INT positives and
@@ -39,7 +45,7 @@ from EgoSpeak to G+A to G+A+pooled-V as follows: onset AP
offset/yield-timing gain comes from G+A; pooled visual input adds almost
nothing to raw transition timing.
## Interpretation
## Historical v1 interpretation
- G+A strongly improves dense forecasting and generic wearer-offset/yield
timing over EgoSpeak, but not reviewed semantic opportunities. In particular,
@@ -77,4 +83,3 @@ nothing to raw transition timing.
Exact aggregate values are stored in [`summary.json`](summary.json) and
[`metrics.csv`](metrics.csv). The earlier EgoSpeak-only record is
[`easycom_egospeak_three_level_eval_2026-09-16`](../easycom_egospeak_three_level_eval_2026-09-16/README.md).