Fix opportunity competing-risk evaluation

This commit is contained in:
Xin Wang
2026-09-16 04:31:18 +08:00
parent 02e83c0c11
commit d4c900ab13
10 changed files with 310 additions and 88 deletions
+16 -13
View File
@@ -54,19 +54,23 @@ result.
- An opportunity-conditioned dense-output evaluator aligns ENTRY positives and
core negatives at non-wearer releases, aligns INT positives and hold-floor
negatives at competing non-wearer onsets, and applies strict competing-risk
censoring. It uses only the original wearer probabilities, excludes wearer
backchannels from headline ENTRY FPR, and records unanchored/ambiguous cases.
censoring. ENTRY positives whose selected release is followed by a competing
other onset or intervening wearer activity are excluded, and ambiguous
multi-competitor INT cases are not assigned an arbitrary onset. It uses only
the original wearer probabilities, excludes wearer backchannels from
headline ENTRY FPR, and records unanchored/ambiguous cases.
### Measured screening result
- The released EgoSpeak MiniROADA A+V checkpoint on Sessions 1--3 obtains
`74.124` 0.2-second mAP and `61.155` ten-horizon mean mAP. Raw wearer
onset/offset anticipation is harder (`24.491/38.024` mean AP). On provisional
LLM-assisted matched opportunities, ENTRY AP is `52.487` versus `32.731%`
prevalence, while INT/YIELD AP is `20.135`, indistinguishable from its
`20.438%` prevalence. Figure-7-style continuous wearer curves therefore
support dense forecasting, not semantic floor-control claims. Full curves,
counts, exclusions, and caveats are frozen in
LLM-assisted matched opportunities, the original v1 evaluator reported
ENTRY/INT AP of `52.487/20.135`; those opportunity values are superseded
because v1 could cross a competing ENTRY onset and arbitrarily selected among
multiple INT competitors. Figure-7-style continuous wearer curves support
dense forecasting, not semantic floor-control claims. The unaffected dense
curves and historical v1 opportunity record are in
[`results/easycom_egospeak_three_level_eval_2026-09-16/`](results/easycom_egospeak_three_level_eval_2026-09-16/README.md).
- F1 Frame-set Attention uses fused G+A as one query over the latest half-open
causal person set. On the existing three-seed, four-epoch test screen it
@@ -74,12 +78,11 @@ result.
`64.95099 +/- 0.09451` ten-horizon mean mAP. This is stable, but is not a
matched improvement over the historical no-dev pooled result. Its three-seed
boundary response/yield AP is `28.423 +/- 0.480` / `48.641 +/- 0.391`; on
provisional reviewed opportunities its ENTRY/INT AP is
`41.561 +/- 0.581` / `22.991 +/- 0.202`. These values are respectively
`-4.461/-1.275` points below the historical pooled-V seed-0 result, so the
current F1 screen does not support the hoped-for semantic-selectivity gain.
A same-split three-seed pooled control and correct/wrong visual controls
remain missing. See the immutable
the superseded v1 opportunity evaluator reported ENTRY/INT AP of
`41.561 +/- 0.581` / `22.991 +/- 0.202`; these values require a v2 rerun and
cannot currently support a semantic-selectivity conclusion. A same-split
three-seed pooled control and correct/wrong visual controls remain missing.
See the historical
[`screening record`](results/easycom_omni_floor_screen_2026-09-16/README.md).
### Measured local-fusion screens