Add EgoSpeak dense event proxy evaluation

2026-09-15 13:35:35 +08:00
parent b3dc02b293
commit a8a09d8fc5
8 changed files with 826 additions and 24 deletions
@@ -0,0 +1,48 @@
+# EgoSpeak dense-trajectory event proxy
+
+`evaluate_egospeak_event_proxy.py` provides a deliberately limited bridge
+between EgoSpeak-compatible dense future-activity predictions and reviewed
+wearer ENTRY/INT opportunities. It does **not** claim that wearer voice
+activity is a semantic floor-event detector.
+
+## Score
+
+For every positive event, the default decision region contains the three 5-Hz
+source rows immediately before its corrected behavior anchor. An independently
+mined hard negative uses the first three rows of its labeled opportunity. A raw
+wearer onset/offset relabeled as a hard negative uses the same pre-boundary
+region as a positive. Regions shorter than 0.6 seconds are excluded and
+counted; longer annotation evidence is not scored.
+
+At each source row, the script recovers the target-aligned wearer trajectory
+at 0.6, 0.8, ..., 2.0 seconds. ENTRY uses its mean and INT uses one minus its
+mean. Max-pooling over the three decision rows produces one score per reviewed
+opportunity. The output must be called an `activity-derived event proxy`.
+
+## Threshold-free provisional run
+
+This reports AP without fitting thresholds on Sessions 1--3:
+
+```bash
+python3 evaluate_egospeak_event_proxy.py \
+  --reviews reports/easycom_entry_int_sessions1_4_v16/easycom_eot_int_provisional_llm_reviews.jsonl \
+  --run egospeak=results/egospeak_miniroada_floor_eval \
+  --sessions Session_1 Session_2 Session_3 \
+  --allow-provisional-llm-reviews \
+  --print-event-examples 5 \
+  --output results/easycom_egospeak_event_proxy/sessions1_3_provisional_summary.json
+```
+
+The command writes JSON, a compact metrics CSV, and an event CSV containing the
+three decision-frame scores and the winning selected-horizon wearer trajectory.
+
+## Recall-matched operating points
+
+`--matched-recalls` on official test sessions additionally requires disjoint
+Sessions 4--12 reviews and predictions via `--calibration-reviews`,
+`--calibration-sessions`, and optionally `--calibration-run`. Do not use
+`--allow-in-sample-thresholds` for reported results.
+
+The default 0.6-second decision window and 0.6--2.0-second trajectory range
+remain protocol choices to freeze using training-side data. Report shorter or
+longer windows only as predeclared sensitivity analyses.