Merge branch 'main' of http://182.92.86.220:8300/wx44wx/icassp2027-experiment

2026-09-17 01:30:39 +08:00
parent 6d0c8b733b 7344d0c469
commit 5c29457cc6
12 changed files with 1605 additions and 1 deletions
@@ -6,6 +6,7 @@ readers to the right comparison.

 | Result record | Question answered |
 |---|---|
+| [`easycom_egospeak_omni_residual_2026-09-16/`](easycom_egospeak_omni_residual_2026-09-16/README.md) | Single-seed frozen EgoSpeak + Omni residual dense/raw endpoint gains; semantic annotation evaluation temporarily paused |
 | [`easycom_omni_floor_screen_2026-09-16/`](easycom_omni_floor_screen_2026-09-16/README.md) | Whether G+A, pooled-V, or current-frame F1 improve raw boundaries and provisional semantic opportunities over EgoSpeak |
 | [`easycom_egospeak_three_level_eval_2026-09-16/`](easycom_egospeak_three_level_eval_2026-09-16/README.md) | How dense forecasting, raw boundary anticipation, and provisional opportunity-conditioned ENTRY/YIELD differ for the EgoSpeak baseline |
 | [easycom_oracle_pooled_wearer_audit_2026-09-10/](easycom_oracle_pooled_wearer_audit_2026-09-10/README.md) | Fixed-four-epoch three-seed oracle test results and the limited wearer gains of historical pooled AV |
@@ -0,0 +1,71 @@
+# Stage-one frozen EgoSpeak + Omni residual screen
+
+## Provenance and scope
+
+These values were reported by the user on 2026-09-16, not recomputed on this
+host. The preceding launcher command specifies seed 0, four fixed epochs on
+EasyCom Sessions 4--12, and final evaluation on Sessions 1--3. The reported
+run is `results/easycom_egospeak_omni_residual_seed0/checkpoint_last.pt`; its
+actual checkpoint hash/config and sample counts were not supplied and remain
+unverified. The base is the released `miniroad_easycom_AV.pth` checkpoint.
+Implementation: [`stage-one runbook`](../../docs/easycom_egospeak_omni_residual.md).
+
+The original MiniROADA A+V path and Omni backbone remain frozen; only the
+tail-phase-aware residual fusion is trained. Existing Omni states use the
+8-s open-stream configuration and a fixed mean over the last-four stack.
+Both full-timeline and causal-valid dense numbers are retained in
+[`metrics.csv`](metrics.csv). All numbers in this record are percentages;
+deltas are percentage points. Displayed values are rounded console output.
+
+## Findings
+
+| Metric | Frozen EgoSpeak | Fused | Delta |
+|---|---:|---:|---:|
+| Causal-valid 0.2-s dense mAP | 74.204 | 77.739 | +3.535 |
+| Causal-valid 0.2-s wearer AP | 63.750 | 72.716 | +8.966 |
+| Causal-valid ten-horizon dense mean mAP | 61.193 | 62.714 | +1.521 |
+| Full-timeline ten-horizon dense mean mAP | 61.155 | 62.676 | +1.521 |
+| Ten-horizon raw onset endpoint AP | 24.491 | 25.091 | +0.600 |
+| Ten-horizon raw offset endpoint AP | 38.024 | 40.604 | +2.580 |
+
+Offset AP improves at all ten horizons, by about 2.8--3.5 points at
+0.4--1.2 s. Onset AP improves at nine horizons; its largest gain is +1.634
+points at 0.8 s, while long-horizon differences are small. This is a
+single-seed positive screen, strongest for future wearer cessation. It is
+not a significance test or evidence of semantic floor-yield understanding.
+A parameter-matched EgoSpeak-only residual control and additional seeds
+remain necessary to attribute the gain specifically to Omni.
+
+## Raw anticipation definition
+
+For each horizon h=0.2,...,2.0 s, use valid source/target pairs from the same
+session. Onset is evaluated only when wearer activity W(t)=0: positive if
+W(t+h)=1, negative if W(t+h)=0, with score p_wearer(t,h). Offset is evaluated
+only when W(t)=1: positive if W(t+h)=0, negative if W(t+h)=1, with score
+1-p_wearer(t,h). AP ranks all eligible samples pooled over Sessions 1--3;
+the headline averages ten horizon APs. This is endpoint state-change
+anticipation, not one-event-one-vote timestamp detection. Intermediate
+transitions can be invisible when source and endpoint states agree.
+
+The evaluator also computed exploratory/in-sample operating points, but
+only threshold-free endpoint APs are recorded here. They need no Session-4
+threshold calibration. Do not use the accompanying in-sample rate columns
+as confirmatory test results.
+
+## Annotation decision and next stage
+
+The user suspects issues with the provisional LLM-assisted semantic
+annotations. Opportunity-conditioned ENTRY/YIELD evaluation is therefore
+temporarily paused pending an annotation audit. Historical provisional
+metrics remain archived; they are neither human-reviewed ground truth nor
+discarded because of model performance. No new semantic result or semantic
+loss is introduced in this screen.
+
+The next requested implementation is selective MiniROADA GRU/anticipation
+fine-tuning from the stage-one checkpoint, keeping original feature
+extractors and Omni frozen. This is an exploratory continuation of the
+previously documented route, not passage of the still-unfinished P0 semantic
+evaluation gate. Do not tune windows, loss weights or stopping on Sessions
+1--3; compare fixed budgets and preserve the frozen stage-one/original base
+as references. A four-epoch continuation has eight total adaptation epochs
+and is not a matched-budget comparison against the four-epoch first stage.
@@ -0,0 +1,21 @@
+run,horizon_seconds,dense_full_map,dense_causal_map,dense_causal_background_ap,dense_causal_wearer_ap,dense_causal_others_ap,onset_endpoint_ap,offset_endpoint_ap
+egospeak,0.2,74.124,74.204,68.189,63.750,90.673,5.139,9.696
+egospeak,0.4,68.291,68.349,58.596,59.595,86.857,9.878,18.846
+egospeak,0.6,64.267,64.313,52.373,56.619,83.947,15.389,27.730
+egospeak,0.8,61.091,61.128,47.629,54.172,81.585,21.093,34.971
+egospeak,1.0,59.734,59.768,46.172,52.916,80.216,27.225,40.646
+egospeak,1.2,58.774,58.805,45.840,51.488,79.088,30.997,45.112
+egospeak,1.4,57.471,57.498,43.587,50.765,78.143,33.563,47.880
+egospeak,1.6,57.064,57.090,43.518,49.739,78.012,34.046,50.444
+egospeak,1.8,55.933,55.954,42.227,48.605,77.031,34.030,52.051
+egospeak,2.0,54.802,54.821,40.349,47.393,76.720,33.550,52.862
+fused,0.2,77.658,77.739,69.105,72.716,91.396,5.722,11.756
+fused,0.4,71.510,71.570,59.729,67.888,87.093,10.749,21.920
+fused,0.6,66.855,66.903,53.297,63.553,83.858,16.613,31.248
+fused,0.8,63.212,63.251,48.874,59.713,81.167,22.727,38.383
+fused,1.0,60.961,60.996,46.574,56.953,79.460,27.946,43.741
+fused,1.2,59.410,59.441,45.243,54.917,78.163,31.260,47.922
+fused,1.4,58.375,58.404,43.983,53.446,77.782,33.502,50.269
+fused,1.6,57.336,57.361,42.934,51.712,77.439,34.053,52.173
+fused,1.8,56.222,56.244,41.284,50.607,76.840,34.202,53.814
+fused,2.0,55.216,55.235,39.820,49.470,76.415,34.137,54.812
@@ -0,0 +1,23 @@
+{
+  "date": "2026-09-16",
+  "source": "user-reported console output; not locally recomputed",
+  "units": "percent; deltas are percentage points",
+  "split": {"train": [4, 5, 6, 7, 8, 9, 10, 11, 12], "test": [1, 2, 3]},
+  "seed": 0,
+  "epochs": 4,
+  "configuration_provenance": "preceding command; actual checkpoint/config not inspected",
+  "checkpoint": "results/easycom_egospeak_omni_residual_seed0/checkpoint_last.pt",
+  "checkpoint_sha256": null,
+  "base_checkpoint": "third_party/EgoSpeak/MiniROAD/checkpoints/miniroad_easycom_AV.pth",
+  "omni_memory_seconds": 8,
+  "omni_layer_fusion": "last-four fixed mean",
+  "num_seeds": 1,
+  "sample_counts": null,
+  "metrics": {
+    "egospeak": {"dense_causal_02_map": 74.204, "dense_causal_02_wearer_ap": 63.750, "dense_causal_mean_map": 61.193, "dense_full_mean_map": 61.155, "onset_mean_ap": 24.491, "offset_mean_ap": 38.024},
+    "fused": {"dense_causal_02_map": 77.739, "dense_causal_02_wearer_ap": 72.716, "dense_causal_mean_map": 62.714, "dense_full_mean_map": 62.676, "onset_mean_ap": 25.091, "offset_mean_ap": 40.604},
+    "fused_minus_egospeak": {"dense_causal_02_map": 3.535, "dense_causal_02_wearer_ap": 8.966, "dense_causal_mean_map": 1.521, "dense_full_mean_map": 1.521, "onset_mean_ap": 0.600, "offset_mean_ap": 2.580}
+  },
+  "semantic_evaluation": "paused: provisional annotation quality requires audit",
+  "claim_boundary": "single-seed dense/endpoint improvement; not semantic ENTRY/YIELD or Omni-specific causal attribution"
+}