Add causal oracle person information screen

2026-09-07 08:41:34 +08:00
parent fb22108c57
commit ecf00185f1
17 changed files with 1364 additions and 20 deletions
@@ -0,0 +1,116 @@
+# EasyCom anonymous track-GRU residual: matched four-epoch screen
+
+## Question
+
+Does causal continuity within anonymous face tracks improve future floor-state
+prediction beyond the same-seed `G+A+pooled-V` head?
+
+This experiment compares the matched pooled checkpoint with a residual branch
+containing a shared per-track GRU and a `G+A` query over current track states.
+The pooled path is copied exactly and frozen. The residual is multiplied by a
+learned scalar initialized at zero.
+
+## Protocol and provenance
+
+- Global features: `open_stream_2fps_8s_last4_stack`.
+- Local sidecars: `local_qwen_1s_encoded_tf4576`.
+- Training: EasyCom Sessions 4--11.
+- Development/model selection: Session 12.
+- Official test: Sessions 1--3.
+- Development seeds: 0, 1 and 2; four epochs.
+- Test seeds observed: seed 0 only.
+- Loss: `lstr-mce`; layer fusion: `concat-mlp`.
+- Person history: 16 observations at 2 FPS, with a 1-second maximum track gap.
+- Reported table protocol: `causal_valid_only`; all values are percentage
+  points.
+- Checkpoints: `checkpoints/easycom_track_gru_residual/` on the remote L20
+  machine.
+- Predictions: `results/easycom_track_gru_residual/` for development and
+  `results/easycom_track_gru_residual_test_check/` for the one-seed test audit.
+
+The test audit was run before the residual optimization defect described below
+was diagnosed. It is recorded for transparency and must not be used to tune a
+corrected architecture.
+
+## Session 12 development result
+
+| Run | 0.2 s mAP | BG AP | wearer AP | others AP | 10-horizon mean mAP |
+|---|---:|---:|---:|---:|---:|
+| pooled seed 0 | 72.91519 | 58.39896 | 68.44729 | 91.89933 | 59.42976 |
+| track-GRU seed 0 | 72.91538 | 58.39965 | 68.44750 | 91.89900 | 59.42956 |
+| pooled seed 1 | 73.02189 | 57.73258 | 69.35309 | 91.97999 | 59.50838 |
+| track-GRU seed 1 | 73.02358 | 57.74019 | 69.35432 | 91.97624 | 59.50736 |
+| pooled seed 2 | 72.45010 | 57.26123 | 68.35503 | 91.73403 | 59.53730 |
+| track-GRU seed 2 | 72.47576 | 57.15675 | 68.59149 | 91.67904 | 59.57771 |
+
+Three-seed paired `track-GRU - pooled` deltas, mean +/- sample standard
+deviation:
+
+| Metric | Delta (percentage points) |
+|---|---:|
+| 0.2 s mAP | +0.00918 +/- 0.01429 |
+| BG AP | -0.03206 +/- 0.06281 |
+| wearer AP | +0.07930 +/- 0.13610 |
+| others AP | -0.01969 +/- 0.03062 |
+| 10-horizon mean mAP | +0.01306 +/- 0.02368 |
+
+The development difference is effectively zero and is much smaller than the
+between-seed variation.
+
+## Sessions 1--3 test audit
+
+Only seed 0 was evaluated after the matched checkpoints had already been
+selected on Session 12.
+
+| Run | 0.2 s mAP | BG AP | wearer AP | others AP | 10-horizon mean mAP |
+|---|---:|---:|---:|---:|---:|
+| pooled seed 0 | 81.74161 | 68.57326 | 84.71790 | 91.93366 | 64.88810 |
+| track-GRU seed 0 | 81.74160 | 68.57367 | 84.71783 | 91.93330 | 64.88780 |
+| track-GRU - pooled | -0.00001 | +0.00041 | -0.00007 | -0.00037 | -0.00030 |
+
+Under the EgoSpeak full-timeline printout, the corresponding ten-horizon means
+were 64.87493 for pooled and 64.87463 for track-GRU. The small difference from
+the table is due to the evaluation mask; protocols must not be mixed.
+
+## Gate diagnostic
+
+The selected track checkpoints retained an almost closed residual gate:
+
+| Seed | Selected epoch | saved `tanh(gate)` | selection-time dev mean mAP |
+|---|---:|---:|---:|
+| 0 | 1 | -0.00152618 | 59.45070 |
+| 1 | 1 | +0.00409810 | 59.52841 |
+| 2 | 2 | +0.01081575 | 59.59214 |
+
+The checkpoint JSON contains history only through the epoch at which that best
+checkpoint was written, so it is not a complete log of later non-improving
+epochs. The values above nevertheless describe the models used for inference.
+
+With logits
+
+```text
+base_logits + tanh(gate) * delta(track_parameters)
+```
+
+the zero-initialized gate makes the first-step gradient of all track/delta
+parameters exactly zero. Only the scalar gate can move initially; subsequent
+branch gradients are scaled by its still-small value. The saved gates and the
+near-identical test predictions are therefore consistent with an optimization
+failure in the residual parameterization.
+
+## Interpretation and decision
+
+- The implemented zero-gated track-GRU produces no meaningful dense-AP gain.
+- This run does **not** establish that track continuity is uninformative,
+  because the residual branch was almost closed throughout the selected
+  models.
+- It also does not support event-level or visual-grounding claims: no corrected
+  competing-risk event evaluation or wrong-track/time-shift control was run.
+- The appropriate next diagnostic is to remove the scalar gate, zero-initialize
+  the last residual output layer, keep the pooled base frozen, and verify that
+  residual-output and GRU gradients become nonzero during optimization.
+- Tune that corrected model on Session 12 only. Do not inspect Sessions 1--3
+  again until its architecture and stopping rule are frozen.
+
+Machine-readable values are in [`summary.json`](summary.json) and per-run rows
+are in [`metrics.csv`](metrics.csv).
@@ -0,0 +1,9 @@
+split,protocol,variant,seed,map_0p2,bg_ap_0p2,wearer_ap_0p2,others_ap_0p2,mean_map,best_epoch,saved_track_gate
+dev_session12,causal_valid_only,pooled,0,72.91519,58.39896,68.44729,91.89933,59.42976,,
+dev_session12,causal_valid_only,track_gru,0,72.91538,58.39965,68.44750,91.89900,59.42956,1,-0.0015261819353327155
+dev_session12,causal_valid_only,pooled,1,73.02189,57.73258,69.35309,91.97999,59.50838,,
+dev_session12,causal_valid_only,track_gru,1,73.02358,57.74019,69.35432,91.97624,59.50736,1,0.004098099190741777
+dev_session12,causal_valid_only,pooled,2,72.45010,57.26123,68.35503,91.73403,59.53730,,
+dev_session12,causal_valid_only,track_gru,2,72.47576,57.15675,68.59149,91.67904,59.57771,2,0.010815753601491451
+test_sessions1_3,causal_valid_only,pooled,0,81.74161,68.57326,84.71790,91.93366,64.88810,,
+test_sessions1_3,causal_valid_only,track_gru,0,81.74160,68.57367,84.71783,91.93330,64.88780,1,-0.0015261819353327155
@@ -0,0 +1,61 @@
+{
+  "record": "easycom_track_gru_residual_4ep_2026-09-07",
+  "dataset": "easycom",
+  "global_features": "open_stream_2fps_8s_last4_stack",
+  "local_features": "local_qwen_1s_encoded_tf4576",
+  "train_sessions": [4, 5, 6, 7, 8, 9, 10, 11],
+  "dev_sessions": [12],
+  "test_sessions": [1, 2, 3],
+  "epochs": 4,
+  "loss": "lstr-mce",
+  "layer_fusion": "concat-mlp",
+  "person_history_frames": 16,
+  "person_history_fps": 2.0,
+  "person_track_max_gap_seconds": 1.0,
+  "protocol": "causal_valid_only",
+  "units": "percentage_points",
+  "dev": {
+    "seeds": [0, 1, 2],
+    "paired_delta_track_minus_pooled": {
+      "map_0p2": {"mean": 0.00918, "sample_std": 0.01429},
+      "bg_ap_0p2": {"mean": -0.03206, "sample_std": 0.06281},
+      "wearer_ap_0p2": {"mean": 0.07930, "sample_std": 0.13610},
+      "others_ap_0p2": {"mean": -0.01969, "sample_std": 0.03062},
+      "mean_map": {"mean": 0.01306, "sample_std": 0.02368}
+    }
+  },
+  "test_audit": {
+    "seeds": [0],
+    "pooled": {
+      "map_0p2": 81.74161,
+      "bg_ap_0p2": 68.57326,
+      "wearer_ap_0p2": 84.71790,
+      "others_ap_0p2": 91.93366,
+      "mean_map": 64.88810,
+      "full_timeline_mean_map": 64.87493
+    },
+    "track_gru": {
+      "map_0p2": 81.74160,
+      "bg_ap_0p2": 68.57367,
+      "wearer_ap_0p2": 84.71783,
+      "others_ap_0p2": 91.93330,
+      "mean_map": 64.88780,
+      "full_timeline_mean_map": 64.87463
+    },
+    "paired_delta_track_minus_pooled": {
+      "map_0p2": -0.00001,
+      "bg_ap_0p2": 0.00041,
+      "wearer_ap_0p2": -0.00007,
+      "others_ap_0p2": -0.00037,
+      "mean_map": -0.00030
+    },
+    "warning": "Observed before diagnosing residual optimization; do not use to tune the corrected architecture."
+  },
+  "selected_track_checkpoints": [
+    {"seed": 0, "best_epoch": 1, "saved_tanh_gate": -0.0015261819353327155, "selection_dev_mean_map": 0.5945069810061214},
+    {"seed": 1, "best_epoch": 1, "saved_tanh_gate": 0.004098099190741777, "selection_dev_mean_map": 0.5952841235072595},
+    {"seed": 2, "best_epoch": 2, "saved_tanh_gate": 0.010815753601491451, "selection_dev_mean_map": 0.5959213772894789}
+  ],
+  "decision": "No meaningful gain. Treat as a zero-gate optimization failure, not evidence against track continuity.",
+  "next_step": "Remove scalar gate, zero-initialize the final residual layer, verify nonzero branch gradients, and screen only on Session 12."
+}