Add causal oracle person information screen
This commit is contained in:
+16
-8
@@ -52,7 +52,7 @@ result.
|
||||
matched improvement over the historical no-dev pooled result; correct/wrong
|
||||
visual controls remain missing.
|
||||
|
||||
### Implemented, awaiting remote runtime verification
|
||||
### Measured local-fusion screens
|
||||
|
||||
- Two current-sidecar screening controls: `audio-person-frame-set` adds a
|
||||
zero-gated audio-to-person read before the F1 query, and
|
||||
@@ -62,10 +62,15 @@ result.
|
||||
- `track-gru-residual` builds strictly causal histories from anonymous provided
|
||||
track IDs, applies one shared GRU per track, lets G+A query current track
|
||||
states, and adds a zero-gated correction to an exactly initialized pooled
|
||||
`G+A+V` checkpoint. Its three focused loader/head/CLI round-trip tests passed
|
||||
on the remote PyTorch/L20 environment on 2026-09-07; the one-session GPU
|
||||
smoke and measured experiment are still pending. See
|
||||
[`docs/easycom_track_gru_residual.md`](docs/easycom_track_gru_residual.md).
|
||||
`G+A+V` checkpoint. The matched Session-12 three-seed screen was null
|
||||
(`+0.01306 +/- 0.02368` percentage points in ten-horizon mean mAP). A seed-0
|
||||
Sessions-1--3 audit reproduced pooled predictions to within `-0.00030`
|
||||
points. Saved residual gates were only `-0.0015` to `+0.0108`; because a zero
|
||||
scalar gate also blocks the initial branch gradient, this is recorded as an
|
||||
optimization failure rather than evidence that track continuity is useless.
|
||||
See the immutable
|
||||
[`result record`](results/easycom_track_gru_residual_4ep_2026-09-07/README.md)
|
||||
and [`implementation note`](docs/easycom_track_gru_residual.md).
|
||||
|
||||
### Current blocker
|
||||
|
||||
@@ -89,9 +94,10 @@ diagnostics until competing-risk censoring is fixed.
|
||||
2. **P1:** probe aggregate activity and per-person active-speaker grounding;
|
||||
compare correct, wrong-person and time-shifted visual inputs.
|
||||
3. **P2:** re-evaluate `G`, `G+A`, `G+A+pooled-V` under the frozen event protocol.
|
||||
4. **P3:** run the matched pooled/anonymous-track-GRU residual screen, then add
|
||||
broken-ID, wrong-time and remove-person controls before claiming that track
|
||||
continuity matters.
|
||||
4. **P3:** repair the matched anonymous-track-GRU residual optimization by
|
||||
replacing the scalar zero gate with a zero-initialized residual output
|
||||
layer, and verify nonzero branch gradients on Session 12. Add broken-ID,
|
||||
wrong-time and remove-person controls only if that corrected screen improves.
|
||||
5. **P4:** attempt person-memory adapter/Thinker LoRA only if P1/P3 pass.
|
||||
|
||||
The detailed architecture, controls and stopping gates for P3 are frozen in
|
||||
@@ -228,6 +234,7 @@ Do not reorganize them physically without compatibility wrappers.
|
||||
| `derive_easycom_interaction_events.py` | Deterministic transcript/timing proxies |
|
||||
| `adjudicate_easycom_interaction_events.py` | Optional ambiguous-event adjudication |
|
||||
| `audit_easycom_floor_events.py` | Count EOT/takeover/backchannel candidates |
|
||||
| `prepare_easycom_oracle_features.py` | Build explicit past-only oracle person-activity sidecars for information-value screening |
|
||||
|
||||
### Local features
|
||||
|
||||
@@ -245,6 +252,7 @@ Do not reorganize them physically without compatibility wrappers.
|
||||
| `evaluate_floor_state_changes.py` | CPU floor-state/event proxy evaluation |
|
||||
| `summarize_local_baselines.py` | Aggregate local baseline `metrics.json` files |
|
||||
| `run_easycom_track_gru_residual.sh` | Matched pooled and anonymous track-GRU training/inference launcher |
|
||||
| `run_easycom_oracle_information.sh` | Prepare and screen aggregate/per-person/shuffled/delayed oracle conditions on Session 12 |
|
||||
| `train_omni_turn_lora.py` | Thinker LoRA training from cached AV |
|
||||
| `eval_omni_turn_lora.py` | Thinker LoRA evaluation |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user