icassp2027-experiment/docs/easycom_floor_event_diagnostics.md

# EasyCom floor-event diagnosis: timing, objective, and addressee

These three experiments explain why a dense future-activity oracle can retain
reasonable mAP while response/yield event AP remains low. They are diagnostics,
not three proposed paper contributions. Keep the primary paper protocol as the
pre-boundary competing-risk evaluation until the evidence says otherwise.

## What each experiment isolates

| Diagnostic | Only intended change | If it improves clearly |
|---|---|---|
| pre-boundary vs post-boundary | whether the aggregate-other boundary has already been observed at the decision source | much of the error is boundary anticipation, not weak AV encoding |
| dense activity head vs direct event head | future-state loss vs response/yield loss, using the same frozen inputs | the dense objective/ranking conversion is the main mismatch |
| addressee vs zero-channel control | explicit past `Target_of_Speech == wearer_id` history | aggregate activity misses conversational routing information |

Do not compare raw AP across the pre- and post-boundary protocols as if they
were the same task. Compare methods within one timing protocol, and use the
pre/post difference only as a difficulty decomposition.

## Shared remote environment

```bash
cd /data1/yfzx/wangx/Code/icassp2027-experiment

export DATASET_ROOT=/data1/yfzx/wangx/Datasets/easycom/extracted_easycom
export FEATURE_ROOT=/data1/yfzx/wangx/Datasets/easycom/omni_features/open_stream_2fps_8s_last4_stack
export TARGET_ROOT=/data1/yfzx/wangx/Datasets/easycom/easycom_target_frame
export LOCAL_ROOT=/data1/yfzx/wangx/Datasets/easycom/omni_features/local_qwen_1s_encoded_tf4576
export ORACLE_ROOT=/data1/yfzx/wangx/Datasets/easycom/omni_features/oracle_person_history_2s
```

Change only paths that differ on the remote host. The default feature mode is
`open-stream-replay`.

## 1. Post-boundary reactive diagnostic for the three existing runs

Point the wrapper at already aligned inference directories. Each directory
must contain `pred/Session_*.npy` and `valid/Session_*.npy`.

```bash
export POOLED_AV_RUN=/path/to/pooled_av_result
export ORACLE_RUN=/path/to/aggregate_visible_oracle_result
export MINIROAD_RUN=/path/to/converted_miniroada_result

bash run_easycom_post_boundary_eval.sh
```

Without `CALIBRATION_SESSIONS`, the wrapper explicitly marks threshold results
as exploratory and selects them in-sample. AP is still useful for this first
diagnosis. For a confirmatory result, provide training-side predictions in the
same run directories and use, for example:

```bash
export CALIBRATION_SESSIONS="Session_12"
bash run_easycom_post_boundary_eval.sh
```

The JSON protocol ID becomes
`wearer_floor_event_post_boundary_diagnostic_v1`. Simultaneous wearer changes
at the observed boundary and zero-length windows before an immediate competing
boundary are excluded and counted explicitly.

## 2. Direct response/yield event head

The direct probe uses frozen Omni states, the same mixed-audio vectors and the
same `aggregate-visible` past-activity oracle. Its layer fusion, audio
projection, oracle person attention and shared MLP trunk are the same
`TurnPredictionHead` components as the dense oracle; only the final two-event
readout and event loss differ. It trains on Sessions 4--11, selects epoch and
thresholds on Session 12, and initially evaluates only Session 12. First run a
one-epoch Session-11/12 shape, mask and checkpoint smoke test:

```bash
CUDA_VISIBLE_DEVICES=0 bash run_easycom_floor_event_probes.sh smoke
```

Then run the single aggregate oracle diagnosis:

```bash
CUDA_VISIBLE_DEVICES=0 \
VARIANTS="aggregate-visible" \
bash run_easycom_floor_event_probes.sh all 0
```

This is the clean comparison against the existing dense aggregate-visible
oracle head. A large direct-head gain supports objective mismatch; a small gain
means the oracle information itself is probably weak for the event decision.

## 3. Addressee oracle with a capacity-matched control

Run all three direct-probe variants:

```bash
CUDA_VISIBLE_DEVICES=0 bash run_easycom_floor_event_probes.sh all 0
```

The variants are:

- `aggregate-visible`: 2 s aggregate activity plus observation mask;
- `aggregate-visible-address-control`: the same input plus an all-zero channel;
- `aggregate-visible-addressee`: the same input plus past bins in which a
  visible non-wearer was explicitly annotated as addressing wearer ID 2.

`Group`, `None`, unknown and other-person targets are not coerced into reliable
negative labels. The addressee channel is privileged, annotation-derived and
analysis-only; it is not an online feature or a deployable causal input.

The runner writes one `summary.json` and `metrics.csv` per variant and a compact
`comparison_dev.csv`. Inspect that comparison before touching Sessions 1--3.

## Optional frozen test evaluation

Only after the variants, epochs and interpretation rule are fixed from Session
12, apply the saved probes to official test Sessions 1--3:

```bash
CUDA_VISIBLE_DEVICES=0 bash run_easycom_floor_event_probes.sh test 0
```

This reuses the saved Session-12 thresholds and writes `comparison_test.csv`.
Do not repeatedly inspect test results and redesign the probe.

## Minimum interpretation rule

For response, jointly inspect mean AP, 2 s AP, achieved response recall, false
entry rate, and short-resume trigger rate. For yield, inspect mean AP, 2 s AP,
achieved yield recall, and false-stop rate. Denominators and censoring counts
must match before comparing variants.

The three outcomes lead to different paper decisions:

1. **Only post-boundary improves:** retain the main anticipation task and frame
   the difficulty as predicting a boundary before it is observable. Do not
   claim that a larger event head solves grounding.
2. **Direct event supervision improves:** the main lesson is target alignment;
   build the method around event-aware adaptation and keep dense mAP secondary.
3. **Addressee improves over both aggregate and zero control:** conversational
   routing is the missing information. This motivates a deployable addressee or
   addressedness estimator, but the oracle itself is not the method.
4. **None improves:** the current timing-derived event definition or available
   observations are insufficient. Revisit annotation/task definition before a
   heavier fusion architecture.