icassp2027-experiment/docs/easycom_event_review_tool.md

# EasyCom wearer-centric EOT/INT pre-label and review pipeline

The workflow has three explicit stages. Speech activity timestamps are
read-only throughout; reviewed events never modify the dense 5 Hz activity
ground truth.

```text
EasyCom participant speech intervals
  -> rule candidates + structured timing evidence
  -> DeepSeek semantic pre-label (optional but recommended)
  -> human Accept / Relabel / Exclude / Second review
```

## 1. Generate candidates

Develop the rules on training-side Sessions 4--12:

```bash
python3 prepare_easycom_eot_int_review.py \
  --dataset-root /path/to/easycom \
  --sessions Session_4 Session_5 Session_6 Session_7 Session_8 Session_9 Session_10 Session_11 Session_12 \
  --output-dir reports/easycom_floor_event_candidates
```

The generator emits two independent sources:

- every wearer `0->1` (`WEARER_ONSET`, `ENTRY_CANDIDATE`) and `1->0`
  (`WEARER_OFFSET`, `YIELD_CANDIDATE`) transition;
- hard-negative regions that need no wearer transition:
  `SAME_SPEAKER_PAUSE`, `OTHER_TO_OTHER`, `OTHER_BACKCHANNEL`, and
  `NON_TAKEOVER_OVERLAP`.

Wearer transitions are initially `unsure`; timing alone is not a semantic
positive label. Hard-negative scans are rule hints and still require AI and
human review. The JSONL retains candidate source, original event time,
participant-level activity, raw durations/gaps/overlap evidence, transcript,
and `Target_of_Speech` when present. `Target_of_Speech` is annotation assistance
only and cannot be a deployable model input.

All heuristic durations are configurable rather than frozen paper rules:

```text
--relation-window-seconds
--same-speaker-pause-max-seconds
--other-handover-max-seconds
--short-vocalization-max-seconds
--continuation-min-seconds
```

Use `--max-candidates 1` for a one-event smoke test. `summary.json` records the
exact configuration and candidate counts. `easycom_session_timelines.json`
stores the read-only participant speech lanes used by the full-session web
timeline; it is not sent to DeepSeek.

## 2. Add DeepSeek pre-labels

First inspect exactly what would be uploaded:

```bash
python3 adjudicate_easycom_eot_int_candidates.py \
  --input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
  --output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
  --model deepseek-v4-flash \
  --max-events 1 \
  --dry-run
```

After setting `DEEPSEEK_API_KEY` and obtaining authorization to upload the
candidate excerpts:

```bash
python3 adjudicate_easycom_eot_int_candidates.py \
  --input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
  --output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
  --model deepseek-v4-flash \
  --allow-external-transcript-upload
```

DeepSeek sees candidate source, structured timing, per-speaker activity, and
transcript together. It returns one of `POSITIVE`, `HARD_NEGATIVE`,
`NOT_APPLICABLE`, `EXCLUDE`, or `UNSURE`; positives additionally receive
`EOT_RESPONSE` or `INT_YIELD`, and hard negatives receive a named subtype. It
cannot modify timestamps. The output preserves both the original rule candidate
and the complete AI decision/provenance.

## 3. Human review

Run the web workspace on the AI-enriched JSONL:

```bash
python3 review_floor_events.py \
  --dataset-root /path/to/easycom \
  --events reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
  --output reports/easycom_event_reviews/reviewer_1.jsonl
```

Open `http://127.0.0.1:8765`. The page shows a session candidate overview,
participant activity near the event, video/audio, transcript, structured timing,
and the AI verdict/confidence/reason. The common path is one operation:

- **Accept** keeps the AI verdict/type;
- **Relabel** expands verdict and subtype controls;
- **Exclude** requires a concrete data-quality reason;
- **Unsure / Second review** preserves the event for adjudication.

Boundary correction is hidden under an optional disclosure and is limited to
one 200 ms bin. The original timestamp is always retained. Each save atomically
rewrites JSONL and CSV while preserving the rule candidate, AI decision, human
decision, relabel flag, exclusion state, reviewer, and provenance.

## Freeze discipline

Develop the guide and resolve disagreements only on Sessions 4--12. Freeze
categories and thresholds before reviewing official test Sessions 1--3. Use two
independent human review files plus a separate adjudication output for paper-
quality labels. DeepSeek is pre-label metadata, not an independent reviewer.