icassp2027-experiment/docs/easycom_event_review_tool.md

EasyCom wearer-centric EOT/INT pre-label and review pipeline

The workflow has three explicit stages. Speech activity timestamps are read-only throughout; reviewed events never modify the dense 5 Hz activity ground truth.

EasyCom participant speech intervals
  -> rule candidates + structured timing evidence
  -> DeepSeek semantic pre-label (optional but recommended)
  -> human Accept / Relabel / Exclude / Second review

1. Generate candidates

Develop the rules on training-side Sessions 4--12:

python3 prepare_easycom_eot_int_review.py \
  --dataset-root /path/to/easycom \
  --sessions Session_4 Session_5 Session_6 Session_7 Session_8 Session_9 Session_10 Session_11 Session_12 \
  --output-dir reports/easycom_floor_event_candidates

The generator emits two independent sources:

• every wearer 0->1 (WEARER_ONSET, ENTRY_CANDIDATE) and 1->0 (WEARER_OFFSET, YIELD_CANDIDATE) transition;
• hard-negative regions that need no wearer transition: SAME_SPEAKER_PAUSE, OTHER_TO_OTHER, OTHER_BACKCHANNEL, and NON_TAKEOVER_OVERLAP.

Wearer transitions are initially unsure; timing alone is not a semantic positive label. Hard-negative scans are rule hints and still require AI and human review. The JSONL retains candidate source, original event time, participant-level activity, raw durations/gaps/overlap evidence, transcript, and Target_of_Speech when present. Target_of_Speech is annotation assistance only and cannot be a deployable model input.

All heuristic durations are configurable rather than frozen paper rules:

--relation-window-seconds
--same-speaker-pause-max-seconds
--other-handover-max-seconds
--short-vocalization-max-seconds
--continuation-min-seconds

Use --max-candidates 1 for a one-event smoke test. summary.json records the exact configuration and candidate counts. easycom_session_timelines.json stores the read-only participant speech lanes used by the full-session web timeline; it is not sent to DeepSeek.

2. Add DeepSeek pre-labels

First inspect exactly what would be uploaded:

python3 adjudicate_easycom_eot_int_candidates.py \
  --input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
  --output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
  --model deepseek-v4-flash \
  --max-events 1 \
  --dry-run

After setting DEEPSEEK_API_KEY and obtaining authorization to upload the candidate excerpts:

python3 adjudicate_easycom_eot_int_candidates.py \
  --input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
  --output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
  --model deepseek-v4-flash \
  --allow-external-transcript-upload

DeepSeek sees candidate source, structured timing, per-speaker activity, and transcript together. It returns one of POSITIVE, HARD_NEGATIVE, NOT_APPLICABLE, EXCLUDE, or UNSURE; positives additionally receive EOT_RESPONSE or INT_YIELD, and hard negatives receive a named subtype. It cannot modify timestamps. The output preserves both the original rule candidate and the complete AI decision/provenance.

3. Human review

Run the web workspace on the AI-enriched JSONL:

python3 review_floor_events.py \
  --dataset-root /path/to/easycom \
  --events reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
  --output reports/easycom_event_reviews/reviewer_1.jsonl

Open http://127.0.0.1:8765. The page shows a session candidate overview, participant activity near the event, video/audio, transcript, structured timing, and the AI verdict/confidence/reason. The common path is one operation:

• Accept keeps the AI verdict/type;
• Relabel expands verdict and subtype controls;
• Exclude requires a concrete data-quality reason;
• Unsure / Second review preserves the event for adjudication.

Boundary correction is hidden under an optional disclosure and is limited to one 200 ms bin. The original timestamp is always retained. Each save atomically rewrites JSONL and CSV while preserving the rule candidate, AI decision, human decision, relabel flag, exclusion state, reviewer, and provenance.

Freeze discipline

Develop the guide and resolve disagreements only on Sessions 4--12. Freeze categories and thresholds before reviewing official test Sessions 1--3. Use two independent human review files plus a separate adjudication output for paper- quality labels. DeepSeek is pre-label metadata, not an independent reviewer.