icassp2027-experiment/docs/easycom_event_review_tool.md

EasyCom EOT/INT review workspace

review_floor_events.py is a local annotation workspace for reviewing the manuscript-aligned regions produced by prepare_easycom_eot_int_review.py. It can also open the older timing/transcript proxy reports from derive_easycom_interaction_events.py and audit_easycom_floor_events.py, but those older reports do not have the directionality or hard-negative region contract required by the current manuscript. Raw EasyCom media stays read-only; human decisions are stored separately as JSONL and CSV.

Prepare manuscript-aligned candidates

Develop the annotation guide on training-side Sessions 4--12:

python3 prepare_easycom_eot_int_review.py \
  --dataset-root /path/to/easycom \
  --sessions Session_4 Session_5 Session_6 Session_7 Session_8 Session_9 Session_10 Session_11 Session_12 \
  --output-dir reports/easycom_floor_event_candidates

For a one-event pilot, add --max-candidates 1. The generator follows the manuscript direction exactly:

• EOT-response begins at an aggregate-other offset while the wearer is silent.
• INT-yield begins at an aggregate-other onset while the wearer is active.
• An observed wearer transition before the competing other transition is a timing-positive candidate.
• A competing other transition, or a fully observed no-transition window, is a timing hard-negative candidate.
• A source boundary without a complete outcome/window is marked for exclusion.

Automatic labels remain candidates until independent human semantic review.

Optional DeepSeek suggestion

DeepSeek suggestions are triage metadata and never count as a human label. The script sends only the candidate transcript excerpt and timing fields. Inspect the exact request without network access first:

python3 adjudicate_easycom_eot_int_candidates.py \
  --input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
  --output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
  --model deepseek-v4-flash \
  --max-events 1 \
  --dry-run

After setting DEEPSEEK_API_KEY, make the authorized request:

python3 adjudicate_easycom_eot_int_candidates.py \
  --input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
  --output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
  --model deepseek-v4-flash \
  --max-events 1 \
  --allow-external-transcript-upload

Point the review server at the DeepSeek-enriched JSONL to show its suggestion beside the video. Keep suggestions hidden from an initial independent human pass if anchoring bias is a concern.

Start the local workspace

python3 review_floor_events.py \
  --dataset-root /path/to/easycom \
  --events reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
  --output reports/easycom_event_reviews/reviewer_1.jsonl

Open http://127.0.0.1:8765. The page resolves the source MP4 and transcript, seeks to a six-second event context, and supports session/split/task/status filtering.

Each save atomically rewrites both JSONL and CSV. The review explicitly records positive, hard_negative, exclude, or uncertain; the hard-negative subtype; boundary validity; onset function; conversational routing; offset outcome; three 200 ms anchor/region corrections; confidence; and reviewer ID.

Freeze discipline

Develop the annotation guide and resolve category disagreements on training Sessions 4--12. Freeze the guide before reviewing official test Sessions 1--3. Do not use test reviews to change candidate windows, label categories, or the R=.8 threshold calibration rule.

For paper-quality labels, use two independent reviewers with separate --output files and adjudicate disagreements in a third output. A non-empty reviewer ID is required on every saved row. DeepSeek is not one of the two reviewers.

This workspace produces audited labels. It does not yet convert them into the final committed-event matching metrics; that integration follows only after the schema and real-data pilot are frozen.