98 lines
4.0 KiB
Markdown
98 lines
4.0 KiB
Markdown
|
# EasyCom EOT/INT review workspace
|
||
|
|
|
||
|
|
`review_floor_events.py` is a local annotation workspace for reviewing the
|
||
|
|
manuscript-aligned regions produced by `prepare_easycom_eot_int_review.py`.
|
||
|
|
It can also open the older timing/transcript proxy reports from
|
||
|
|
`derive_easycom_interaction_events.py` and `audit_easycom_floor_events.py`, but
|
||
|
|
those older reports do not have the directionality or hard-negative region
|
||
|
|
contract required by the current manuscript. Raw EasyCom media stays read-only;
|
||
|
|
human decisions are stored separately as JSONL and CSV.
|
||
|
|
|
||
|
|
## Prepare manuscript-aligned candidates
|
||
|
|
|
||
|
|
Develop the annotation guide on training-side Sessions 4--12:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
python3 prepare_easycom_eot_int_review.py \
|
||
|
|
--dataset-root /path/to/easycom \
|
||
|
|
--sessions Session_4 Session_5 Session_6 Session_7 Session_8 Session_9 Session_10 Session_11 Session_12 \
|
||
|
|
--output-dir reports/easycom_floor_event_candidates
|
||
|
|
```
|
||
|
|
|
||
|
|
For a one-event pilot, add `--max-candidates 1`. The generator follows the
|
||
|
|
manuscript direction exactly:
|
||
|
|
|
||
|
|
- EOT-response begins at an aggregate-other offset while the wearer is silent.
|
||
|
|
- INT-yield begins at an aggregate-other onset while the wearer is active.
|
||
|
|
- An observed wearer transition before the competing other transition is a
|
||
|
|
timing-positive candidate.
|
||
|
|
- A competing other transition, or a fully observed no-transition window, is a
|
||
|
|
timing hard-negative candidate.
|
||
|
|
- A source boundary without a complete outcome/window is marked for exclusion.
|
||
|
|
|
||
|
|
Automatic labels remain candidates until independent human semantic review.
|
||
|
|
|
||
|
|
## Optional DeepSeek suggestion
|
||
|
|
|
||
|
|
DeepSeek suggestions are triage metadata and never count as a human label. The
|
||
|
|
script sends only the candidate transcript excerpt and timing fields. Inspect
|
||
|
|
the exact request without network access first:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
python3 adjudicate_easycom_eot_int_candidates.py \
|
||
|
|
--input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
|
||
|
|
--output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
|
||
|
|
--model deepseek-v4-flash \
|
||
|
|
--max-events 1 \
|
||
|
|
--dry-run
|
||
|
|
```
|
||
|
|
|
||
|
|
After setting `DEEPSEEK_API_KEY`, make the authorized request:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
python3 adjudicate_easycom_eot_int_candidates.py \
|
||
|
|
--input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
|
||
|
|
--output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
|
||
|
|
--model deepseek-v4-flash \
|
||
|
|
--max-events 1 \
|
||
|
|
--allow-external-transcript-upload
|
||
|
|
```
|
||
|
|
|
||
|
|
Point the review server at the DeepSeek-enriched JSONL to show its suggestion
|
||
|
|
beside the video. Keep suggestions hidden from an initial independent human
|
||
|
|
pass if anchoring bias is a concern.
|
||
|
|
|
||
|
|
## Start the local workspace
|
||
|
|
|
||
|
|
```bash
|
||
|
|
python3 review_floor_events.py \
|
||
|
|
--dataset-root /path/to/easycom \
|
||
|
|
--events reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
|
||
|
|
--output reports/easycom_event_reviews/reviewer_1.jsonl
|
||
|
|
```
|
||
|
|
|
||
|
|
Open `http://127.0.0.1:8765`. The page resolves the source MP4 and transcript,
|
||
|
|
seeks to a six-second event context, and supports session/split/task/status
|
||
|
|
filtering.
|
||
|
|
|
||
|
|
Each save atomically rewrites both JSONL and CSV. The review explicitly records
|
||
|
|
`positive`, `hard_negative`, `exclude`, or `uncertain`; the hard-negative
|
||
|
|
subtype; boundary validity; onset function; conversational routing; offset
|
||
|
|
outcome; three 200 ms anchor/region corrections; confidence; and reviewer ID.
|
||
|
|
|
||
|
|
## Freeze discipline
|
||
|
|
|
||
|
|
Develop the annotation guide and resolve category disagreements on training
|
||
|
|
Sessions 4--12. Freeze the guide before reviewing official test Sessions 1--3.
|
||
|
|
Do not use test reviews to change candidate windows, label categories, or the
|
||
|
|
R=.8 threshold calibration rule.
|
||
|
|
|
||
|
|
For paper-quality labels, use two independent reviewers with separate
|
||
|
|
`--output` files and adjudicate disagreements in a third output. A non-empty
|
||
|
|
reviewer ID is required on every saved row. DeepSeek is not one of the two
|
||
|
|
reviewers.
|
||
|
|
|
||
|
|
This workspace produces audited labels. It does not yet convert them into the
|
||
|
|
final committed-event matching metrics; that integration follows only after
|
||
|
|
the schema and real-data pilot are frozen.
|