117 lines
4.5 KiB
Markdown
117 lines
4.5 KiB
Markdown
# EasyCom wearer-centric EOT/INT pre-label and review pipeline
|
|
|
|
The workflow has three explicit stages. Speech activity timestamps are
|
|
read-only throughout; reviewed events never modify the dense 5 Hz activity
|
|
ground truth.
|
|
|
|
```text
|
|
EasyCom participant speech intervals
|
|
-> rule candidates + structured timing evidence
|
|
-> DeepSeek semantic pre-label (optional but recommended)
|
|
-> human Accept / Relabel / Exclude / Second review
|
|
```
|
|
|
|
## 1. Generate candidates
|
|
|
|
Develop the rules on training-side Sessions 4--12:
|
|
|
|
```bash
|
|
python3 prepare_easycom_eot_int_review.py \
|
|
--dataset-root /path/to/easycom \
|
|
--sessions Session_4 Session_5 Session_6 Session_7 Session_8 Session_9 Session_10 Session_11 Session_12 \
|
|
--output-dir reports/easycom_floor_event_candidates
|
|
```
|
|
|
|
The generator emits two independent sources:
|
|
|
|
- every wearer `0->1` (`WEARER_ONSET`, `ENTRY_CANDIDATE`) and `1->0`
|
|
(`WEARER_OFFSET`, `YIELD_CANDIDATE`) transition;
|
|
- hard-negative regions that need no wearer transition:
|
|
`SAME_SPEAKER_PAUSE`, `OTHER_TO_OTHER`, `OTHER_BACKCHANNEL`, and
|
|
`NON_TAKEOVER_OVERLAP`.
|
|
|
|
Wearer transitions are initially `unsure`; timing alone is not a semantic
|
|
positive label. Hard-negative scans are rule hints and still require AI and
|
|
human review. The JSONL retains candidate source, original event time,
|
|
participant-level activity, raw durations/gaps/overlap evidence, transcript,
|
|
and `Target_of_Speech` when present. `Target_of_Speech` is annotation assistance
|
|
only and cannot be a deployable model input.
|
|
|
|
All heuristic durations are configurable rather than frozen paper rules:
|
|
|
|
```text
|
|
--relation-window-seconds
|
|
--same-speaker-pause-max-seconds
|
|
--other-handover-max-seconds
|
|
--short-vocalization-max-seconds
|
|
--continuation-min-seconds
|
|
```
|
|
|
|
Use `--max-candidates 1` for a one-event smoke test. `summary.json` records the
|
|
exact configuration and candidate counts. `easycom_session_timelines.json`
|
|
stores the read-only participant speech lanes used by the full-session web
|
|
timeline; it is not sent to DeepSeek.
|
|
|
|
## 2. Add DeepSeek pre-labels
|
|
|
|
First inspect exactly what would be uploaded:
|
|
|
|
```bash
|
|
python3 adjudicate_easycom_eot_int_candidates.py \
|
|
--input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
|
|
--output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
|
|
--model deepseek-v4-flash \
|
|
--max-events 1 \
|
|
--dry-run
|
|
```
|
|
|
|
After setting `DEEPSEEK_API_KEY` and obtaining authorization to upload the
|
|
candidate excerpts:
|
|
|
|
```bash
|
|
python3 adjudicate_easycom_eot_int_candidates.py \
|
|
--input reports/easycom_floor_event_candidates/easycom_eot_int_candidates.jsonl \
|
|
--output reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
|
|
--model deepseek-v4-flash \
|
|
--allow-external-transcript-upload
|
|
```
|
|
|
|
DeepSeek sees candidate source, structured timing, per-speaker activity, and
|
|
transcript together. It returns one of `POSITIVE`, `HARD_NEGATIVE`,
|
|
`NOT_APPLICABLE`, `EXCLUDE`, or `UNSURE`; positives additionally receive
|
|
`EOT_RESPONSE` or `INT_YIELD`, and hard negatives receive a named subtype. It
|
|
cannot modify timestamps. The output preserves both the original rule candidate
|
|
and the complete AI decision/provenance.
|
|
|
|
## 3. Human review
|
|
|
|
Run the web workspace on the AI-enriched JSONL:
|
|
|
|
```bash
|
|
python3 review_floor_events.py \
|
|
--dataset-root /path/to/easycom \
|
|
--events reports/easycom_floor_event_candidates/easycom_eot_int_deepseek.jsonl \
|
|
--output reports/easycom_event_reviews/reviewer_1.jsonl
|
|
```
|
|
|
|
Open `http://127.0.0.1:8765`. The page shows a session candidate overview,
|
|
participant activity near the event, video/audio, transcript, structured timing,
|
|
and the AI verdict/confidence/reason. The common path is one operation:
|
|
|
|
- **Accept** keeps the AI verdict/type;
|
|
- **Relabel** expands verdict and subtype controls;
|
|
- **Exclude** requires a concrete data-quality reason;
|
|
- **Unsure / Second review** preserves the event for adjudication.
|
|
|
|
Boundary correction is hidden under an optional disclosure and is limited to
|
|
one 200 ms bin. The original timestamp is always retained. Each save atomically
|
|
rewrites JSONL and CSV while preserving the rule candidate, AI decision, human
|
|
decision, relabel flag, exclusion state, reviewer, and provenance.
|
|
|
|
## Freeze discipline
|
|
|
|
Develop the guide and resolve disagreements only on Sessions 4--12. Freeze
|
|
categories and thresholds before reviewing official test Sessions 1--3. Use two
|
|
independent human review files plus a separate adjudication output for paper-
|
|
quality labels. DeepSeek is pre-label metadata, not an independent reviewer.
|