Delegate wearer boundary semantics to review

This commit is contained in:
Xin Wang
2026-09-15 02:12:18 +08:00
parent 71dfd8808a
commit 3a127a3f9f
12 changed files with 596 additions and 404 deletions
+46 -56
View File
@@ -6,7 +6,7 @@ ground truth.
```text
EasyCom participant speech intervals
-> floor-episode grouping + rule candidates + structured timing evidence
-> all wearer boundaries + independent rule regions + structured factual evidence
-> DeepSeek semantic pre-label (optional but recommended)
-> human Accept / Relabel / Exclude
```
@@ -25,47 +25,37 @@ python3 prepare_easycom_eot_int_review.py \
The generator emits two independent files:
- `easycom_eot_int_candidates.jsonl` (legacy-compatible filename) contains every
floor-level wearer `0->1` behavior anchor (`WEARER_ONSET`,
`WEARER_ENTRY_CANDIDATE`), only episode-ending wearer
`1->0` boundaries where a new non-wearer onset satisfies
`wearer_floor_start < other_onset < wearer_offset` and the same activity
crosses the offset (`WEARER_OFFSET`, `YIELD_CANDIDATE`), and hard-negative
raw wearer `0->1` boundary (`WEARER_ONSET`, `WEARER_ENTRY_CANDIDATE`), every
raw wearer `1->0` boundary (`WEARER_OFFSET`, `YIELD_CANDIDATE`), and hard-negative
regions that need no wearer transition:
`SAME_SPEAKER_PAUSE`, `OTHER_TO_OTHER`, `OTHER_BACKCHANNEL`, and
`NON_TAKEOVER_OVERLAP`;
- `easycom_eot_int_controls.jsonl` contains short within-turn raw onsets
(`WEARER_RESUME`) and wearer offsets that fail
the strict INT ordering/crossing prerequisite (`WEARER_OFFSET_CONTROL`). They
are retained for audits, but are not sent to DeepSeek or placed in the
primary human-review queue.
- `easycom_eot_int_controls.jsonl` is retained for file-format compatibility
and legacy inputs. The current boundary protocol does not deterministically
demote wearer onsets or offsets into this file.
A raw wearer onset is merged into its preceding floor episode only when its
preceding silence is at most 0.8 seconds and no plausible sustained non-wearer
takeover occurs in that gap. Short feedback-like lexical or nonverbal activity
is recorded but does not by itself split the episode. A non-feedback other
contribution that occupies at least `continuation_min_seconds` of the gap or
continues across the resumed onset conservatively preserves a new wearer-entry
candidate. Every decision is stored in `temporal_evidence`, including the raw
gap, feedback IDs, competing IDs, threshold, and `floor_episode_merged` flag.
No short-gap merge or takeover heuristic controls wearer-boundary admission.
For each raw onset, `temporal_evidence` stores the previous wearer offset,
silence duration, every other-speaker interval intersecting that gap, interval
coverage, whether it continues across the onset, transcript, and
`Target_of_Speech`. DeepSeek and the human reviewer decide whether the onset is
a genuine entry, within-turn resumption, backchannel, or excluded non-event.
Entry construction retains every floor-level wearer onset, then searches backward
for preceding non-wearer releases as optional context. A release is an admissible option only when the
wearer stays silent and no new non-wearer onset or already-active other floor
invalidates the interval before the wearer onset. Silence length alone does not
invalidate an option. Each wearer onset produces exactly one primary row. A
missing release does not demote an entry to a control: an entry may occur from
silence or while another speaker is active. `entry_context_timing` records
`AFTER_OTHER_RELEASE`, `OTHER_ACTIVE`, or `FROM_SILENCE_OR_UNRESOLVED` as
metadata. When present, `opportunity_options` contains the only release anchors
that DeepSeek or a human may optionally select; neither may invent a timestamp.
Entry construction searches backward for nearby non-wearer releases as optional
context. Every option carries factual flags for intervening wearer activity,
intervening other onsets, and activity at the wearer onset; the generator does
not declare one a valid conversational cause. Each wearer onset produces exactly
one primary row. When present, `opportunity_options` contains the only release
anchors that DeepSeek or a human may optionally select; neither may invent a
timestamp.
Floor-level wearer onsets and strictly qualified offsets carry the rule-stage value
`candidate`; timing alone is not a semantic positive label. Deterministic rows
outside the primary queue carry `control`, never a semantic verdict. The INT prerequisite is
`wearer_floor_start < other.start < wearer_offset < other.end`, with a
configurable one-frame (50 ms) tolerance only at the offset boundary.
Pre-existing other activity and raw offsets inside a merged wearer episode stay
in the controls file. Hard-negative scans are rule hints and still require AI
All wearer onsets and offsets carry the rule-stage value `candidate`; timing
alone is not a semantic positive label. For each offset, the JSON records the
wearer activity-run start, nearby other intervals, pre-existing and new-other
flags, activity immediately before/after the boundary, strict and configurable
50-ms-tolerant crossing, overlap duration, and subsequent speech onsets.
`wearer_start < other.start < wearer_offset < other.end` is evidence rather than
an INT prerequisite. Hard-negative scans are rule hints and still require AI
and human review. The JSONL retains candidate source, original event time,
participant-level activity, raw durations/gaps/overlap evidence, transcript,
and `Target_of_Speech` when present. `Target_of_Speech` is annotation assistance
@@ -80,7 +70,6 @@ All heuristic durations are configurable rather than frozen paper rules:
--short-vocalization-max-seconds
--continuation-min-seconds
--int-offset-tolerance-seconds
--wearer-resume-merge-max-seconds
```
Use `--max-candidates 1` for a one-event smoke test. `summary.json` records the
@@ -90,7 +79,7 @@ timeline; it is not sent to DeepSeek.
## 2. Add DeepSeek pre-labels
Prompt v11 first reconstructs the exact participant-resolved chronology and
Prompt v14 first reconstructs the exact participant-resolved chronology and
then uses transcript content and `Target_of_Speech` to determine conversational
function. It treats the EasyCom transcript marker `[L]` as laughter-only evidence,
not as a lexical floor take. Wearer laughter during another participant's
@@ -98,8 +87,8 @@ retained turn is directed toward `ENTRY_WEARER_BACKCHANNEL`; non-wearer laughter
overlapping a retained wearer turn is directed toward `INT_OTHER_BACKCHANNEL`.
Ambiguous laughter falls back to `EXCLUDE`, and `[L]` alone is neither evidence
of a floor take nor evidence of a visible smile. Preceding release options are
optional entry metadata, whereas strict INT ordering remains a deterministic
retrieval prerequisite. A positive `WEARER_ENTRY` may optionally select one
optional entry metadata. No entry/within-turn or INT/takeover semantic decision
is made before the prompt. A positive `WEARER_ENTRY` may optionally select one
supplied release; entries from silence or during other activity use `null`.
`needs_human_review` is not requested from DeepSeek: every pre-label enters the
@@ -135,10 +124,14 @@ transcript together. It returns one of `POSITIVE`, `HARD_NEGATIVE`, or
`WEARER_ENTRY` or `INT_YIELD`, and hard negatives receive a named subtype. It
cannot modify timestamps. The output preserves the original rule candidate,
the selected preceding-release UID when applicable, and the complete AI
decision/provenance. `EXCLUDE` is the sole non-scoring outcome. It covers
out-of-scope non-events (including a normal wearer completion that merely
coincides with feedback), insufficient evidence, unresolvable data quality, or
disagreement between the two model passes.
decision/provenance. `EXCLUDE` is the sole non-scoring outcome, but it is never
an untyped bucket. Every AI and human exclusion stores one closed-vocabulary
`exclude_reason` and the derived `exclude_reason_group`. `OUT_OF_SCOPE_*` means
the annotations are valid but the behavior is outside the scored target;
`INSUFFICIENT_EVIDENCE` means the supplied valid material cannot resolve
conversational function; and `DATA_*` identifies unusable annotation,
recording, context, artifact, or timestamp quality. Model-pass disagreement is
recorded as `INSUFFICIENT_EVIDENCE`, never as data corruption.
Hard negatives use behavior-specific subtypes. ENTRY distinguishes a same-other
mid-turn pause, a different-other transition, and a wearer backchannel that
@@ -156,8 +149,9 @@ all candidate/pass work, including the current pass, batch, percentage, and
elapsed time. Redirected logs emit periodic progress lines instead of carriage
return animation.
As defense in depth, the adjudicator also detects and skips non-crossing
`WEARER_OFFSET` rows from older v2 candidate files before making any API call.
Every current `WEARER_ONSET` and `WEARER_OFFSET` row is sent for semantic
adjudication regardless of its timing pattern. Explicit control rows from older
protocol files remain excluded unless controls are intentionally reviewed.
## 3. Human review
@@ -192,22 +186,18 @@ decision, relabel flag, exclusion state, confidence, and provenance. The current
UI does not ask for a reviewer name; the export retains an empty `reviewer` field
only for compatibility with older review files.
The primary review queue also filters legacy non-crossing `WEARER_OFFSET` rows
and explicit `WEARER_RESUME` controls. To audit the controls intentionally,
The primary review queue filters only explicit legacy control rows. To audit
those controls intentionally,
pass `--events .../easycom_eot_int_controls.jsonl --include-controls` and use a
separate review output file. The old `--include-offset-controls` spelling
remains an alias.
Protocol wording for the paper appendix:
> Raw wearer speech onsets are first grouped into floor-level speech episodes
> by merging short within-turn resumptions that are not separated by a
> competing floor takeover. Every resulting floor-level 0→1 transition is
> proposed as a wearer-entry candidate; a preceding non-wearer release is
> retained as context rather than required for positive status.
> The original speech intervals are preserved unchanged for dense forecasting
> and auditability; this grouping is used only for behavioral event annotation.
> Every observed wearer speech onset and offset is retained as a semantic review
> candidate. Deterministic construction records only participant-resolved timing,
> transcript, and target-of-speech facts; it does not infer within-turn
> resumption, floor entry, interruption, or takeover.
## Freeze discipline