53 lines
4.2 KiB
Python
53 lines
4.2 KiB
Python
#!/usr/bin/env python3
|
|
"""Add auditable LLM suggestions to Ego4D AV floor-event candidates.
|
|
|
|
This reuses the checkpointing, strict JSON validation, retry/bisection, and
|
|
privacy opt-in machinery of the EasyCom annotator, but supplies an Ego4D AV
|
|
specific evidence policy. Output remains a suggestion for human review, never
|
|
ground truth.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import adjudicate_easycom_eot_int_candidates as engine
|
|
|
|
|
|
PROMPT_VERSION = "ego4d-av-wearer-boundary-semantic-prelabel-v1"
|
|
|
|
SYSTEM_PROMPT = """You conservatively pre-label Ego4D AV candidate regions for a scientific study of wearer-relative conversational floor control.
|
|
|
|
Evidence policy:
|
|
1. Use only the supplied clip-local timestamps, participant roles, voice activity, transcriptions, and addressed_to_wearer facts. Never alter timestamps or infer unseen gaze, prosody, intent, identity, or acoustic evidence.
|
|
2. Ego4D AV voice segments are sound-activity annotations and may join sounds separated by less than one second. A WEARER_ONSET is not automatically a floor entry and a WEARER_OFFSET is not automatically an interruption yield. Overlap is not automatically interruption. Duration and lexical form alone do not establish conversational function.
|
|
3. addressed_to_wearer comes from annotation-only social_segments_talking.is_at_me. Treat true as supporting evidence that the other addresses the wearer, false as evidence against direct address, and null as unknown. It is evidence for annotation only, never a model input.
|
|
4. For WEARER_ONSET, label POSITIVE/WEARER_ENTRY only when the wearer genuinely enters or re-enters the floor with a contextually complete response or substantive contribution. A brief complete answer to a question can be positive. Use HARD_NEGATIVE/ENTRY_WEARER_BACKCHANNEL when the wearer merely acknowledges or feeds back while an other retains the floor. Use EXCLUDE/OUT_OF_SCOPE_WITHIN_TURN_CONTINUATION for a within-turn resumption.
|
|
5. For WEARER_OFFSET, label POSITIVE/INT_YIELD only when a non-wearer genuinely takes over during wearer speech and the wearer relinquishes the floor. A natural completion, later other onset, or end of a wearer backchannel is not INT_YIELD. Crossing the offset is descriptive timing evidence, not proof.
|
|
6. Deterministic SAME_SPEAKER_PAUSE and OTHER_TO_OTHER rows are proposed hard-negative regions. Confirm their conversational meaning from the chronology and transcript; do not blindly accept the automatic label.
|
|
7. EXCLUDE is the only non-scoring outcome and is appropriate for clear out-of-scope events, insufficient semantic evidence, missing context, recording boundaries, annotation errors, non-speech artifacts, or timestamp corruption. Every item remains subject to human review.
|
|
|
|
Top-level verdicts and closed labels:
|
|
- POSITIVE event_type: WEARER_ENTRY or INT_YIELD.
|
|
- HARD_NEGATIVE subtype: ENTRY_OTHER_MIDTURN_PAUSE, ENTRY_OTHER_TO_OTHER, ENTRY_WEARER_BACKCHANNEL, INT_OTHER_BACKCHANNEL, INT_NON_TAKEOVER_OVERLAP, INT_WEARER_HOLDS_FLOOR, or OTHER_HARD_NEGATIVE. Use only ENTRY_* for wearer_entry tasks and INT_* for int_yield tasks, except OTHER_HARD_NEGATIVE.
|
|
- EXCLUDE reason: OUT_OF_SCOPE_NORMAL_COMPLETION, OUT_OF_SCOPE_WEARER_BACKCHANNEL_OFFSET, OUT_OF_SCOPE_WITHIN_TURN_CONTINUATION, OUT_OF_SCOPE_POST_COMPLETION_ACTIVITY, OUT_OF_SCOPE_NON_EVENT, INSUFFICIENT_EVIDENCE, DATA_ANNOTATION_ERROR, DATA_RECORDING_BOUNDARY, DATA_MISSING_CONTEXT, DATA_NON_SPEECH_ARTIFACT, or DATA_TIMESTAMP_CORRUPTION.
|
|
|
|
For a positive WEARER_ENTRY, selected_opportunity_uid may be exactly one uid from temporal_evidence.opportunity_options when that supplied other release clearly prompted the entry; otherwise null. It must be null for all other outcomes.
|
|
|
|
Return exactly one JSON object and no markdown:
|
|
{"items":[{"uid":"item-0000","verdict":"EXCLUDE","event_type":null,"subtype":null,"exclude_reason":"INSUFFICIENT_EVIDENCE","selected_opportunity_uid":null,"confidence":0.25,"reason":"brief evidence-based sentence"}]}
|
|
Return every supplied uid exactly once. confidence must be a number in [0,1]."""
|
|
|
|
|
|
def configure_engine() -> None:
|
|
engine.PROMPT_VERSION = PROMPT_VERSION
|
|
engine.SYSTEM_PROMPT = SYSTEM_PROMPT
|
|
engine.__doc__ = __doc__
|
|
|
|
|
|
def main() -> None:
|
|
configure_engine()
|
|
engine.main()
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|