Commit Graph
15 Commits
Author SHA1 Message Date
Eric Wang 89acf4f617 fix(prompts): accept conversational time confirmations and improve clarification 2026-10-28 17:46:22 +08:00
Eric Wang ddcfe5bd05 feat(evals): add shared case groups and selectable test suites 2026-10-28 13:51:08 +08:00
Eric Wang 3317d652db fix(agent): immediately transfer accidents involving three or more vehicles 2026-10-28 13:36:31 +08:00
Eric Wang 20c5e39e9d fix(agent): reuse collected vehicle counts and avoid repeated questions 2026-10-28 11:30:56 +08:00
Eric Wang e5d2a78886 feat(config): named LLM provider list via env vars, runtime selection
- Config.llm_profiles: LLM_PROVIDERS declares names, each group defined by
  LLM_<NAME>_BASE_URL/_API_KEY/_MODEL (required) and optional _TIMEOUT_SECONDS/
  _MAX_RETRIES/_TEMPERATURE/_EXTRA_BODY; JSON-dict form of LLM_PROVIDERS also
  supported. LLM_PROVIDER selects the active group at runtime; empty/fake =
  offline scripted model
- removed fixed openai/kimi groups (AGENT_LLM_PROVIDER, KIMI_LLM_*)
- validate(): incomplete groups and unknown LLM_PROVIDER fail fast with
  the configured-name list
- OpenAILLMService: http client trust_env from LLM_HTTP_TRUST_ENV (default
  False - local gateways must bypass the Windows system proxy)
- build_llm_service(provider): explicit name or LLM_PROVIDER default
- .env/.env.example: deepseek + kimi groups side by side, switch by
  LLM_PROVIDER=kimi (thinking off via EXTRA_BODY reasoning_effort=none)
- eval probe scripts updated to the new config interface

Verified: 172 unit tests; live calls succeed on both providers
(deepseek 28s, kimi 6.6s, both emitting valid <state> prefixes)
2026-10-22 13:59:18 +08:00
Eric Wang e72d9c9bc0 feat(agent): align with workflow@20261022 - turn_context messages and prompt tails
- merge main (b0ff2e0..c7ca781): workflow/20261022 moves dynamic context
  out of fixed prompts - collection/extraction prompts carry no inline vars;
  reference time and current form travel in the user message <turn_context>
  block; photo/confirm data kept at prompt tails
- prompts: all 8 files regenerated from 20261022 (byte-identity tests
  point at the new export)
- collect: user message = <turn_context>(本轮参考时间) + <user_input>;
  extraction message adds 当前表单; single trusted clock per turn threads
  through reply/extraction/repair
- photo: raw input only (20261022 photo nodes use system_text, no wrapper)
- workflow fake llm: unwraps <user_input> before event parsing
- fields: accident_time normalized to extraction format (YYYY年M月D日H点m分,
  strip zero padding) - fixes form assertion mismatch with padded output
- evals/run.py: trust_env=False - bypass Windows system proxy (registry
  proxy at 192.168.0.4:7897 was intercepting localhost and returning 503)
- verified: 172 unit tests; ALL golden suites pass on real model
  (46/46): collection_skip 12/12, photo_skip 4/4, no_false 2/2,
  button_guidance 12/12, non_motor_vehicle 14/14, time_confirmation 2/2
2026-10-22 12:28:24 +08:00
Eric Wang 9e28f69b3c feat(agent): trusted time evidence and collection prompt optimization
- time_context.py: server-side observation clocks for relative-time
  evidence (anti client-forgery); resolves X分钟前/小时前 against the
  observation timestamp of the turn that produced them, never re-basing
  history on the current clock; flags historical references without a
  reliable baseline
- state: dialogue_timestamps keyed by dialogue message index, persisted
  with checkpoint; welcome seeds index 0
- collect: single clock per turn shared by reply/extraction/repair;
  time_evidence injected into runtime context; reply_validator hook
  rejects time claims lacking evidence (one shared repair budget);
  model still interprets facts and chooses actions
- prompts: collection prompt restructured (shorter, state-bound-to-action
  table, gatekeeper principle, time source rules); extraction prompt
  aligned; workflow@20261021 json updated to match
- evals: prompt_probes / prompt_compare / time_boundary probe scripts
  (supplementary checks, original cases untouched)
- docs: collection-prompt-optimization.md (method + baselines),
  time-boundary-fix.md (problem, design, verification)
- gitignore: editor artifact dirs (.agents/.claude/.vscode, skills-lock)

Verified: 171 unit tests pass (16 new time-context tests)
2026-10-22 09:41:32 +08:00
Eric Wang fb9271efc5 fix(workflow): bind collection states to actions and relax eval wording 2026-10-21 16:51:19 +08:00
Eric Wang 248eb41c78 fix(workflow): align non-motor accident extraction and evaluations 2026-10-21 15:34:11 +08:00
Eric Wang 6b8a548df3 fix(workflow): offer photo button guidance only when needed 2026-10-16 15:58:41 +08:00
Eric Wang 22853f1b5a fix(workflow): skip known accident details and guide off-topic replies 2026-10-16 15:16:16 +08:00
Eric Wang 4787d6eb79 fix(workflow): skip plate confirmation after photos and guide driver verification 2026-10-16 14:21:45 +08:00
Xin Wang f40643dd87 Refine prompts and evaluation cases for accident reporting AI
- Updated prompts to enhance clarity and consistency in user interactions, particularly focusing on safety prioritization and accurate information collection.
- Introduced new YAML evaluation cases to test various user input scenarios, ensuring robust handling of short, ambiguous, or irrelevant responses.
- Enhanced guidelines for transitioning to human assistance, emphasizing strict conditions for triggering state changes based on user responses.
2026-07-30 14:19:57 +08:00
Xin Wang 4e621e2f0c Enhance evaluation runner with transcript handling and new case formats
- Updated the evaluation runner to support live dialogue printing and transcript saving in both Markdown and JSON formats.
- Introduced new data classes for managing transcripts and their components, improving structure and readability.
- Modified existing YAML case files to align with the new direct client mode and updated stage codes.
- Added new test cases for the direct client mode, ensuring comprehensive coverage of user interactions.
2026-07-29 11:13:07 +08:00
Xin Wang e6437ab332 Move workflow evaluations to standalone runner 2026-07-29 10:35:45 +08:00