Replaces the 'complete answer sliced into chunks' stopgap with token-
level streaming. Graph nodes are untouched: they still consume the
batched LLMService.complete interface and validate full replies.
- agent/llm: OpenAILLMService.stream_tokens() iterates the provider
stream with the same message construction, temperature and extra_body
forwarding as complete(); identical error classification (401 ->
LLMAuthError, 429 -> LLMRateLimitError, timeout via asyncio.timeout
around the whole iteration); deliberately no retries (tokens already
emitted cannot be recalled)
- backends/langgraph: _TokenPassThroughLLM wraps the injected LLM for
stream turns - dialogue calls push tokens to a queue bridge while
accumulating the full text for node validation; calls marked with the
【字段提取】 marker take the silent inner.complete path so extraction
JSON never leaks into the client stream; LLMs without stream_tokens
fall back to single-chunk push (fake/test compatibility)
- stream(): graph runs in a background task; the generator yields
TextDelta per token and FormUpdate after the turn; graph-task errors
re-raise with the existing ChatBackend error mapping; generator close
(client disconnect) cancels the graph task and the in-flight LLM call
- fixed-answer turns with no LLM call (welcome) fall back to emitting
the full response text, preserving previous behavior
- endpoint untouched: its buffering parser already handles arbitrary
chunk boundaries (state tags split across tokens, stage-first order)
Verified: 241 unit tests (11 new streaming tests: token incrementality,
extraction bypass, FormUpdate ordering, error mapping on both paths,
disconnect cancellation, fallback chunking, SSE protocol end-to-end
with per-token text_delta events)
Replaces the unequal SLA (LLM 30s/2 retries vs FastGPT client 60s/3)
that made the LangGraph backend look less stable under gray release.
- core/fastgpt_client: FASTGPT_TIMEOUT_SECONDS / FASTGPT_MAX_RETRIES
module constants (were inline literals in lifespan)
- agent/llm: OpenAILLMService defaults 60.0s / 3 retries
- core/config: named-profile fallback defaults 60/3 (prefixed and
JSON-dict forms)
- contract test test/backends/test_sla_alignment.py pins both sides to
the same constants so the SLA cannot drift apart again
- retry backoff intentionally differs (FastGPT fixed 1s vs LLM
exponential) and is documented; aligned parameters are per-call
timeout and max retry count
Verified: 215 unit tests (3 new SLA contract tests)
- prompts/collection.md: versioned collection system prompt extracted
from workflow node yQaBxfrWjbRr with source/version/porting notes
- llm.py: LLMService protocol; FakeLLMService (scripted, records
calls); OpenAILLMService (async, wait_for timeout with cancellation,
retry only on timeout/429/connection, errors normalized)
- collect.py: run_collect reference impl - one LLM call produces
state prefix + body, controlled repair (max 1 retry) for missing
prefix or illegal stage proposal, whitelist guard, CollectReplyError
without raw model content; dialogue history + confirmed fields
injected into prompt so the model skips known info (作业4-5)
- fields.py / clock.py: three-state field candidates (unknown/
candidate/confirmed), per-type validation (count not inferred from
description, future time rejected), injectable Clock
- graph.py: welcome node - first message returns fixed 1001 greeting
without LLM (mirrors workflow is_first_message -> Assigned Reply#3),
route empty stage to welcome
- endpoints/delete_session: backend switch - langgraph clears graph
checkpoint, fastgpt path unchanged
- config: AGENT_LLM_PROVIDER fake|openai with validation
- untrack .env (secrets), move template to root .env.example
Verified:
- pytest 103 passed; smoke turns 1/2/1
- Real LLM e2e: POST /chat multi-turn context works
- Golden eval collection_skip_known_info: 12/12 cases pass
- Golden eval no_false: 8/10 cases pass; 4 confirm/photo cases fail
at final step - model routes on short reply '对' (prompt-adherence
gap, photo/confirm stages are assignment 5 scope); recorded in
evals/results/20261020-144856
Known limits: field extraction from model reply not wired (stub);
photo/confirm nodes still scaffold; clarify counting is approximate.
- src/backends/langgraph.py: LangGraphBackend implements ChatBackend;
complete() runs one graph turn per session (thread_id=sessionId) and
returns state-prefixed content; stream() emits the full state prefix
before body fragments (full-answer slicing, not yet real incremental
output - assignment 8)
- src/core/config.py: AGENT_BACKEND=fastgpt|langgraph, langgraph mode
no longer requires FastGPT credentials at startup
- src/core/fastgpt_client.py: lifespan builds the backend by switch
- test/backends/test_langgraph_backend.py: adapter acceptance tests
Verified:
- pytest: 48 passed
- AGENT_BACKEND=langgraph uvicorn: POST /chat -> 1002;
browser_addon -> browser_addon.1002; SSE = stage_code, text_delta,
single done
Known limit: stream slices a complete answer; no formUpdate deltas yet
(assignment 6).
- Replaced direct FastGPT client usage with a backend-neutral chat interface, allowing for improved flexibility and maintainability.
- Introduced a new `ChatBackend` protocol and implemented `FastGPTBackend` to handle chat operations.
- Updated the chat endpoint to utilize the new backend structure, enhancing the handling of chat requests and responses.
- Added comprehensive tests to ensure compatibility and functionality of the new backend integration.
Eric Wang