Replaces the 'complete answer sliced into chunks' stopgap with token-
level streaming. Graph nodes are untouched: they still consume the
batched LLMService.complete interface and validate full replies.
- agent/llm: OpenAILLMService.stream_tokens() iterates the provider
stream with the same message construction, temperature and extra_body
forwarding as complete(); identical error classification (401 ->
LLMAuthError, 429 -> LLMRateLimitError, timeout via asyncio.timeout
around the whole iteration); deliberately no retries (tokens already
emitted cannot be recalled)
- backends/langgraph: _TokenPassThroughLLM wraps the injected LLM for
stream turns - dialogue calls push tokens to a queue bridge while
accumulating the full text for node validation; calls marked with the
【字段提取】 marker take the silent inner.complete path so extraction
JSON never leaks into the client stream; LLMs without stream_tokens
fall back to single-chunk push (fake/test compatibility)
- stream(): graph runs in a background task; the generator yields
TextDelta per token and FormUpdate after the turn; graph-task errors
re-raise with the existing ChatBackend error mapping; generator close
(client disconnect) cancels the graph task and the in-flight LLM call
- fixed-answer turns with no LLM call (welcome) fall back to emitting
the full response text, preserving previous behavior
- endpoint untouched: its buffering parser already handles arbitrary
chunk boundaries (state tags split across tokens, stage-first order)
Verified: 241 unit tests (11 new streaming tests: token incrementality,
extraction bypass, FormUpdate ordering, error mapping on both paths,
disconnect cancellation, fallback chunking, SSE protocol end-to-end
with per-token text_delta events)
Eric Wang
952bffe6f0