- Validate stage prefixes before forwarding model tokens. Invalid prefixes
may use the existing repair budget; once output starts, validation failure
aborts the turn without appending a regenerated reply. Bound the token
queue and close inner streams when consumption stops.
- Replace delete-time lock eviction with reference-counted session locks
covering holders and waiters. Serialize set/get with chat/stream/delete
and release cancelled waiters without replacing a live lock.
- Store person synonyms under workflow-native driver keys and expose
consistent alias views to single/group reads and graph prompt injection.
- Separate committed field patches from public formUpdate visibility.
Commit callbacks keep repository fields synchronized even when updates
are hidden or the consumer closes after graph completion.
- Disable OpenAI SDK automatic retries. Own bounded retry attempts in the
application, use 1/2/4 second backoff, retry streams only before text is
emitted, enforce attempt timeouts, and close upstream/client resources.
- Translate FastGPT SDK max_retries (total attempts) from the configured
extra-retry count, and verify the real SDK loop uses four attempts.
- Document the per-LLM budget and streaming failure semantics without
claiming the whole multi-call HTTP turn has the same timeout.
Validation: 267 offline tests passed, including actual SDK requests through
MockTransport, stream repair/abort, timeout/close, lock races, aliases and
formUpdate visibility; existing four dependency deprecation warnings.
Replaces the 'complete answer sliced into chunks' stopgap with token-
level streaming. Graph nodes are untouched: they still consume the
batched LLMService.complete interface and validate full replies.
- agent/llm: OpenAILLMService.stream_tokens() iterates the provider
stream with the same message construction, temperature and extra_body
forwarding as complete(); identical error classification (401 ->
LLMAuthError, 429 -> LLMRateLimitError, timeout via asyncio.timeout
around the whole iteration); deliberately no retries (tokens already
emitted cannot be recalled)
- backends/langgraph: _TokenPassThroughLLM wraps the injected LLM for
stream turns - dialogue calls push tokens to a queue bridge while
accumulating the full text for node validation; calls marked with the
【字段提取】 marker take the silent inner.complete path so extraction
JSON never leaks into the client stream; LLMs without stream_tokens
fall back to single-chunk push (fake/test compatibility)
- stream(): graph runs in a background task; the generator yields
TextDelta per token and FormUpdate after the turn; graph-task errors
re-raise with the existing ChatBackend error mapping; generator close
(client disconnect) cancels the graph task and the in-flight LLM call
- fixed-answer turns with no LLM call (welcome) fall back to emitting
the full response text, preserving previous behavior
- endpoint untouched: its buffering parser already handles arbitrary
chunk boundaries (state tags split across tokens, stage-first order)
Verified: 241 unit tests (11 new streaming tests: token incrementality,
extraction bypass, FormUpdate ordering, error mapping on both paths,
disconnect cancellation, fallback chunking, SSE protocol end-to-end
with per-token text_delta events)
Eric Wang