fix(backends): make streamed replies and session state consistent
- Validate stage prefixes before forwarding model tokens. Invalid prefixes may use the existing repair budget; once output starts, validation failure aborts the turn without appending a regenerated reply. Bound the token queue and close inner streams when consumption stops. - Replace delete-time lock eviction with reference-counted session locks covering holders and waiters. Serialize set/get with chat/stream/delete and release cancelled waiters without replacing a live lock. - Store person synonyms under workflow-native driver keys and expose consistent alias views to single/group reads and graph prompt injection. - Separate committed field patches from public formUpdate visibility. Commit callbacks keep repository fields synchronized even when updates are hidden or the consumer closes after graph completion. - Disable OpenAI SDK automatic retries. Own bounded retry attempts in the application, use 1/2/4 second backoff, retry streams only before text is emitted, enforce attempt timeouts, and close upstream/client resources. - Translate FastGPT SDK max_retries (total attempts) from the configured extra-retry count, and verify the real SDK loop uses four attempts. - Document the per-LLM budget and streaming failure semantics without claiming the whole multi-call HTTP turn has the same timeout. Validation: 267 offline tests passed, including actual SDK requests through MockTransport, stream repair/abort, timeout/close, lock races, aliases and formUpdate visibility; existing four dependency deprecation warnings.
This commit is contained in:
@@ -16,11 +16,11 @@ _chat_backend: ChatBackend | None = None
|
||||
_langgraph_graph = None
|
||||
_app_service: AccidentAppService | None = None
|
||||
|
||||
# 后端超时/重试 SLA(两后端对齐;契约测试防止再漂移)。
|
||||
# FastGPT 侧:客户端每次请求总超时 + 重试次数;
|
||||
# LangGraph 侧:LLM 每次调用总超时 + 重试次数(退避为指数式,
|
||||
# 与 FastGPT 固定 1s 不同——总尝试次数与单次超时可比,见
|
||||
# test/backends/test_sla_alignment.py)。
|
||||
# 两后端缺省 60 秒、最多额外重试 3 次,1/2/4 秒退避。
|
||||
# FastGPT SDK 的 max_retries 实为总尝试数,注入时转换为 1 + 重试次数。
|
||||
# LLM 关闭 SDK 自动重试;流式发出文本后不再重试。
|
||||
# LLM 预算按单次模型调用计算,FastGPT timeout 为 HTTP SDK 超时,
|
||||
# 均不承诺多次模型调用的整个聊天回合具有相同总期限。
|
||||
FASTGPT_TIMEOUT_SECONDS = 60.0
|
||||
FASTGPT_MAX_RETRIES = 3
|
||||
|
||||
@@ -30,6 +30,7 @@ async def lifespan(app: FastAPI):
|
||||
"""Manage chat backend resource lifecycle."""
|
||||
global _chat_backend, _fastgpt_client
|
||||
Config.validate()
|
||||
llm = None
|
||||
|
||||
if Config.AGENT_BACKEND == "langgraph":
|
||||
# 作业 2:LangGraph fake 模式不要求 FastGPT 密钥。
|
||||
@@ -38,7 +39,8 @@ async def lifespan(app: FastAPI):
|
||||
|
||||
global _langgraph_graph, _app_service
|
||||
_langgraph_graph = build_graph()
|
||||
_chat_backend = LangGraphBackend(_langgraph_graph, llm=build_llm_service())
|
||||
llm = build_llm_service()
|
||||
_chat_backend = LangGraphBackend(_langgraph_graph, llm=llm)
|
||||
# 作业 6:应用服务 + 内存 repository(作业 7 换 PostgreSQL)。
|
||||
_app_service = AccidentAppService(
|
||||
_chat_backend, InMemorySessionRepository()
|
||||
@@ -48,7 +50,7 @@ async def lifespan(app: FastAPI):
|
||||
api_key=Config.FASTGPT_API_KEY,
|
||||
base_url=Config.FASTGPT_BASE_URL,
|
||||
timeout=FASTGPT_TIMEOUT_SECONDS,
|
||||
max_retries=FASTGPT_MAX_RETRIES,
|
||||
max_retries=FASTGPT_MAX_RETRIES + 1,
|
||||
retry_delay=1.0,
|
||||
enable_logging=Config.DEBUG,
|
||||
)
|
||||
@@ -61,6 +63,8 @@ async def lifespan(app: FastAPI):
|
||||
_chat_backend = None
|
||||
_langgraph_graph = None
|
||||
_app_service = None
|
||||
if llm is not None and hasattr(llm, "aclose"):
|
||||
await llm.aclose()
|
||||
if _fastgpt_client:
|
||||
await _fastgpt_client.__aexit__(None, None, None)
|
||||
_fastgpt_client = None
|
||||
|
||||
Reference in New Issue
Block a user