fix(backends): make streamed replies and session state consistent

- Validate stage prefixes before forwarding model tokens. Invalid prefixes
  may use the existing repair budget; once output starts, validation failure
  aborts the turn without appending a regenerated reply. Bound the token
  queue and close inner streams when consumption stops.
- Replace delete-time lock eviction with reference-counted session locks
  covering holders and waiters. Serialize set/get with chat/stream/delete
  and release cancelled waiters without replacing a live lock.
- Store person synonyms under workflow-native driver keys and expose
  consistent alias views to single/group reads and graph prompt injection.
- Separate committed field patches from public formUpdate visibility.
  Commit callbacks keep repository fields synchronized even when updates
  are hidden or the consumer closes after graph completion.
- Disable OpenAI SDK automatic retries. Own bounded retry attempts in the
  application, use 1/2/4 second backoff, retry streams only before text is
  emitted, enforce attempt timeouts, and close upstream/client resources.
- Translate FastGPT SDK max_retries (total attempts) from the configured
  extra-retry count, and verify the real SDK loop uses four attempts.
- Document the per-LLM budget and streaming failure semantics without
  claiming the whole multi-call HTTP turn has the same timeout.

Validation: 267 offline tests passed, including actual SDK requests through
MockTransport, stream repair/abort, timeout/close, lock races, aliases and
formUpdate visibility; existing four dependency deprecation warnings.
This commit is contained in:
Eric Wang
2026-10-23 15:54:51 +08:00
parent 952bffe6f0
commit 24e2cfbe10
18 changed files with 852 additions and 316 deletions
+41 -7
View File
@@ -1,11 +1,8 @@
"""后端超时/重试 SLA 对齐契约测试。
"""超时/重试配置与 SDK 适配契约。
背景(可替代性清单):FastGPT 客户端为 60s/3 次,LangGraph LLM 侧
曾是 30s/2 次,客户端感知到的超时行为不等价。本测试把两端常量
钉在一起,防止再次漂移。
注:退避策略有意不同——FastGPT 固定 1s,LLM 指数退避(限流场景
更友好);对齐的是"单次调用总超时"与"最大重试次数"两个 SLA 参数。
LLM max_retries 表示额外重试次数;FastGPT SDK 同名参数实际表示
总尝试次数,需要 +1 转换。退避统一为 1、2、4 秒。
超时分别约束一次 LLM 尝试与 FastGPT HTTP SDK,不代表整轮总期限。
"""
import inspect
@@ -40,3 +37,40 @@ def test_named_profile_defaults_match_fastgpt_client(monkeypatch):
profile = Config.llm_profile("probe")
assert profile["timeout_seconds"] == FASTGPT_TIMEOUT_SECONDS
assert profile["max_retries"] == FASTGPT_MAX_RETRIES
import asyncio
from unittest.mock import AsyncMock
import httpx
import pytest
from fastapi import FastAPI
from fastgpt_client.exceptions import APIError
from src.core import fastgpt_client as wiring
from src.core.config import Config
@pytest.mark.asyncio
async def test_fastgpt_sdk_uses_same_attempt_count_and_backoff(monkeypatch):
monkeypatch.setattr(Config, "validate", lambda: None)
monkeypatch.setattr(Config, "AGENT_BACKEND", "fastgpt")
monkeypatch.setattr(Config, "FASTGPT_BASE_URL", "http://audit")
monkeypatch.setattr(Config, "FASTGPT_API_KEY", "audit")
sleep = AsyncMock()
monkeypatch.setattr(asyncio, "sleep", sleep)
async with wiring.lifespan(FastAPI()):
client = wiring.get_fastgpt_client()
request = AsyncMock(return_value=httpx.Response(503))
with pytest.raises(APIError):
await client._retry_request_async(request, "audit")
assert request.await_count == 4
assert [call.args[0] for call in sleep.await_args_list] == [1, 2, 4]
@pytest.mark.asyncio
async def test_llm_backoff_uses_same_delays(monkeypatch):
sleep = AsyncMock()
monkeypatch.setattr(asyncio, "sleep", sleep)
llm = OpenAILLMService(model="audit", api_key="audit")
for attempt in range(3):
await llm._retry_delay(attempt)
assert [call.args[0] for call in sleep.await_args_list] == [1, 2, 4]