feat: add dynamic variables and startup result handling in batch run schemas

- Introduced dynamic_variables field in BatchRunCreate schema to support flexible input types.
- Added BatchStartupResult schema to encapsulate startup state, including greeting and auto-confirmed messages.
- Updated BatchRunCaseOut schema to include startup results, enhancing the structure for test run outputs.
- Enhanced test run orchestration to prepare and validate dynamic configurations, ensuring isolated variable management across test cases.
- Implemented changes in the frontend to accommodate new data structures for improved test case execution and result tracking.
This commit is contained in:
Xin Wang
2026-08-12 16:55:25 +08:00
parent 79a239489a
commit 7560f9a525
12 changed files with 1464 additions and 1004 deletions
+12
View File
@@ -59,11 +59,23 @@ export type BatchTurnResult = {
evaluations: BatchEvaluationResult[];
};
export type BatchStartupResult = {
completed: boolean;
greeting: string;
assistantOutputs: string[];
autoConfirmedMessages: Array<{
title?: string;
message?: string;
actions?: Array<{ id?: string; label?: string }>;
}>;
};
export type BatchRunCase = {
id: string;
name: string;
status: BatchCaseStatus;
turns: BatchTurnResult[];
startup?: BatchStartupResult;
overallCriteria: BatchEvaluationResult[];
attemptCount: number;
maxAttempts: number;