feat: add dynamic variables and startup result handling in batch run schemas
- Introduced dynamic_variables field in BatchRunCreate schema to support flexible input types. - Added BatchStartupResult schema to encapsulate startup state, including greeting and auto-confirmed messages. - Updated BatchRunCaseOut schema to include startup results, enhancing the structure for test run outputs. - Enhanced test run orchestration to prepare and validate dynamic configurations, ensuring isolated variable management across test cases. - Implemented changes in the frontend to accommodate new data structures for improved test case execution and result tracking.
This commit is contained in:
@@ -59,11 +59,23 @@ export type BatchTurnResult = {
|
||||
evaluations: BatchEvaluationResult[];
|
||||
};
|
||||
|
||||
export type BatchStartupResult = {
|
||||
completed: boolean;
|
||||
greeting: string;
|
||||
assistantOutputs: string[];
|
||||
autoConfirmedMessages: Array<{
|
||||
title?: string;
|
||||
message?: string;
|
||||
actions?: Array<{ id?: string; label?: string }>;
|
||||
}>;
|
||||
};
|
||||
|
||||
export type BatchRunCase = {
|
||||
id: string;
|
||||
name: string;
|
||||
status: BatchCaseStatus;
|
||||
turns: BatchTurnResult[];
|
||||
startup?: BatchStartupResult;
|
||||
overallCriteria: BatchEvaluationResult[];
|
||||
attemptCount: number;
|
||||
maxAttempts: number;
|
||||
|
||||
Reference in New Issue
Block a user