Commit Graph
62 Commits
Author SHA1 Message Date
Xin Wang d003ee77a5 refactor: remove legacy assistant compatibility 2026-08-18 10:07:18 +08:00
Xin Wang abfcd4cd19 fix: support photo client tool in debug preview 2026-08-14 18:28:52 +08:00
Xin Wang 5be21f5ccc feat: organize tools and enable qwen realtime calls 2026-08-14 00:15:00 +08:00
Xin Wang eb111a3890 feat: unify debug runtime timelines 2026-08-13 09:30:43 +08:00
Xin Wang 94a02d7e2b fix: support vision prompt modes 2026-08-12 19:13:04 +08:00
Xin Wang 09b07751b6 feat: add configurable vision focus prompts 2026-08-12 17:29:30 +08:00
Xin Wang 7560f9a525 feat: add dynamic variables and startup result handling in batch run schemas
- Introduced dynamic_variables field in BatchRunCreate schema to support flexible input types.
- Added BatchStartupResult schema to encapsulate startup state, including greeting and auto-confirmed messages.
- Updated BatchRunCaseOut schema to include startup results, enhancing the structure for test run outputs.
- Enhanced test run orchestration to prepare and validate dynamic configurations, ensuring isolated variable management across test cases.
- Implemented changes in the frontend to accommodate new data structures for improved test case execution and result tracking.
2026-08-12 16:55:25 +08:00
Xin Wang 79a239489a feat: enhance call end coordination with state management
- Added support for tracking call end states (pending, speaking, cancelled) in CallEndCoordinator.
- Introduced a callback mechanism for state changes to allow external handling of call end states.
- Updated pipeline and tests to accommodate new state management features, ensuring proper handling of call end scenarios.
2026-08-12 14:25:24 +08:00
Xin Wang d0bb5b3868 Make end call tool can be cancelled 2026-08-12 09:18:40 +08:00
Xin Wang fc026e765e fix: record message confirmations in context 2026-08-11 14:15:10 +08:00
Xin Wang b2e99baa19 Implement webhook 2026-08-07 14:08:49 +08:00
Xin Wang 56a32e81ca Add analysis feature 2026-08-07 10:59:47 +08:00
Xin Wang 4a07abd7fc fix(llm): adapt async tool results for compatible APIs 2026-08-06 10:22:05 +08:00
Xin Wang aaca128f82 fix(workflow): track fixed speech at transport output 2026-08-06 10:21:36 +08:00
Xin Wang 90bb27a377 fix: release input mute after async client tools 2026-08-06 00:12:34 +08:00
Xin Wang 0c0a51da95 Improve image upload in debug drawer 2026-08-05 21:28:51 +08:00
Xin WangandCursor e36ca308b8 feat: support uploaded images in debug voice preview
Allow paste/drag temporary image assets so vision turns work without a live camera frame.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-05 19:52:37 +08:00
Xin Wang 317a0600bc feat: persist photos and enrich conversation history 2026-08-05 16:14:03 +08:00
Xin Wang 0e9a1bb857 fix: cancel pipeline on peer disconnect 2026-08-05 14:34:20 +08:00
Xin Wang 16156028c9 fix: manage WebRTC pipeline shutdown 2026-08-05 13:48:29 +08:00
Xin Wang 74a8be2357 refactor: unify system tools as resources 2026-08-04 17:05:26 +08:00
Xin Wang 731a372df9 feat: add system tools and state updates 2026-08-04 15:27:39 +08:00
Xin Wang 5bf5987fe4 feat: add prompt opening behavior modes 2026-08-04 13:29:21 +08:00
Xin Wang 78bba090a2 feat: separate prompt opening and entry behavior 2026-08-04 12:21:07 +08:00
Xin Wang 1af1cd7fed feat: trigger prompt reply after opening confirmation 2026-08-04 11:49:49 +08:00
Xin Wang 775e4058bc fix: relay confirmation interrupts past muted input 2026-08-04 11:12:34 +08:00
Xin Wang da828aca41 Try to fix confirm and interrupt 2026-08-04 10:19:59 +08:00
Xin Wang a16ecd8e01 fix: interrupt message output before tool result 2026-08-04 09:30:52 +08:00
Xin Wang c3cb410c17 fix: await message confirmation interruption 2026-08-03 17:11:43 +08:00
Xin Wang 67389fc5a1 fix: honor strict workflow agent entry mode 2026-08-03 16:34:26 +08:00
Xin Wang 06719e4801 fix: interrupt message speech on confirmation 2026-08-03 16:30:18 +08:00
Xin Wang 8064bb7152 refactor(workflow): simplify edge configuration 2026-08-03 15:17:01 +08:00
Xin Wang ef434c80b2 fix(workflow): prevent fixed speech continuation 2026-08-03 13:02:31 +08:00
Xin Wang 4c43e167db feat(workflow): enhance message stages and image routing 2026-08-03 12:38:02 +08:00
Xin Wang f3439b21d1 feat: route workflow image inputs natively 2026-08-03 10:55:57 +08:00
Xin Wang f5c36a62aa feat: make workflow messages resumable 2026-08-03 07:54:51 +08:00
Xin Wang 9daa46ed4d feat: add deterministic message interaction stages 2026-08-03 07:10:41 +08:00
Xin Wang 479a516546 fix: isolate startup tools and preview dialogs 2026-08-02 00:07:35 +08:00
Xin Wang 0331f8cd07 feat: add prompt startup actions and shared vision config 2026-08-01 23:31:14 +08:00
Xin Wang b747144ff1 feat: add workflow action outcomes and tracing 2026-08-01 11:21:31 +08:00
Xin Wang ad5ff061bb feat: add session-scoped client tool waits 2026-07-31 23:53:47 +08:00
Xin Wang f155f98e6e feat: add workflow action runtime policies 2026-07-31 23:30:04 +08:00
Xin Wang c2f0f5eb04 feat: add session updates and message dialogs 2026-07-31 00:01:01 +08:00
Xin Wang 913435785e feat: add configurable client tools and photo input 2026-07-30 19:06:03 +08:00
Xin Wang 510a277b5a feat(qwen-audio): enhance user transcript handling in QwenAudioRealtimeService
- Added support for managing user transcripts, including pending states and timestamps.
- Implemented methods to handle user transcript completion and deferred assistant messages.
- Updated event handling to ensure user transcripts are emitted before assistant responses.
- Enhanced tests to verify the correct order of transcript and assistant message emissions during user interactions.
2026-07-29 17:01:02 +08:00
Xin Wang 774825593d feat(qwen-audio): add Alibaba Cloud Qwen-Audio Realtime service integration
- Introduced Qwen-Audio Realtime service for speech-to-speech processing in Pipecat.
- Updated interface catalog to include Qwen-Audio Realtime capabilities.
- Enhanced model resource testing to support new service.
- Added configuration options for audio sample rates and turn detection modes.
- Updated documentation to reflect integration details and usage instructions.
2026-07-21 15:57:43 +08:00
Xin Wang f027ed99b7 feat(mcp): enhance MCP transport support and add server management UI
- Extend McpTransport to support "sse" in schemas.
- Refactor McpToolClient to handle both "streamable_http" and "sse" transports.
- Introduce McpServerDialog for managing MCP server configurations, including transport settings and tool synchronization.
- Replace McpServersSection with the new dialog component for improved server management.
- Add tests for MCP transport handling and server dialog functionality.
2026-07-19 12:04:11 +08:00
Xin Wang cd319d27b9 merge: add workflow vision configuration 2026-07-18 00:10:28 +08:00
Xin Wang 28c6380d8a feat(workflow): add per-agent vision configuration 2026-07-18 00:00:28 +08:00
Xin Wang e39bb48ba8 feat: add MCP tool integration 2026-07-18 00:00:06 +08:00