- Refactor TextInputProcessor to handle immediate and silent text inputs, improving user experience during voice interactions.
- Introduce PassthroughLLMAssistantAggregator to manage LLM responses while preserving context for downstream TTS processing.
- Update event handling for text input and client readiness, ensuring timely updates to the conversation context.
- Modify run_pipeline to integrate new aggregators and streamline message handling, enhancing overall pipeline efficiency.
- Improve message ordering in useVoicePreview to ensure accurate display of chat messages based on timestamps.
- Rename `setSelectedDeviceId` to `selectDevice` in `DebugVoicePanel` and `VoiceSessionControls` for clarity and consistency.
- Update `useVoicePreview` hook to implement the `selectDevice` function, enabling dynamic microphone switching during voice sessions.
- Enhance device selection logic to support real-time audio track replacement without requiring session reconnection.
- Introduce new Xfyun ASR and TTS services, enabling integration with iFlytek's voice recognition and synthesis capabilities.
- Update AssistantConfig model to include interface types for STT and TTS.
- Enhance credential testing to validate Xfyun credentials.
- Modify service factory to create Xfyun services based on configuration.
- Update README with new configuration details for Xfyun integration.
- Add new frontend components for visualizing audio streams and managing user interactions.
- Add audio input selection to DebugVoicePanel, allowing users to choose their microphone device.
- Update useVoicePreview hook to manage available audio inputs and selected device state.
- Enhance device enumeration and selection handling to ensure a seamless user experience during voice interactions.
- Add a new Docker configuration for the UI in launch.json to facilitate development.
- Refactor pipeline.py to integrate a TranscriptProcessor for managing user and assistant transcripts, including event handlers for real-time updates and message handling.
- Update useVoicePreview.ts to establish a data channel for sending and receiving text messages, improving interaction flow.
- Modify AssistantPage.tsx to support displaying chat messages and sending user input, enhancing the user experience during voice interactions.
- Revise DebugTranscriptPanel to dynamically render chat messages with timestamps, improving the visual representation of conversation history.
- Update README to reflect the integration of the DebugVoicePanel with WebSocket support for voice interactions.
- Refactor voice_webrtc.py to improve error handling during WebRTC signaling and include assistant_id in the offer payload.
- Add useVoicePreview hook to manage microphone access and WebRTC connections for real-time voice previews.
- Modify AssistantPage to incorporate new visualizer options and pass assistantId to DebugVoicePanel, enhancing user experience during audio interactions.
- Update API model to include new fields for voice, speed, and language, supporting TTS and ASR configurations.
- Introduce three new audio visualizer components: AuraVisualizer, SpectrumVisualizer, and WaveVisualizer, enhancing the audio interaction experience.
- Replace the deprecated VoiceVisualizer with the new visualizers, ensuring a cohesive visual language across components.
- Update the AssistantPage to support dynamic visualization style switching, improving user engagement during audio interactions.
- Refactor DebugVoicePanel to accommodate the new visualizer props and enhance the overall debugging interface.
Xin Wang