- Add new models for `KnowledgeDocument` and `KnowledgeChunk` to manage document ingestion and chunking.
- Implement S3-compatible storage integration for knowledge documents, allowing for file uploads and retrieval.
- Introduce API endpoints for managing knowledge bases and documents, including creation, deletion, and searching.
- Update frontend components to support knowledge base configuration and document management, improving user interaction.
- Enhance backend services for knowledge processing and retrieval, ensuring robust handling of document statuses and errors.
- Introduce a new `turnConfig` field in `AssistantConfig` and `Assistant` models to manage user interaction settings.
- Implement `TurnConfig`, `BargeInConfig`, `VadConfig`, and `TurnDetectionConfig` schemas to define turn management strategies.
- Update the backend to handle turn configuration in the database and during assistant operations.
- Enhance frontend components with a `TurnConfigEditor` for configuring turn settings, including VAD and barge-in strategies.
- Modify existing pages to integrate turn configuration, improving user experience and interaction capabilities.
- Introduce new fields `dify_api_url` and `dify_api_key` in `AssistantConfig` for Dify API integration.
- Update `requirements.txt` to include `dify-client-python` for Dify SDK support.
- Modify `config_resolver` to handle Dify connection information.
- Add a new `globalNode` type in workflow specifications to provide unified settings across workflows.
- Enhance node specifications with additional constraints and default values for better configuration management.
- Update frontend components to support the new `globalNode` type and its properties, improving workflow editor functionality.
- Add a new guideline emphasizing the importance of articulating thoughts in simple language before coding.
- Maintain existing focus on building with MVP principles for better code maintainability and readability.
- Create a new JSON schema file for documentation settings.
- Add core features documentation detailing assistant configuration and model integration.
- Include an index page introducing the AI 视频助手 and its platform components.
- Document integration methods for model services and external agents.
- Provide a quickstart guide for setting up the local development environment using Docker.
- Add light and dark logo SVG files for branding purposes.
- Introduce a new state variable `callEnded` in `useVoicePreview` to manage call termination more effectively.
- Modify `MobileCallPage` to adjust the `inCall` condition based on `callEnded`, ensuring accurate UI updates during and after calls.
- Update rendering logic to display appropriate messages based on call status, enhancing user feedback during call transitions.
- Refactor button visibility to only show when in an active call, improving user interface clarity.
- Update the effect to reset the view to "camera" when the call status is "idle" or "failed", improving user experience during call interruptions.
- Modify the rendering condition for the MobileCallTranscript to only display when inCall is true, ensuring chat messages are shown appropriately during active calls.
- Implement a new API endpoint for duplicating tools in the backend.
- Add a duplicate_tool function in ComponentsToolsPage to handle tool duplication requests.
- Update the UI to include a dropdown menu option for duplicating tools, improving user experience.
- Refactor the remove function to accept a resource object instead of an ID for better clarity and maintainability.
- Implement a new API endpoint for deleting conversations in the backend.
- Enhance the HistoryPage component to include a dropdown menu for conversation actions, allowing users to delete conversations.
- Update the pagination logic to handle conversation removal and improve user experience during deletion.
- Adjust the loading state and error handling for the delete operation, ensuring smooth interaction.
- Introduce new database models for conversation sessions, messages, and artifacts to support conversation history tracking.
- Implement API routes for listing conversations and retrieving detailed conversation data, enhancing user interaction with historical records.
- Add a conversation recorder service to persist conversation messages in real-time without disrupting ongoing calls.
- Update the frontend to display conversation history, including filtering and sorting options, improving user experience.
- Enhance the pipeline to integrate conversation history recording seamlessly during interactions.
- Introduce a new `NetworkQualityIndicator` component to visually represent network quality status during calls.
- Integrate the `NetworkQualityIndicator` into `AssistantPage` and `MobileCallPage`, enhancing user awareness of connection quality.
- Update `use-voice-preview` hook to manage network quality metrics, improving overall call experience and performance monitoring.
- Introduce a new `MobileCallTranscript` component to display chat messages during calls, improving user interaction.
- Implement message timestamp formatting and automatic scrolling for better usability.
- Add a toggle button to switch between camera view and chat view, enhancing the user experience during calls.
- Refactor the layout of the `MobileCallPage` to accommodate the new chat functionality while maintaining existing video call features.
- Introduce a new `RuntimeTool` model to encapsulate tool data for runtime sessions, including attributes like `id`, `name`, `function_name`, `type`, and `description`.
- Update the `AssistantConfig` model to include a list of reusable tools, allowing for better management of tools within assistant configurations.
- Modify the `config_resolver` service to fetch and resolve tools associated with assistants, ensuring they are available during runtime.
- Refactor tool-related CRUD operations in the `tools` route to support the new runtime execution model, enhancing the overall tool management system.
- Update documentation and comments to reflect changes in tool execution and configuration handling, improving clarity for future development.
- Introduce a new `sync_default_tools` function in `session.py` to ensure essential reusable tools are created without overwriting existing edits.
- Update the `lifespan` context manager in `app.py` to call `sync_default_tools`, enhancing the initialization process for the application.
- This change improves the management of default tools within the system, ensuring they are available for use while preserving user modifications.
- Refactor `CallDeviceSelect` to `CallAudioDeviceSelect`, allowing simultaneous selection of microphone and output devices.
- Introduce dropdown menus for selecting audio input and output devices, improving user experience.
- Update `useVoicePreview` hook to manage audio output devices, including support for selecting and applying output devices.
- Modify `MobileCallPage` to integrate new device selection components, ensuring seamless audio and video call functionality.
- Introduce a new `auth-redirect` module to manage safe return paths for login and redirection.
- Update the login page to utilize `readReturnToFromLocation` for redirecting users post-login.
- Modify the `AuthGate` component to use `loginPathWithReturnTo` for redirecting unauthorized users, enhancing security and user experience.
- Adjust API request handling to incorporate the new redirection logic, ensuring users are directed to the appropriate login page with return paths.
- Introduce a new `Tool` model and `AssistantToolBinding` for managing reusable tools within the application.
- Implement CRUD operations for tools in the new `tools` route, allowing for the creation, retrieval, updating, and deletion of tools.
- Update the `Assistant` model to include a list of tool IDs, enabling assistants to utilize these tools.
- Enhance the backend routes to synchronize tool bindings with assistants, ensuring proper management of tool associations.
- Add frontend components for tool management, including a tool picker in the assistant configuration, improving user experience in tool selection.
- Create a mobile call page to facilitate video calls, integrating camera and microphone selection for enhanced communication capabilities.
- Update API definitions to include tool-related types and operations, ensuring consistency across the application.
- Add a migration script to create the necessary database tables for tools and bindings, supporting the new functionality.
- Extend the `allowedDevOrigins` in `next.config.ts` to include "118.89.89.89" for enhanced development flexibility.
- Add a new `.DS_Store` file in the `scripts` directory, which is a system file created by macOS for folder metadata.
- Introduce a new `auth` module with login, logout, and user verification endpoints for a single admin user.
- Update backend routes to require admin authentication for sensitive operations, enhancing security.
- Modify frontend components to include an authentication provider and gate, ensuring only authorized users can access the application.
- Implement a login page for admin access, improving user experience and security management.
- Update API request handling to redirect unauthorized users to the login page, ensuring proper access control.
- Modify `docker-compose.yaml` to use environment variables for HTTP and HTTPS port mappings, allowing flexibility in port configuration.
- Enhance `README.md` with instructions for using non-standard HTTPS ports, including examples for remote development setups.
- Introduce a new `nginx-remote-dev` service in `docker-compose.yaml` for remote development environments, allowing Nginx to proxy requests to the host machine's frontend and backend services.
- Create a new Nginx configuration file `ai-video.remote-dev.conf` tailored for remote setups, ensuring proper handling of WebSocket and API requests.
- Update `README.md` to include detailed instructions for setting up remote development with Nginx, including certificate generation and service initialization.
- Enhance the `setup-certs.sh` script to support additional hostnames for certificate signing, improving flexibility for remote access.
- Revise documentation to clarify the development paths and requirements for local and remote environments.
- Replace the `config.py` module with a new `settings.py` to streamline environment variable management, focusing on database, CORS, and TURN settings.
- Update references throughout the backend codebase to use the new `settings` module instead of the deprecated `config`.
- Modify the `.env.example` file to reflect the new configuration approach, indicating that model provider credentials should be maintained separately.
- Enhance the `AssistantConfig` model to clarify the source of runtime connection information, ensuring it is injected from model resources rather than relying on defaults from the environment.
- Introduce new user scripts for audio and video management in the Tampermonkey environment, enhancing WebRTC capabilities.
- Remove the showPassword state and associated logic for toggling password visibility.
- Streamline the password input to always use type "password" and directly handle value changes.
- Update placeholder text for clarity while maintaining existing functionality for stored values.
- Integrate Alembic for managing database schema migrations, replacing the previous SQL schema management.
- Update the FastAPI application to synchronize interface definitions at startup.
- Modify the Docker Compose command to run Alembic migrations before starting the API.
- Enhance the Makefile with new commands for database migration and revision management.
- Remove outdated SQL schema and seed files, transitioning to a more dynamic migration approach.
- Add initial migration scripts and configuration for Alembic, ensuring a structured database evolution.
- Extend the AssistantConfig model to include agent_interface_type, agent_values, and agent_secrets for enhanced agent management.
- Update ModelType to include "Agent" for broader capability support.
- Seed new agent models and configurations in the database for Dify, FastGPT, and OpenCode applications.
- Modify the Assistant routes to validate and handle agent capabilities.
- Implement agent resource resolution in the config resolver for runtime configuration.
- Enhance the interface catalog with agent-specific fields and definitions.
- Update frontend components to manage agent configurations, including form handling and state management for agent-related inputs.
- Introduce a new user script `audio-sender-sine-tester.js` that lists current WebRTC audio senders and allows temporary sine wave transmission through selected senders.
- Implement functionality for managing audio sender candidates, tracking audio statistics, and providing real-time logging of audio activities.
- Enhance audio testing capabilities with sine wave generation and track replacement features, improving overall audio management in the WebRTC environment.
- This addition supports better monitoring and testing of audio streams in applications utilizing WebRTC.
- Delete the user scripts `copilot-native-element.js` and `copilot-native-static.js` that implemented native-track bridges for audio and video input selection in WebRTC.
- These scripts provided functionalities for managing remote audio and video tracks, recording capabilities, and chat logging features for AI interactions.
- The removal is part of a refactoring effort to streamline the codebase and improve overall media management.
- Introduce a new user script `video-track-probe.js` that probes WebRTC video input/output tracks and displays active RTP statistics.
- Implement functionality to manage video track candidates, including live video detection and performance metrics.
- Enhance logging features to provide real-time feedback on video track status and statistics.
- Integrate mechanisms for tracking video elements and their associated media streams, improving overall media management in the WebRTC environment.
- Introduce a new user script `audio-track-probe.js` that probes WebRTC audio input/output tracks and displays active RTP statistics.
- Implement functionality to manage audio track candidates, including live audio detection and audio level analysis.
- Enhance logging features to provide real-time feedback on audio track status and metrics.
- Integrate audio context management for improved audio visualization and monitoring capabilities.
- Update the description to clarify that participant audio is fed into AI after both sides are connected.
- Introduce new functions to manage audio track patterns and enhance the logic for determining participant and operator presence.
- Improve gate status messaging to reflect the online status of participants and operators.
- Add volume handling functions for better audio visualization and feedback.
- Refactor existing functions to streamline audio processing and improve overall code clarity.
- Introduce a new user script `copilot-element2-two-party.js` that implements a two-party gated bridge for audio input to AI, ensuring both sides are connected before processing.
- Implement functionality for managing audio and video tracks, including recording capabilities for AI input.
- Enhance chat logging features to provide real-time feedback on AI interactions and recording status.
- Integrate mechanisms for handling media streams, improving overall media management in the WebRTC environment.
- Introduce a new user script `copilot-native-direct-output.js` that facilitates direct AI response track output to business senders.
- Implement functionality for managing audio and video tracks, including recording capabilities for AI input.
- Enhance chat logging features to provide real-time feedback on AI interactions and recording status.
- Integrate mechanisms for handling media streams, improving overall media management in the WebRTC environment.
- Introduce a new user script `copilot-native.js` that implements a native-track bridge for audio and video input selection in WebRTC.
- Implement functionality for managing remote audio and video tracks, including recording capabilities for AI input.
- Enhance chat logging features to provide real-time feedback on AI interactions and recording status.
- Integrate mechanisms for handling media streams, improving overall media management in the WebRTC environment.
- Introduce a new user script `copilot-native-element.js` that implements a native-track bridge for audio and video input selection in WebRTC.
- Implement functionality for managing remote audio and video tracks, including recording capabilities for AI input.
- Enhance chat logging features to provide real-time feedback on AI interactions and recording status.
- Integrate mechanisms for handling media streams, improving overall media management in the WebRTC environment.
- Update vision model authorization logic to rely solely on server-side configuration, removing client offer dependency for vision capabilities.
- Enhance clarity in comments regarding the role of server configurations in enabling vision understanding.
- Update `requirements.txt` to include Pillow for image handling.
- Refactor vision model validation logic in `voice_webrtc.py` to improve error handling for unsupported image input.
- Introduce new functions in `pipeline.py` for image data processing and analysis using vision models.
- Implement `VisionCaptureProcessor` to manage video frame requests for auxiliary vision model analysis.
- Enhance the pipeline to support image input requests and integrate vision model responses into the processing flow.
- Introduce a new user script `copilot-native-static.js` that implements a native-track bridge with static input/output selection rules for WebRTC.
- Implement functionality for managing remote audio and video tracks, including recording capabilities for AI input.
- Enhance chat logging features to provide real-time feedback on AI interactions and recording status.
- Integrate mechanisms for handling media streams and improve overall media management in the WebRTC environment.
- Introduce a new function to capture video streams from playable HTML elements, enhancing the ability to process remote video feeds.
- Implement logic to manage and store captured video streams, ensuring only live tracks are utilized.
- Update existing audio processing functions to integrate video handling, improving overall media management in the WebRTC environment.
- Enhance logging for better debugging and user feedback regarding video stream capture status.
- Add new database management targets: db-up, db-schema, db-seed-interface-definitions, and db-init to streamline database setup and seeding processes.
- Update db-seed and db-reset dependencies to include new initialization steps for better data integrity.
- Introduce new SQL files for schema definition and interface definitions seeding, ensuring a consistent database structure.
- Refactor existing seed scripts to align with new dependencies and improve clarity in database operations.
- Add db-drop-schema target to facilitate schema deletion and recreation for development purposes.
- Introduce db-reset-hard target to streamline the process of resetting the database and seeding data after schema changes.
- Update .PHONY targets for improved clarity and maintainability.
- Remove db-init target from db-seed and db-reset dependencies to simplify the seed process.
- Update .PHONY targets for clarity and maintainability.
- Enhance comments for better understanding of database operations.
- Introduce new fields in AssistantConfig, schemas, and database models to support vision capabilities, including `vision_enabled` and `vision_model_resource_id`.
- Enhance validation logic in routes to ensure proper handling of vision models and their requirements.
- Update the AssistantPage and related frontend components to include options for enabling vision understanding and selecting appropriate vision models.
- Modify database seed scripts to include vision-related data for assistants, ensuring consistent setup.
- Refactor related functions to integrate vision model handling in the audio-visual processing pipeline.
- Add `hasUnsavedChanges` prop to `DebugDrawer` to indicate unsaved changes in the AssistantPage.
- Update `DebugVoicePanel` to display appropriate messages and disable actions when there are unsaved changes.
- Refactor related components to ensure consistent handling of unsaved changes across the AssistantPage, improving user experience during interactions.
- Introduce a new `support_image_input` field in model resources, allowing models to indicate support for image input.
- Update the backend models, schemas, and database seed scripts to accommodate the new field.
- Enhance the AssistantConfig and related routes to handle image input capabilities, ensuring proper validation and error handling.
- Modify the frontend components to include toggles for enabling visual understanding and filtering models based on image input support.
- Implement necessary adjustments in the voice preview and pipeline to integrate video stream handling alongside audio functionalities.
- Introduce a new user script `copilot.js` for WebRTC that implements PCM ScriptProcessor to address packet loss issues.
- Add features for real-time audio monitoring and improved audio stream handling, including mechanisms for capturing and processing remote audio streams.
- Implement a robust audio context setup with gain nodes and oscillators for enhanced audio management.
- Enhance logging and state management for better debugging and monitoring of audio streams in the WebRTC environment.
- Introduce a new utility class `.scrollbar-subtle` in `globals.css` for a refined scrollbar appearance.
- Update `AssistantPage` to apply the subtle scrollbar styling to overflow containers, enhancing the user interface.
- Reset message state and sequence in `useVoicePreview` hook upon resource release, ensuring clean state management.
- Introduce a new `.env.example` file for environment variable setup, including `PUBLIC_IP`, `TURN_SECRET`, and `TURN_URLS` for WebRTC TURN server configuration.
- Update `docker-compose.yaml` to support TURN server deployment with necessary environment variables and commands.
- Enhance backend configuration and routes to include WebRTC ICE server settings, allowing for STUN/TURN server integration.
- Implement a new service for managing WebRTC ICE server configurations, providing credentials for TURN when configured.
- Modify frontend API to fetch ICE server configurations dynamically, improving support for cross-network voice preview.
- Add a new `useCameraPreview` hook for managing camera access and video stream handling.
- Integrate camera preview capabilities into the AssistantPage, allowing users to toggle video stream visibility.
- Introduce a new toggle for enabling visual understanding, which activates the video stream preview alongside voice functionalities.
- Update the DebugDrawer component to support switching between chat and video views based on the visual understanding setting.
- Refactor related components to accommodate the new camera features and improve user interaction during debugging.
- Add new fields in AssistantConfig for FastGPT connection details, including `fastgpt_api_url`, `fastgpt_api_key`, and `fastgpt_app_id`.
- Update the pipeline to utilize the new FastGPT configuration, ensuring proper integration with external services.
- Introduce type handling for different assistant types, including support for realtime modes and external brain management.
- Refactor frontend components to include hints for FastGPT configuration inputs, improving user guidance during setup.
- Introduce a new parameter `audio_out_end_silence_secs` in the `_base_params` function to control the duration of silence added after the end frame, allowing for smoother call termination.
- Set the default value to 0 to ensure immediate hang-up after the end speech, enhancing user experience during call endings.
- Introduce mechanisms in the pipeline to ensure that the end call process waits for the completion of the end speech before hanging up, improving user experience during call termination.
- Update the useVoicePreview hook to handle server-initiated call endings gracefully, distinguishing between normal and error disconnections.
- Adjust TTS stop frame timeout settings to optimize the timing of call terminations, ensuring timely responses without unnecessary delays.
- Refactor related components to support the new end call logic, enhancing overall workflow management and user interaction.
- Introduce edge transition speech functionality in the WorkflowEngine to provide optional speech during node transitions.
- Update pipeline execution to utilize the new transition speech feature, enhancing user experience by masking delays during transitions.
- Modify frontend components to support transition speech in edge specifications, allowing users to define and edit transition speech for edges.
- Refactor edge handling logic in the WorkflowEditor to accommodate the new transition speech field, improving workflow management capabilities.
- Change the 'addable' property of a specific node type to true, allowing for dynamic addition of nodes.
- Modify the GenericNode component to include a new icon and adjust styles for better visual representation.
- Update node handling logic to prevent deletion of 'startCall' nodes and improve node change handling in the workflow editor.
- Refactor layout and styling in the WorkflowEditor for a more polished user interface.
- Introduce a new WorkflowEngine class to manage workflow graphs, enabling dynamic node-based interactions.
- Update AssistantConfig to include a graph field for workflow definitions, allowing for flexible configuration.
- Modify pipeline execution to support workflow-driven dialogue, integrating node transitions and system prompts based on active nodes.
- Enhance frontend components to visualize active nodes and provide debugging capabilities, including highlighting the current node during interactions.
- Refactor existing components to accommodate new workflow functionalities and improve overall user experience.
- Introduce a new workflow editor component for visualizing and managing workflows, allowing users to add nodes and define connections.
- Implement backend support for node types, including validation and constraints for workflow graphs.
- Add new API endpoints for retrieving node types and their specifications.
- Enhance the AssistantPage to integrate the workflow editor, enabling users to create and edit workflows directly.
- Update frontend components to support new workflow functionalities, including condition edges and generic nodes.
- Refactor existing code to accommodate the new workflow features and improve overall structure.
- Add new fields to AssistantConfig for realtime interface configuration, including types, values, and secrets.
- Introduce StepFunRealtimeService to handle speech-to-speech processing via WebSocket, integrating STT, LLM, and TTS functionalities.
- Refactor pipeline execution to support a new realtime mode, allowing direct text input processing and immediate responses.
- Update model resource testing to include validation for StepFun Realtime connections.
- Enhance service factory to create realtime services based on configuration settings.
- Modify README documentation to reflect new realtime capabilities and usage instructions.
- Introduce event handlers in PassthroughLLMAssistantAggregator for managing LLM text streaming, including start, delta, and end events.
- Implement a new method to finalize text streams, ensuring proper handling of interruptions.
- Update useVoicePreview to support new message types for LLM text streaming, allowing real-time updates to chat messages.
- Enhance message sorting logic to maintain order based on timestamps and sequence numbers, improving user experience during voice interactions.
- Refactor TextInputProcessor to handle immediate and silent text inputs, improving user experience during voice interactions.
- Introduce PassthroughLLMAssistantAggregator to manage LLM responses while preserving context for downstream TTS processing.
- Update event handling for text input and client readiness, ensuring timely updates to the conversation context.
- Modify run_pipeline to integrate new aggregators and streamline message handling, enhancing overall pipeline efficiency.
- Improve message ordering in useVoicePreview to ensure accurate display of chat messages based on timestamps.
- Rename `setSelectedDeviceId` to `selectDevice` in `DebugVoicePanel` and `VoiceSessionControls` for clarity and consistency.
- Update `useVoicePreview` hook to implement the `selectDevice` function, enabling dynamic microphone switching during voice sessions.
- Enhance device selection logic to support real-time audio track replacement without requiring session reconnection.
- Introduce a new model structure for managing interface definitions and model resources, enhancing the backend's capability to handle various service integrations.
- Update the Makefile to reflect changes in database seeding and resource management commands.
- Remove the deprecated credentials management routes and replace them with a unified model registry API.
- Modify existing routes and schemas to align with the new model structure, ensuring seamless integration with the frontend.
- Enhance database seeding scripts to populate new model resources and their configurations.
- Update README documentation to reflect the new architecture and usage instructions for model resources and interface definitions.
- Introduce new Xfyun ASR and TTS services, enabling integration with iFlytek's voice recognition and synthesis capabilities.
- Update AssistantConfig model to include interface types for STT and TTS.
- Enhance credential testing to validate Xfyun credentials.
- Modify service factory to create Xfyun services based on configuration.
- Update README with new configuration details for Xfyun integration.
- Add new frontend components for visualizing audio streams and managing user interactions.
- Add audio input selection to DebugVoicePanel, allowing users to choose their microphone device.
- Update useVoicePreview hook to manage available audio inputs and selected device state.
- Enhance device enumeration and selection handling to ensure a seamless user experience during voice interactions.
- Add a new Docker configuration for the UI in launch.json to facilitate development.
- Refactor pipeline.py to integrate a TranscriptProcessor for managing user and assistant transcripts, including event handlers for real-time updates and message handling.
- Update useVoicePreview.ts to establish a data channel for sending and receiving text messages, improving interaction flow.
- Modify AssistantPage.tsx to support displaying chat messages and sending user input, enhancing the user experience during voice interactions.
- Revise DebugTranscriptPanel to dynamically render chat messages with timestamps, improving the visual representation of conversation history.
- Update CLAUDE.md to reflect changes in the navigation model, emphasizing the use of App Router routes for sidebar sections.
- Refactor layout.tsx to wrap children in AppShell, enhancing the overall layout structure.
- Replace AppShell usage in page.tsx with HomePage component for better separation of concerns.
- Introduce new pages for assistants, components, dashboard, history, profile, and test, each rendering their respective components.
- Revise Sidebar component to utilize Next.js Link for navigation and improve active state handling based on the current pathname.
- Update AssistantPage to support routing-driven modes (list, choose, edit) and streamline form handling for assistant creation and editing.
- Introduce `setup-certs.sh` script for generating trusted local TLS certificates using mkcert.
- Add Nginx configuration files for local and Docker environments to handle HTTPS requests and proxy to backend services.
- Update `docker-compose.yaml` to include Nginx service for unified TLS entry and adjust frontend service ports for local development.
- Create `AGENTS.md` and `README.md` files to document the local HTTPS setup process and usage instructions.
- Modify backend startup commands in `README.md` for consistency with new requirements.
- Add `.gitignore` to exclude generated certificates from version control.
- Update README to reflect the integration of the DebugVoicePanel with WebSocket support for voice interactions.
- Refactor voice_webrtc.py to improve error handling during WebRTC signaling and include assistant_id in the offer payload.
- Add useVoicePreview hook to manage microphone access and WebRTC connections for real-time voice previews.
- Modify AssistantPage to incorporate new visualizer options and pass assistantId to DebugVoicePanel, enhancing user experience during audio interactions.
- Update API model to include new fields for voice, speed, and language, supporting TTS and ASR configurations.
- Implement rgbToHsl and hslToRgb functions for color space conversions.
- Introduce adaptPalette function to adjust colors based on dark/light themes, enhancing visual consistency.
- Add isDarkTheme function to determine the current theme state, improving theme handling across visual components.
- Update SpectrumVisualizer component to enhance the visual representation of audio frequencies with a new layout and smoother animations.
- Modify prop descriptions for clarity and adjust the number of frequency bars for better performance.
- Implement a refined drawing logic that maintains visual consistency across different themes and improves the overall user experience during audio playback.
- Introduce the NebulaVisualizer component, featuring particles that respond to audio input, enhancing the visual experience.
- Refactor AuraVisualizer, SpectrumVisualizer, and WaveVisualizer to utilize the adaptPalette function for improved theme handling.
- Update visualizer logic to enhance responsiveness and visual effects based on audio analysis, ensuring a cohesive user experience across components.
- Introduce the AssistantIdentity component to show the current assistant ID and provide a copy-to-clipboard feature.
- Update multiple sections of AssistantPage to include the AssistantIdentity component, enhancing user interaction with assistant IDs.
- Ensure the component handles the display of the assistant ID and provides feedback when copied successfully.
- Introduce three new audio visualizer components: AuraVisualizer, SpectrumVisualizer, and WaveVisualizer, enhancing the audio interaction experience.
- Replace the deprecated VoiceVisualizer with the new visualizers, ensuring a cohesive visual language across components.
- Update the AssistantPage to support dynamic visualization style switching, improving user engagement during audio interactions.
- Refactor DebugVoicePanel to accommodate the new visualizer props and enhance the overall debugging interface.
- Remove debug mode state management from AssistantPage, simplifying the component structure.
- Update DebugDrawer to eliminate mode selection, focusing on voice interaction features.
- Enhance the VoiceVisualizer component with improved visual effects and responsiveness to audio input.
- Adjust styles and layout for better user experience in the debugging interface.
- Introduce new fields for voice, speed, and language in the AssistantConfig and ProviderCredential models to support TTS and ASR configurations.
- Update the database schema and seeding script to accommodate the new fields, ensuring backward compatibility.
- Implement credential testing endpoints and logic to validate OpenAI-compatible credentials, enhancing user experience and reliability.
- Modify frontend components to include new fields in the credential forms and improve connection testing feedback.
- Refactor related services and API interactions to support the new credential testing feature.
- Introduce a new model field in the OpenCode form to manage language model selection.
- Refactor the form handling logic to improve data loading and error management for OpenCode assistants.
- Update UI components to utilize ResourceSelectField for model and voice configuration, enhancing user experience.
- Clear form fields when creating new OpenCode entries to ensure a fresh start for users.
- Update AssistantPage to use a stored value mask for API keys, improving security and user experience.
- Modify ComponentsModelsPage to display the current API key contextually, enhancing clarity for users.
- Adjust related components to ensure consistent handling of API key visibility and management.
- Update AssistantPage to handle API key input more securely by removing placeholder values and allowing empty submissions to retain existing keys.
- Introduce a new SecretInputField component for API key entry, improving user experience with visibility toggling and contextual hints.
- Modify ComponentsModelsPage to reflect similar API key handling, ensuring users can manage keys effectively while providing feedback on existing configurations.
- Add EditorBackButton for better navigation within the AssistantPage.
- Change 'DeepSeek-V3' to 'DeepSeek-Chat' and update its API key.
- Rename 'OpenAI TTS' to 'SiliconFlow-CosyVoice2-0.5B' and update its details.
- Add new models: 'SiliconFlow-TeleSpeechASR' and 'SiliconFlow-Qwen3-Embedding-4B' with corresponding API keys and configurations.
- Adjust existing entries to ensure consistency in the database seeding process.
- Update Makefile to include new database seed commands for assistants and credentials.
- Refactor assistant model to use explicit fields instead of a config dictionary, improving data integrity and clarity.
- Implement new seeding SQL script for assistants, ensuring dependencies on credentials are respected.
- Modify backend routes and frontend components to accommodate the new assistant structure, including direct field access for prompt, API URL, and keys.
- Enhance the AssistantPage component to handle the new data structure and streamline the save process for different assistant types.
Implement server-side duplication for both assistants and credentials, allowing users to create copies with unique IDs and modified names. Update the respective API routes and frontend components to handle duplication requests, ensuring sensitive information is securely managed. Enhance the AssistantPage and ComponentsModelsPage to support this new feature, including loading and error handling for the duplication process.
Add CRUD functionality for knowledge bases, including routes for listing, creating, updating, and deleting knowledge bases. Update the assistant model to include foreign key references to knowledge bases and modify the assistant configuration to handle external API keys securely. Refactor related services and routes to accommodate these changes, ensuring proper handling of credential resolution and configuration normalization.
Implement a new feature that allows users to create a copy of an existing assistant. The duplicate is inserted after the original, with the name suffixed by "副本" and the updated timestamp. Update the state management to handle the list of assistants accordingly, and add a dropdown menu item for triggering the duplication.
Xin Wang