Commit Graph
113 Commits
Author SHA1 Message Date
Xin Wang 4011ba9b2a Record EasyCom encoder-only results and horizon analysis 2026-08-28 11:15:09 +08:00
Xin Wang 321238a92f Add EasyCom encoder-only comparison workflow 2026-08-28 10:44:13 +08:00
Xin Wang ee5e735fc2 Add encoder-only Ego4D comparison workflow 2026-08-28 09:36:35 +08:00
Xin Wang 18e8773685 Handle two-frame Ego4D bbox tails and preserve worker errors 2026-08-28 09:26:45 +08:00
Xin Wang 668996bfa8 Handle verified Ego4D terminal bounding-box records 2026-08-28 01:11:51 +08:00
Xin Wang 5fd285e369 Add multi-GPU Ego4D local feature extraction without symlinks 2026-08-28 01:01:25 +08:00
Xin Wang 9bd204b468 Add more preview 2026-08-28 00:46:05 +08:00
Xin Wang b5ce64702a Add Ego4D local AV ablation workflow 2026-08-28 00:25:54 +08:00
Xin Wang 8b2fbd2622 Merge remote-tracking branch 'aliyun/main' 2026-08-28 00:21:47 +08:00
Xin Wang 5f45e7c115 Add EasyCom interaction weak-label pipeline 2026-08-28 00:07:54 +08:00
Xin Wang c4feeb47f4 Fix transformers version 2026-08-27 15:24:59 +08:00
Xin Wang 694473b475 Fix bbox offset 2026-08-27 15:08:38 +08:00
Xin Wang 11f197724f Add pooled local AV baselines and research notes 2026-08-27 14:19:33 +08:00
Xin Wang d34725778f Avoid upstream Omni loading-info speaker crash 2026-08-27 13:50:02 +08:00
Xin Wang 5e34563739 Support upstream Omni class for ROMA loading 2026-08-27 13:45:25 +08:00
Xin Wang 655d27a4ca Add ROMA frozen backbone extraction support 2026-08-27 13:35:59 +08:00
Xin Wang 1925ff71a7 Record preliminary transition auxiliary results 2026-08-27 09:25:08 +08:00
Xin Wang 5923b89d60 Add transition-aware R1 and R2 objectives 2026-08-26 17:18:50 +08:00
Xin Wang 0590df1e88 Record EasyCom loss structure ablation 2026-08-26 16:30:48 +08:00
Xin Wang f35b9108ac Add overlap-aware turn prediction objectives 2026-08-26 15:03:56 +08:00
Xin Wang f45ea9cbc6 Add higher-capacity last-four-layer fusion 2026-08-26 13:06:18 +08:00
Xin Wang 43e0e14cf5 Add learnable last-four-layer query fusion 2026-08-26 10:29:43 +08:00
Xin Wang dbef1fc173 Add final-layer mixture readout 2026-08-26 09:56:54 +08:00
Xin Wang ca6dd88923 Add native block phase diagnostic 2026-08-26 09:10:21 +08:00
Xin Wang 56c8fe0a0e Fix final LoRA checkpoint report 2026-08-25 09:45:47 +08:00
Xin Wang aa4278e13e Fix final-layer LoRA target matching 2026-08-24 15:38:45 +08:00
Xin Wang 7fdda297d3 Save periodic LoRA weight checkpoints 2026-08-24 15:31:44 +08:00
Xin Wang 7acab44ea2 Add low-drift LoRA training controls 2026-08-24 15:24:53 +08:00
Xin Wang b7856ece4d Archive EasyCom sparse supervision results 2026-08-24 12:02:49 +08:00
Xin Wang edd37dc36a Add offline sparse supervision control 2026-08-24 11:45:11 +08:00
Xin Wang 43cf1f386e Add frozen Thinker sparse head-only control 2026-08-24 11:32:03 +08:00
Xin Wang f6dbce30e4 Reuse global history frames in overlap working memory 2026-08-24 11:17:39 +08:00
Xin Wang 1731f2b6f7 Optimize overlap working-memory audio preprocessing 2026-08-24 10:18:55 +08:00
Xin Wang 20446c0bd8 Add overlap working-memory ablation and LoRA epoch2 record 2026-08-24 09:52:56 +08:00
Xin Wang 3cc0c3da85 Add frozen Thinker cache-path control 2026-08-24 01:07:42 +08:00
Xin Wang c5e2e044a8 Add dense-cache evaluation for Omni LoRA 2026-08-24 00:27:42 +08:00
Xin Wang a755ab4fc3 Add candidate improvement for explicit working and long memory representation in Omni LoRA 2026-08-23 23:49:57 +08:00
Xin Wang 909ab66c09 Add adapter checkpoint support for continued training in Omni LoRA 2026-08-23 23:49:09 +08:00
Xin Wang 7cdb2d90c2 Add multi-GPU DDP for Omni LoRA training 2026-08-23 22:41:39 +08:00
Xin Wang cb4abfdf82 Add transition-sparse Omni AV caching 2026-08-23 18:28:07 +08:00
Xin Wang 8b373cd19b Optimize online-tail audio preprocessing 2026-08-23 16:49:10 +08:00
Xin Wang c844673984 Add TorchCodec video backend 2026-08-23 16:41:05 +08:00
Xin Wang b875a7512e Optimize Omni audio preprocessing 2026-08-23 16:26:01 +08:00
Xin Wang 85ed8a670c Batch Omni AV cache extraction 2026-08-23 16:03:07 +08:00
Xin Wang c2162e1170 Add two-GPU AV producer for Omni LoRA 2026-08-23 15:29:26 +08:00
Xin Wang dc932cc165 Add online current-tail LoRA benchmark 2026-08-23 15:10:16 +08:00
Xin Wang 1069a0beee Add cached AV pipeline for Omni turn LoRA 2026-08-23 14:40:03 +08:00
Xin Wang 2eb61e30de Add README for Omni-only Turn-Taking Experiment detailing project goals, architecture, current status, and EasyCom results. Include comparisons with EgoSpeak and clarify dataset handling and evaluation processes. 2026-08-19 23:09:56 +08:00
Xin Wang 2e773706bd Enhance EasyCom and Ego4D integration by adding dataset support and manifest handling. Update normalization functions to accommodate both datasets, and modify training and inference scripts to require manifest files for Ego4D. Improve argument parsing and validation for dataset-specific operations, ensuring proper session management and error handling. Update documentation to reflect changes in dataset usage and evaluation processes. 2026-08-19 20:49:28 +08:00
Xin Wang 92bf93c96b Update README with verification status of EasyCom components and add results from recent experiments comparing open-stream replay and causal-prefix methods. Introduce new metrics CSV and detailed README for the EasyCom results, including current performance metrics and required follow-up actions before publication. 2026-08-19 11:29:27 +08:00