wx44wx/icassp2027-experiment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add causal MiniCPM-o streaming feature extraction

Browse Source
This commit is contained in:
Xin Wang
2026-09-17 01:08:50 +08:00
parent 45facc8bf4
commit 6d0c8b733b
5 changed files with 1533 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1
View File
@@ -121,6 +121,7 @@ change what the Stage-0 result measures.
| `easycom_manifest.py` | Verified on EasyCom | Discover one-minute clips, preserve session timecodes, and check 5 Hz targets |
| `qwen_omni_features.py` | Implemented, not executed here | Load only the 3B Thinker and extract fused states from one AV window |
| `streaming_turn_model.py` | Verified on EasyCom | Produce one causal Omni state every 200 ms, with causal-prefix and open-stream modes |
| `streaming_minicpmo_features_directly.py` | Implemented; real-GPU smoke pending | Extract input-only MiniCPM-o 4.5 native 1 Hz held states or audited 5 Hz cache-plus-partial-tail states |
| `TurnPredictionHead` | Trained and evaluated | Map `[2048]` to `[10, 3]` logits |
| `train_turn_head.py` | Verified on EasyCom | Train the frozen-Omni anticipation head with the official EasyCom split |
| `infer_turn_head.py` | Verified on EasyCom | Save EgoSpeak-format predictions and reproduce per-horizon AP/mAP |