Files
icassp2027-experiment/results/easycom_dense_last8_qvo_full_train_epoch1_2026-08-25

EasyCom dense last-8 LoRA, full official train split, one epoch

Recorded on 2026-08-25 from user-reported remote evaluator output. This is a single-seed diagnostic experiment.

Experiment contract

  • Backbone: Qwen2.5-Omni-3B Thinker.
  • Input representation: dense cached AV phases, 2 FPS video, 5 Hz prediction, and 8 seconds of causal open-stream memory.
  • Head: pretrained frozen 8-second head from checkpoints/easycom_open_stream_2fps_8s_head_seed0.pth.
  • LoRA: q_proj, v_proj, and o_proj in Thinker layers 28--35, rank 8, alpha 16, dropout 0.05, learning rate 5e-6, and weight decay 0.01.
  • Training: all official EasyCom train Sessions 4--12, one fixed epoch, 4526 updates, global batch 16, and no development checkpoint selection.
  • Evaluation: official test Sessions 3, 2, and 1 using the EgoSpeak Table 2 full-timeline protocol.
  • Training runtime: 1475.936 seconds on eight GPUs, 3.067 updates/second, with 8.86 GiB as the maximum per-rank CUDA allocation.
  • Cached loss: 1.272857 before training and 1.269106 after training.

The evaluated checkpoint was:

checkpoints/
  omni_turn_lora_dense_8s_last8_qvo_frozen_head_train4to12_epoch1_ddp8/

Official Sessions 1--3 result

Horizon LoRA mAP Frozen dense mAP Delta Background AP Wearer AP Others AP
0.2s 71.62100 71.69695 -0.07595 54.41759 76.41473 84.03068
0.4s 66.38400 66.46468 -0.08068 47.56390 70.58441 81.00369
0.6s 61.93324 62.03503 -0.10179 42.35133 64.80023 78.64817
0.8s 58.64198 58.77409 -0.13211 39.04739 59.81664 77.06191
1.0s 56.30504 56.43606 -0.13102 37.17699 55.95400 75.78412
1.2s 54.87690 55.00306 -0.12616 36.30712 53.40678 74.91682
1.4s 53.60541 53.73114 -0.12573 35.40362 51.12561 74.28699
1.6s 52.89234 53.04042 -0.14808 35.22795 49.68491 73.76416
1.8s 52.26201 52.42454 -0.16253 34.76888 48.59595 73.42119
2.0s 51.82720 52.00315 -0.17595 34.76777 47.63587 73.07796
Mean 58.03491 58.16091 -0.12600 39.70325 57.80191 76.59957

Controlled comparisons

Checkpoint Train sessions Epoch 0.2s mAP Mean mAP Mean delta vs frozen
Frozen dense control 4--12 head -- 71.69695 58.16091 --
Restricted LoRA 4--12 1 71.62100 58.03491 -0.12600
Restricted LoRA 4--11 1 71.59585 58.01500 -0.14591
Restricted LoRA 4--11 3 71.65000 57.88833 -0.27258

Adding Session 12 to the one-epoch LoRA training improves 0.2-second mAP by 0.02515 and mean mAP by 0.01991 relative to the Sessions 4--11 epoch-1 checkpoint. All ten horizons improve, but only by 0.00413--0.05420 points. This is too small to explain or reverse the negative transfer.

Relative to the frozen dense control, background mean AP is unchanged (+0.01891), while camera-wearer and other-person mean AP remain lower by -0.24232 and -0.15460. The final mean gap is -0.12600, less than half the epoch-3 gap but still negative.

Interpretation

  1. Using the complete official training split helps consistently but only marginally; withholding Session 12 was not the cause of the regression.
  2. One epoch is preferable to three for this restricted adapter because it preserves longer-horizon representations better.
  3. The adapter still does not improve the frozen Omni representation. This experiment should be reported as a negative-transfer ablation rather than the main result.
  4. The next primary comparison should use a stronger frozen backbone, such as Qwen2.5-Omni-7B, without LoRA.

Provenance and limitations

  • Source: user-reported eval_omni_turn_lora.py console output.
  • Raw remote metrics.json archived here: no.
  • Strict causal-valid metrics supplied: no.
  • Multi-seed: no.
  • Test sessions had already been inspected in earlier LoRA diagnostics, so this is not an untouched test-set selection result.