Add caching mechanism for completed AV blocks in streaming model. Introduce CompletedBlockPieceCache class to manage reusable blocks during strict-window replay, enhancing efficiency. Update profiling hooks for cache operations and refine README documentation to reflect changes in block handling and replay strategies.
This commit is contained in:
@@ -477,7 +477,9 @@ two practical changes for this dataset:
|
||||
1. the current 0--2 second partial block is evaluated speculatively every
|
||||
0.2 seconds and discarded unless it reaches a complete block boundary;
|
||||
2. the KV history is bounded and rebuilt from recent complete blocks rather
|
||||
than accumulating for an entire multi-minute session.
|
||||
than accumulating for an entire multi-minute session. Complete-block AV
|
||||
embeddings are retained in a bounded ring, so rolling rebuilds rerun only
|
||||
the Thinker and do not rerun the processor, audio tower, or video tower.
|
||||
|
||||
It should therefore be described as **bounded Qwen-style open streaming**, not
|
||||
as bit-for-bit reproduction of an unbounded official serving session.
|
||||
@@ -522,7 +524,10 @@ prediction is emitted every 0.2 seconds.
|
||||
budget it rebuilds the open cache from the latest nine completed blocks
|
||||
(18 seconds) when a new block starts, then adds the current 0--2 second tail.
|
||||
Its effective history therefore varies from 18 to 20 seconds and never drops
|
||||
to an empty 0.2-second context at a 20-second boundary.
|
||||
to an empty 0.2-second context at a 20-second boundary. The nine blocks'
|
||||
prepared AV embeddings and local TMRoPE coordinates are cached once; strict
|
||||
window replay still recomputes the Thinker KV so retained blocks no longer
|
||||
carry indirect influence from observations that fell out of the window.
|
||||
|
||||
For a short native-stream fidelity check, set `--memory-seconds` to at least
|
||||
the tested clip duration. For example, the options
|
||||
|
||||
Reference in New Issue
Block a user