Skip to content

Workflow status is tracked in GitHub: https://github.com/emulebb/emulebb/issues/101. This local document is retained as an engineering spec/evidence record.

REF-048 - Reduce queue and tracking scan cost without policy drift

Summary

Several active queue and tracking paths still pay avoidable linear-scan or repeated-selection costs. The safe opportunity is to reduce scan work while preserving the existing queue policy, network timing, packet order, and stale entry semantics.

This is a small refactor lane, not a queue-policy rewrite.

Current Evidence

  • CDownloadQueue local server source-request selection repeatedly searches for the next best request, effectively O(15 * N) for the current batch.
  • Kad packet tracking and callback tracking lists mostly clean expired tail entries during insert paths, so idle periods can leave stale records that are still scanned on later queries.
  • Client duplicate/source lookup paths still contain broad linear scans, but the mutable key set makes secondary indexes risky without diagnostics.
  • CDownloadQueue::SortByPriority still uses CList::FindIndex during heap sorting. This is lower priority because tie-order regressions are easy.

Intended Shape

  1. Factor stale-tail cleanup into Kad packet-tracking and callback-query paths before scans, only removing entries that are already expired by current rules.
  2. Replace repeated local server source-request selection with a one-pass top-N selection that preserves the same priority and wait-order ties.
  3. Add targeted counters or debug-only assertions around candidate counts and selected request order before attempting riskier client lookup indexes.
  4. Consider vector-backed stable priority sorting only after measuring priority resort cost on large download lists.

Scope Constraints

  • Do not change source admission, upload/download scheduling, queue caps, request cadence, retry timing, or server packet format.
  • Do not add secondary client indexes until all key mutation points are known and debug-verified.
  • Do not snapshot raw client pointers across locks without an explicit lifetime guard.
  • Keep any stale-entry cleanup equivalent to entries the current code would already ignore.

Acceptance Criteria

  • [ ] Kad and callback query paths remove expired tail entries before scanning.
  • [ ] Local server source-request batching chooses the same top 15 files as the current repeated-selection logic for golden queue fixtures.
  • [ ] OP_GETSOURCES frame content and ordering remain unchanged for tested fixtures.
  • [ ] Metrics or debug assertions document candidate counts and selected order.
  • [ ] Client secondary indexes remain deferred unless diagnostics proves a clear hotspot and every key update point is covered.

Validation

  • python -m emule_workspace validate
  • focused native queue-selection golden tests
  • Kad packet-tracking expiry tests
  • source-request packet golden comparison
  • x64 Debug and Release app builds before commit