Skip to content

Workflow status is tracked in GitHub: https://github.com/emulebb/emulebb/issues/102. This local document is retained as an engineering spec/evidence record.

REF-049 - Shorten upload hot-path locks only after adding lifetime guards

Summary

Upload disk dispatch and upload bandwidth throttling both hold broad locks while doing per-client or socket work. Reducing those lock windows could improve throughput and responsiveness under many active uploaders, but the locks also act as implicit lifetime guards today.

This item tracks the safe prerequisite work: make the lifetime model explicit first, then shorten locks.

Current Evidence

  • UploadDiskIOThread performs per-client disk-dispatch work while the upload list lock protects UploadingToClient_Struct lifetime.
  • UploadBandwidthThrottler::RunInternal holds the global queue lock while it calls socket send and poll methods.
  • UploadBandwidthThrottler::RunInternal currently calls SendControlData() and SendFileAndControlData() while holding queueLocker, so disconnect, removal, and rebind latency can be coupled to peer/socket send behavior.
  • CClientUDPSocket::SendControlData() can sleep for 20 ms after a would-block send while it is still inside the throttler dispatch path and under the UDP send lock.
  • Duplicate control-queue notifications are intentionally allowed today, but they can create no-op wakeups and repeated work under storms.
  • Recent packet and socket ownership hardening makes this direction more realistic, but raw-pointer snapshots would reintroduce use-after-free risk.

Intended Shape

  1. Add explicit upload-entry and socket/send lifetime guards that can be held outside the list or throttler queue lock.
  2. Snapshot only guarded dispatch/send work under the lock.
  3. Release the broad lock before disk dispatch, Winsock send, or poll work.
  4. Remove sleep/yield behavior from any path that still runs under throttler queue locks or per-socket send locks; requeue and let readiness/wakeup drive retry instead.
  5. Add lock-wait, pending-read, control-queue depth, and send-loop counters so the change can be measured.
  6. Consider per-socket pending-bit dedupe for control-queue notifications only after the wakeup path has lost-wakeup tests.

Scope Constraints

  • Do not snapshot raw CUpDownClient*, socket pointers, or upload-entry pointers across lock release without an explicit ref or dispatch token.
  • Do not change upload slot policy, bandwidth limits, packet cadence, queue scoring, or retry timing.
  • Do not merge this with broad async socket replacement work.
  • Treat lock shortening as a second slice after lifetime guards and counters are in place.

Acceptance Criteria

  • [ ] Upload disk dispatch has an explicit guarded lifetime handoff.
  • [ ] Throttler socket send work has an explicit send/lifetime guard.
  • [ ] Broad locks are released before expensive per-client or socket work only where guarded ownership proves safety.
  • [ ] UDP control send retries do not sleep while holding throttler queue locks or per-socket send locks.
  • [ ] Stress tests cover rapid disconnect/removal while disk reads or throttler sends are pending.
  • [ ] Application Verifier or equivalent leak/use-after-free checks pass during upload churn.
  • [ ] Counters show lock wait and queue-depth behavior before and after the lock-window changes.

Validation

  • python -m emule_workspace validate
  • upload churn tests with rapid disconnects and removals
  • pending-read counter parity checks
  • throttler control-queue storm tests
  • x64 Debug and Release app builds before commit