Skip to content

Shared Files 50k-file tree refresh stress gate

Summary

Add an Beta 0.7.3 release gate that stress-tests Shared Files tree refresh behavior with 50000 shared files while the UI is rapidly repainting, selecting, collapsing, expanding, sorting, and reloading.

The current live Shared Files scenarios cover reload preservation, monitored folder updates, and dynamic folder lifecycle behavior, but they do not yet prove the large-tree race boundary that matters for Beta 0.7.3 stability.

Risk Being Covered

  • stale CDirectoryItem* or HTREEITEM use after tree rebuild
  • owner-draw list callbacks racing backing model changes
  • UI hangs during large recursive reloads
  • invalid row counts or stale rows after directory filter changes
  • GDI/USER/private-byte growth during repeated redraw and tree churn

Execution Plan

  1. Extend the Shared Files fixture generator with a selectable stress profile that creates at least 50000 shared files and at least 10k observable nodes across deep, wide, duplicate-name, long-path, Unicode, empty-folder, recursive, and non-recursive roots.
  2. Extend the Shared Files live UI harness with explicit collapse support and a rapid tree-churn driver that sends repeated expand, collapse, select, reload, list-sort, and repaint/update operations.
  3. Run the churn driver while monitored filesystem changes are creating, deleting, and replacing files under shared roots.
  4. Cross-check UI-visible row counts against /api/v1/shared-files and /api/v1/shared-directories snapshots after each churn phase.
  5. Capture failure artifacts: tree path, selected directory, visible row count, REST snapshots, UI screenshot, process exit code, timeout reason, and process handle/GDI/USER/private-byte deltas.
  6. Add a short smoke budget for routine release checks and a longer soak budget for operator validation.

Acceptance Criteria

  • [x] the stress fixture reliably creates at least 50000 shared files and at least 10k tree/list nodes
  • [x] rapid expand/collapse/select/reload/sort/paint churn is implemented in the live Shared Files UI harness
  • [x] rapid expand/collapse/select/reload/sort/paint churn completes without crash or hang
  • [x] UI row counts and REST shared-file snapshots are checked for convergence before and after churn
  • [x] UI row counts and REST shared-file snapshots converge after churn in a recorded live smoke artifact
  • [x] monitored-folder updates during churn do not leave stale or duplicate rows
  • [x] GDI, USER, handle, and private-byte resource snapshots are emitted by the stress scenario
  • [x] GDI, USER, handle, thread, and private-byte growth stay within documented release thresholds
  • [x] the tree-refresh-stress-10k scenario is wired into the aggregate live Shared Files UI suite
  • [x] smoke and soak churn counts are selectable through supported workspace live-e2e entrypoints

Progress Evidence

  • Test harness commit: 92002da.
  • Test harness commit: aea5e55.
  • Test harness commit: 6ebc3a7.
  • Test harness commit: 8d63a45.
  • Test harness commit: e751fbb.
  • Build orchestration commit: 756819d.
  • Implemented an opt-in shared_files_tree_stress generated fixture with 50000 shared files and 10k+ observable tree/list nodes.
  • Added the tree-refresh-stress-10k Shared Files UI scenario with rapid select, collapse, expand, reload, sort, and paint churn.
  • Added process resource snapshots before and after churn.
  • Enabled the isolated REST API for the tree stress scenario and added /api/v1/shared-files row-count convergence checks before and after churn.
  • Exposed --shared-files-tree-stress-churn-cycles in the aggregate live suite and -SharedFilesTreeStressChurnCycles in the supported workspace entrypoint.
  • Added targeted Python coverage for the stress node-count floor.
  • Hardened the tree stress fixture to select the generated stress subtree before the initial row gate and use a longer main-window wait for the heavy scenario.
  • Updated the release target to 50000 shared files, kept the 10k observable node floor, and raised heavy-scenario waits to 900s for the main window and 1800s for row convergence.
  • Failed live smoke artifact: repos\emulebb-build-tests\reports\shared-files-ui-e2e\20260508-125931-eMule-main-release\tree-refresh-stress-10k\result.json. The hardened recursive profile still timed out waiting for the eMule main window with observable_node_count=11329, expected_row_count=1024, directory_count=10305, and shared_directory_count=10305; this is now tracked as BUG-101.
  • Passing live smoke artifact: repos\emulebb-build-tests\reports\shared-files-ui-e2e\20260508-170043-eMule-main-release\result.json. The aggregate Shared Files UI suite passed all six scenarios, including tree-refresh-stress-10k, dynamic-folder-lifecycle, and monitored-folder-events.
  • Passing 50k stress artifact: repos\emulebb-build-tests\reports\shared-files-ui-e2e\20260508-170043-eMule-main-release\tree-refresh-stress-10k\result.json. The scenario reached initial_row_count=50000, initial_rest_row_count=50000, final_row_count=50000, final_rest_row_count=50000, cached_relaunch_row_count=50000, and cached_relaunch_rest_row_count=50000.
  • Cold-vs-cached measurements from the passing artifact: cold_startup_ready_seconds=16.086, cold_ui_rows_ready_seconds=179.1, cold_hashing_done_seconds=196.305, cached_startup_ready_seconds=5.707, cached_ui_rows_ready_seconds=0, and cached_startup_ready_speedup_vs_cold_hash_done=34.397.
  • Resource deltas after churn from the passing artifact: gdi_objects=3, user_objects=2, handles=18, private_bytes=12918784, and working_set_bytes=17395712.
  • Test harness commit f79199e promoted the 50k tree-stress resource thresholds into hard pass/fail harness checks.
  • Passing 50k operator soak artifact: repos\emulebb-build-tests\reports\shared-files-ui-e2e\20260508-204401-eMule-main-release\tree-refresh-stress-10k\result.json. The scenario ran 160 churn cycles and reached initial_row_count=50000, initial_rest_row_count=50000, final_row_count=50000, final_rest_row_count=50000, cached_relaunch_row_count=50000, and cached_relaunch_rest_row_count=50000.
  • Soak cold-vs-cached measurements: cold_startup_ready_seconds=19.334, cold_ui_rows_ready_seconds=230.0, cold_hashing_done_seconds=250.548, cached_startup_ready_seconds=6.934, cached_ui_rows_ready_seconds=0, cached_queue_skip_verified=true, and cached_startup_ready_speedup_vs_cold_hash_done=36.133.
  • Soak resource thresholds passed with resource_thresholds.ok=true and zero violations. Observed post-churn deltas were handles +14, GDI objects +3, USER objects +1, private bytes +32296960, and working set bytes -155840512.
  • Validation:
  • python -m pytest tests\python\test_shared_files_ui_e2e.py -q
  • python -m pytest tests\python\test_live_e2e_suite.py -q
  • python -m pytest tests\python\test_shared_files_ui_e2e.py tests\python\test_live_e2e_suite.py -q
  • python scripts\shared-files-ui-e2e.py --help
  • python scripts\run-live-e2e-suite.py --help
  • python scripts\create-long-paths-tree.py --help
  • git -C repos\emulebb-build-tests diff --check
  • git -C repos\emulebb-build diff --check
  • python -m emule_workspace validate

Pending Release Evidence

  • None.

Relationship To Other Items