Skip to content

Workflow status is tracked in GitHub: https://github.com/emulebb/emulebb/issues/146. This local document is retained as an engineering spec/evidence record.

CI-052 - RC2+ Installer-Backed Test Gate Rationalization

Summary

Rationalize release testing for RC2 and later candidates. RC1 is frozen historical evidence; current test campaigns and modules may evolve for RC2+ without preserving RC1 command or module compatibility.

The release gate should stay focused on supported release surfaces, reuse the same scenario definitions across host and VM execution, and start most release-relevant tests from the packaged PowerShell suite installer so package layout, profile materialization, scripts, manifests, and PDB placement are validated as part of normal proof.

Current Problem

Release testing has grown through independent lanes: local live E2E, reusable campaign scenarios, Windows VM profiles, package rows, and historical RC1 evidence. That makes it too easy for the host gate and VM confidence checks to drift, for tests to bypass the installer path, and for weak spots to compete with frozen legacy surfaces that should be ignored until removal.

RC2+ needs one explicit test contract:

  • RC1 evidence is frozen and no longer a target for active test compatibility.
  • Local and VM campaign scenarios should be mostly paritetic.
  • The installer-backed Bootstrap-eMuleBBSuite.ps1 path is the default start for release-relevant local/live scenario execution.
  • VM checks should reuse the same scenario/profile catalog and differ only where clean guest execution requires it.
  • Frozen surfaces stay excluded from release-gated testing unless a supported shared infrastructure path is affected.

Target Contract

  • Centralize the active eMuleBB release-test version in build orchestration.
  • Default release campaign, package, local install, live E2E materialization, campaign scenario, and Windows VM options to the active RC2+ version.
  • Keep campaign manifests explicit about the target RC version, especially package commands and expected package manifest evidence.
  • Make local reusable campaign scenarios installer-backed by default.
  • Keep Windows VM rows as confidence evidence unless a package/installer delta invalidates clean guest proof and explicitly promotes the row.
  • Allow campaign schemas, scenario modules, and tests to evolve for RC2+ without retaining RC1 compatibility shims.

Acceptance Criteria

  • Active release/test defaults target 0.7.3-rc.2 or a single active-version constant, not scattered RC1 literals.
  • emulebb-0.7.3 and emulebb-0.7.3-overnight manifests target RC2+ and package commands pass the release version explicitly.
  • Local reusable campaign scenarios enable installer materialization and forward the scenario release version into the materialized install.
  • Release campaign execution forwards explicit package release versions and clean flags to package creators.
  • Host and VM scenario command generation share the same catalog and release version defaults.
  • Release strategy docs state that RC1 is frozen, RC2+ campaigns may evolve, frozen surfaces are excluded, and VM rows are mostly nonblocking confidence unless promoted by a package/installer delta.
  • Focused Python tests cover the changed orchestration and manifest contract.

Evidence

Completed on 2026-06-07.

  • repos/emulebb-build centralizes the active release/test default as ACTIVE_EMULEBB_RELEASE_VERSION = "0.7.3-rc.2" and uses it for package, local install, live E2E materialization, reusable campaign scenarios, cleanup, and Windows VM options.
  • repos/emulebb-build-tests campaign manifests target 0.7.3-rc.2, keep reusable scenario host and VM commands tied to the shared scenario catalog, and leave VM proof nonblocking/on-demand unless explicitly promoted.
  • Local reusable campaign execution materializes installer-backed test installs by default, forwarding the active release version into the packaged PowerShell suite installer path.
  • Release strategy and campaign docs now state that RC1 evidence is frozen, RC2+ campaign modules may evolve, frozen surfaces are excluded, and local/VM rows should stay mostly paritetic.
  • Validation evidence from the implementation slice: python -m pytest tests -q in repos/emulebb-build passed with 425 passed; python -m pytest tests/python/test_release_campaigns.py tests/python/test_campaign_scenarios.py -q in repos/emulebb-build-tests passed with 29 passed; git diff --check passed for touched repos.