Skip to content

BUG 150

id: BUG-150 workflow: local github_issue: title: False multiple_names Caution on cosmetic search-name differences status: DONE priority: Minor category: bug labels: [search, fake-detection, confidence, 0.8.x] milestone: 0.8.0 created: 2026-06-11 source: 2026-06-11 operator review of Confidence "Caution: 15%" on same-content results

Problem

The Confidence column flags multiple_names ("meaningful name disagreement for the same hash") → Caution: 15% on search results that are clearly the same content under different release names. Divergence was decided by exact equality of the joined canonical name string (FakeFileDetectorSeams::Analyze), so three cosmetic differences each counted as a divergent name:

  1. token order differences (same words, swapped);
  2. one name being a subset of a more descriptive one (terse vs full description);
  3. a single stray release token present in only one name (e.g. a year that one copy keeps and another strips).

These are not evidence of a fake, so they should not lower confidence.

Decision / scope

  • Compare observed names by their content-token set, order-independently: two names are the same content when their token sets are equal, or the smaller set (>= 2 tokens) is a subset of the larger. Only flag multiple_names when

    = 2 genuinely distinct (mutually non-subset) groups remain.

  • Single-token names still require exact equality (no subset merge), so a one-word name vs a two-word name still diverges.
  • No change to the canonicalization/token-stripping rules, the noise-token lists, or the year heuristic; no REST/UI schema change; no localization.

Key files

  • srchybrid\FilenameTokenizationSeams.h (expose kept content tokens on CanonicalName), srchybrid\FakeFileDetectorSeams.h (Analyze: token-set fixpoint grouping + IsSameNameContent helper), repos\emulebb-build-tests\src\fake_file_detector.tests.cpp (regression cases with synthetic names).

Acceptance criteria

  • Same-content results that differ only by token order, a subset description, or a stray release/year token no longer flag multiple_names (Confidence "Looks good").
  • Genuinely different titles for the same hash (disjoint token sets) and single-token name mismatches still flag multiple_names.
  • Debug|x64, Release|x64, Release|x64 diagnostics build clean; native doctest suite (existing + new cases) passes; validate green.

Outcome (2026-06-11)

  • FilenameTokenizationSeams.h: CanonicalName now exposes the kept content tokens (unchanged stripping).
  • FakeFileDetectorSeams.h Analyze: divergence is decided by token-set grouping — IsSameNameContent (equal, or smaller set with >= 2 tokens is a subset of the larger) merged to a fixpoint; multiple_names fires only on >= 2 mutually distinct groups. Single-token names still require exact equality.
  • Tests: added three regression cases (stray-token, subset, reorder) using synthetic names per the commit content/privacy policy; existing real-divergence and single-token cases stay green.
  • Verification: Debug/Release/Release-diagnostics x64 builds clean; native doctest suite passes; validate green.

Follow-up (2026-06-11)

More operator cases showed equal/subset matching was not enough: same-hash results that share a large core but each add different descriptive words (cast/crew, genre), or carry per-name junk (an ed2k/UUID hash suffix, a season/episode marker, an episode subtitle), still flagged. Strengthened the divergence comparison:

  • FakeFileDetectorSeams.h: a SignificantNameTokens step drops non-identifying tokens before comparison — hash/UUID hex fragments, bare numbers, NNxNN season/episode markers, and a multilingual stopword set (English, Spanish, Italian, Portuguese, German). IsSameNameContent now merges on token overlap (a shared core of >= 2 tokens covering >= 60% of the smaller significant set) instead of strict subset, so cast/crew or genre-word differences no longer split a group. Names that reduce to no significant token are excluded from divergence. Disjoint names (genuine mislabeling) still flag.
  • Tests: added synthetic regression cases for hash-suffix + episode markers, a shared-core-with-different-cast example, and a multilingual connector example.
  • Also corrected a literal absolute-path example in the new privacy policy text that tripped the documentation-path audit.
  • Verification: 3 x64 builds clean; native doctest suite passes; validate green.

Follow-up 2 (2026-06-12)

Live testing still showed Caution on same-hash Italian releases. Two remaining gaps: the 60% overlap threshold split names whose extra cast/crew words dominated, and format tokens that were not stripped (1080i unlike 1080p, container tokens such as mkv/avi/mp4) inflated the sets. Final pass:

  • FakeFileDetectorSeams.h: added IsFormatNoiseNameToken (container/format tokens and interlaced/progressive resolutions like 1080i) to the significant-token filter, and relaxed IsSameNameContent to merge whenever names share >= 2 significant title tokens (same hash = same bytes, so a shared core means the same content). Near-disjoint names — and unrelated titles that share only format tokens — still flag.
  • Tests: added cases for a shared core buried in format/resolution junk (merges) and for two unrelated titles sharing only format tokens (still flags).
  • Verification: 3 x64 builds clean; native doctest suite passes; validate green.