A benchmark needs a source tier, not only a percentile

I contributed to Media Benchmarks across its dashboard, advertiser overview, filtering, conversion-rate coverage, calculation services and maintenance. The operator's question sounds simple: how does this campaign compare with similar campaigns? Answering it requires more than placing a dot on a curve.

The comparison is only as meaningful as the cohort behind it.

Make the fallback ladder inspectable

The calculation service extracted in the authored change centralizes platform mappings, baseline values, vertical adjustments, deterministic simulation and percentile interpolation. Its benchmark resolver tries vertical plus spend tier, vertical without spend tier, global plus spend tier and global without spend tier. It then uses configured or built-in manual fallback values.

The resolver returns the selected tier with the metrics.

I would keep that tier attached to every explanation. A global or manual fallback can keep the tool usable when a narrow cohort is missing, but it changes what the comparison means. A configured value is not a newly measured cohort, and deterministic simulation is not observed campaign performance.

The inspected compare route checks for impressions and clicks in the warehouse schema. If that check fails or those columns are missing, it falls back to simulation. Its baseline and seeded generator make repeatable examples possible; they do not supply external validation. My case study does not treat simulated ranks as client results.

Inspect the system diagram

Open full-size SVG →Horizontal version →

Resolve campaign identity before comparing it

The tactic service uses a campaign-and-connector mapping when available. It otherwise matches campaign-name keywords and finally returns Other. Platform connector strings are normalized separately. That makes explicit mapping the first choice while retaining a usable heuristic fallback.

I can explain why the heuristic needs scrutiny.

Substring rules can misclassify names, and the order of keyword checks matters. The benchmark percentile function interpolates between three thresholds and caps its upper tail. It ranks the numeric value; that alone does not determine whether a higher percentile is desirable for every metric. Cost metrics and rate metrics require different operator interpretations.

My other authored changes added CVR coverage and overview columns, repaired tactic names containing slashes in recalculation IDs, connected trend displays and fixed filtering and admin-state problems. Those changes belong in the evolution of one tool. They are not separate claims that I created every benchmark data source.

Cache the explanation with the right invalidation story

The insights cache hashes advertiser identity, campaign name, spend, rate and cost metrics, and vertical. Numeric values are rounded to defined precision before hashing. A matching entry can be reused for seven days. Storage errors are logged and treated as a miss or a failed write.

The cache key is a design decision.

I would explain both what it tracks and what it leaves out. The inspected hash does not include a model version, prompt version or selected benchmark-cohort revision. Those inputs can change the interpretation even when campaign metrics remain the same. The code supports a reuse policy; it does not prove that reuse is always semantically fresh or that token spending fell by a measured amount.

The authored change later repaired a missing document-reference getter in a recalculation test mock. That is maintenance evidence. An authored or repaired test is not a fresh passing test run in this research session.

Invented planning walkthrough

1. Select fictional campaign Harbor Search with an explicit campaign-to-tactic mapping.
2. Remove its vertical-and-spend cohort. Show the resolver selecting the next available tier and retaining that tier label.
3. Remove every stored cohort. Show manual fallback values as configured context, not empirical campaign measurements.
4. Reconstruct missing warehouse performance columns. Label the resulting performance example as simulated.
5. Change the rounded campaign metrics and show a different insights key. Then explain why a changed prompt or cohort would need broader invalidation.

I would use this demonstration to discuss provenance before showing a polished curve. It contains no customer metrics or live warehouse query.

What changed, and what remains unmeasured

My contribution record and inspected source support shared calculation services, explicit fallback resolution, mapping precedence, additional metrics and UI repairs. The available evidence does not establish statistically representative cohorts, predictive validity, operator adoption or improved campaign performance.