Simulation called it three times out of three

Q3 audience-simulation pilot results, and the case for a ten-account expansion.

Q3 pilot review2026-07-07Suraj · Socialtrait

Summary Q3 pilot reviewConfidential

Three pilots, one pattern: simulation called every winner

Enterprise pilots in CPG, fintech and retail — 12 audience cells, April to June 2026.

Winners called
3of 3
every pilot, before launch
Interval hit rate
92%
11 of 12 cells inside the forecast
Time to insight
4hrs
panels took 19 days median
Media saved
$494k
$340k from one flagged creative
Takeaway

Simulation is accurate enough to gate media spend today.

01 Accuracy
Accuracy Q3 pilot reviewConfidential

The forecast tracked real sales lift for eight straight weeks

Largest weekly miss
0.4pts
+8+6+4 +20 Wk 123 456 78 Market 7.4 Forecast 7.1
Fig 01 Weekly sales lift vs. control, CPG pilot, points. Source: retailer scan data
  • Forecast locked before launch; no re-fitting after week 1
  • 0.3 pts mean absolute error across all eight weeks
  • 7 of 8 week-on-week turns called in the right direction
Takeaway

A locked forecast held for two months: this is prediction, not curve-fitting.

Accuracy Q3 pilot reviewConfidential

The winner held in every category, not just the easy one

PilotCalled before launchMarket winner CellsMean error, ptsMedia saved
CPGBeverage launchConcept BConcept B 40.3$96k
FintechCard acquisitionOffer AOffer A 40.6$340k
RetailSeasonal campaignFilm CFilm C 40.4$58k
All pilots3 of 3 120.4$494k
Takeaway

Fintech, the hardest call, paid for the whole pilot on its own.

Accuracy Q3 pilot reviewConfidential

Simulated shoppers explained why each winner won

MK
Maya, 24
Simulated Gen-Z shopper · CPG pilot
Price-awareTrend-ledMobile-first
Concept B looks like something my friends would post. A looks like my dad’s gym drink.
— Maya · Simulated Gen-Z shopper
DO
Daniel, 38
Simulated card switcher · Fintech pilot
Rate-sensitiveTwo cardsSkeptic
Cashback I understand. Points I’ve never once redeemed.
— Daniel · Simulated card switcher
Takeaway

Every prediction arrives with a reason a creative team can act on.

Speed Q3 pilot reviewConfidential

Panels report after the decision. Simulation reports before it.

A 19-day panel lands after a 9-day creative cycle has already shipped. A 4-hour study lands inside it.

Speed Q3 pilot reviewConfidential

Answers land inside the campaign cycle, not after it

Median time to insight
4hrs
Field study Online panel Socialtrait 28 days 19 days 4 hours
Fig 02 Median question to segmented result. Source: pilot instrumentation
  • 9-day median creative iteration cycle at pilot clients
  • Same-day re-tests after every creative revision
  • Studies run overnight; results land before stand-up
Takeaway

Research finally moves at the speed the campaign already does.

02 Expansion
Expansion Q3 pilot reviewConfidential

$53k and one engineer buy a ten-account proof by October

Budget requested
$53k
  1. JulyOnboard four Pipeline accounts, eight studies, same instrumentation
  2. AugustFour new logos Twelve studies and a first pricing test
  3. SeptemberTwo validations Enterprise studies scored against media holdouts
  4. OctoberReadout Accuracy dashboard reviewed against the pre-registered bar
The ask

Approve budget and headcount by July 18 to finish before Q4 planning.

Expansion Q3 pilot reviewConfidential

Ten accounts is the smallest test that could prove us wrong.

The bar is pre-registered: 80% interval hit rate across 30 cells, or we stop.

Decision requested Q3 pilot reviewConfidential

Expand the pilot to ten accounts this quarter.

$53k and one engineer, July to October, scored against the same accuracy dashboard.

Approve by July 18

suraj@socialtrait.aiPlan detail in ST-OP-014