By introducing a gamified “Mystery Box” popup instead of a standard evergreen version, we believed the surprise-and-delight mechanic would intrigue visitors, boost engagement, and ultimately drive higher email and SMS opt-in rates. We also expected that excitement might translate into higher orders and revenue.

Tests were conducted across two verticals:

‍

• Beauty Brand – A controlled test comparing evergreen vs. “Mystery Box” scratch-to-reveal popup.
• Candle Brand – Comparison of a Mystery Box popup against evergreen seasonal popups.

‍

Two versions were compared:

‍

‍

Key metrics tracked:

‍

• Email Opt-in Rate
• SMS Opt-in Rate

‍

Conclusion:

For both the beauty and candle brands, the Mystery Box popup did not deliver the expected gains.

‍

• Beauty Brand: The sharp drop in email (−24.5%) and SMS (−24.8%) opt-ins means the net effect on list growth was negative.
• Candle Brand: The Mystery Box format also reduced opt-ins (−9.7% email, −6.9% SMS) compared to evergreen popups, resulting in fewer overall subscribers.

‍

Key Takeaway:

‍

Gamified popups like Mystery Box can sometimes create distraction or friction instead of excitement. For these brands, the evergreen popup format proved more reliable at consistently driving signups. The lesson: not all gamification mechanics resonate equally — what feels playful to some audiences may feel gimmicky to others.

‍