I have a confession: for years, I treated “campaign performance” like it was mostly about the email. The subject line. The preview text. The button colour. The vibe. The emotional support GIF. You know, the classics.
Then I learned the hard way that sometimes your email is fine, and the real problem is what happens after the click. Which is rude, because it means you can write a perfectly good email and still lose.
This post is a marketing data case study about one campaign that looked like it was doing okay on the surface, but was quietly leaking value in a single, fixable place. The team made one smart pivot based on a small insight, and the ripple effect was wildly disproportionate.
Also, it’s a reminder that “data-driven” does not mean “collect everything and stare at dashboards until you become one with the spreadsheet.” It means you find the one decision that actually matters, then you test it like an adult.
The real problem: when your data is “fine” but your results are not
Most struggling campaigns don’t look like a flaming dumpster. They look like this:
You have traffic. You have opens. You have clicks. You have meetings where someone says, “It’s not that bad.” You also have the gut feeling that your performance should be better, because you are not new at this, and you know what good looks like.
This is where teams tend to do two unhelpful things:
First, they optimize what’s easiest to touch, not what’s most likely to move the outcome. In the email world, that’s usually the inbox stuff, because it’s accessible, visible, and honestly more fun than fixing a form.
Second, they pick a metric that’s emotionally comforting. Opens, clicks, “engagement.” (If metrics were people, open rate is the friend who says “we should totally hang out soon” and never follows up.)
Email can deliver incredible ROI, but only when the whole system holds up. Litmus has reported that many companies see email ROI in the range of 10:1 to 36:1, which is great news for anyone trying to justify a budget without resorting to interpretive dance. But ROI is not a property of the email channel alone. It’s a property of the entire journey your email pushes people into.
So if your campaign is “fine” but not working, the question is not “How do I squeeze another 0.2% CTR?” The question is, “Where is the value falling out of the funnel?”
The case study: the $60 million pivot hiding inside one splash page
This is the part where I borrow a story from political fundraising, but I’m keeping it strictly about marketing mechanics, not politics.
Back in December 2007, Obama’s campaign team ran a test on their website splash page, basically the page where visitors could sign up. Dan Siroker (then Director of Analytics for the campaign, later co-founder of Optimizely) wrote up how they tested two elements: the media at the top (images/videos) and the call-to-action button copy.
They didn’t just do a polite A/B test either. They ran a full-factorial multivariate test across button copy and media options, creating 24 combinations total.
Here’s the punchline:
The winning combo was a “Learn More” button paired with a family image.
The winning variation hit an 11.6% sign-up rate versus 8.26% on the original page, a 40.6% improvement.
And this is where the “one pivot” gets spicy.
Siroker estimates that, if that uplift held over time, it would mean roughly 2.88 million additional email addresses captured during the campaign.
He also reports the campaign saw an average of $21 donated per email address submitted through the splash page, which he uses to estimate the experiment translated into roughly $60 million in additional donations.
That is an outrageous outcome from changing a button from “Sign up” to “Learn more” and swapping the hero media. Outrageous in the best way.
Also mildly humiliating for anyone (me) who has ever written a 2,000-word strategy doc when what we really needed was to test one decision on one page.
What the “small insight” actually was (and why it mattered)
The insight wasn’t “buttons are important.” That’s like saying “food impacts energy” and calling it a wellness breakthrough.
The insight was this: your audience’s hesitation is often not where you think it is.
The campaign staff originally favoured a specific video. The test showed that all the videos performed worse than all the images. That’s a big deal because it demonstrates a classic marketing problem: internal preference masquerading as user preference.
But the bigger psychological insight is in the button copy:
“Sign up” is a commitment. “Learn more” is a step.
When someone is on a splash page, especially for something they are not 100% sure about yet, “Sign up” can feel like you’re asking them to adopt a cat. “Learn more” feels like you’re asking them to pet the cat at a safe distance.
Same page. Same offer. Different perceived commitment.
This is where smart data use comes in. The team didn’t just look at a dip in sign-ups and decide to “improve the creative.” They identified the decision point that was blocking the outcome: the moment someone decides whether they trust you enough to give you their email.
That’s the pivot. Not the button. The pivot is choosing the right moment to optimize.
Why this is an email marketing story, even though it happened on a website
Because email does not exist in isolation. Email is the loudest voice in a group project where the landing page forgot to do the homework.
If your email is pushing people to a page that adds friction, confuses intent, or demands too much too soon, you will spend your life arguing about subject lines while your conversions quietly die in the background.
This is also why I’m constantly annoyed about “the metric you pick.” A click is not a value. A click is a handoff. The value is what happens next.
In the Obama campaign’s broader email program, A/B testing was treated as foundational, not optional. MarketingExperiments reported that testing email language and subject lines could create dramatic differences, including cases where the gap between the best and worst email was nearly $2 million in revenue. WIRED also describes how the campaign learned to trust testing over gut, citing extensive testing on donation pages and email elements like salutations (including that “Hey” worked well in some contexts).
So yes, the splash page test is “web,” but the downstream impact is list growth, and list growth compounds into email performance for months or years. That is lifecycle marketing in its natural habitat.
The One Pivot Framework: how to find the highest-leverage insight in your own campaign
Here’s the decision framework I use when I’m trying to avoid “random acts of optimization,” which is what happens when everyone is busy but nothing improves.
1) Start with the outcome, then work backwards one step at a time
Pick the outcome you actually care about. Not the proxy. Not the comfort metric.
If it’s revenue, say revenue. If the trials started, say trials started. If it’s donations, say donations.
Then walk backwards through the journey:
Outcome happens because of a conversion.
Conversion happens because of a completed action.
Completed action happens because of a decision.
A decision happens because the person feels safe, clear, and motivated enough to do the thing.
Your job is to find the weakest link, not the shiniest one.
2) Look for the biggest drop-off with the smallest effort to change
This is where “smart data use” becomes practical.
You do not need a perfect attribution model to notice that 5,000 people clicked and 12 people converted. You just need to admit that something is off, and then identify the most likely point of failure.
In the case study, the splash page was the gate to list growth. Increase sign-ups there, and everything downstream becomes easier.
That’s leverage.
3) Write one hypothesis that connects a user emotion to a measurable behaviour
This is the part most teams skip, and then they wonder why their tests are random.
A good hypothesis sounds like:
“If we reduce perceived commitment at the sign-up moment, more people will submit their email.”
That connects psychology to behaviour. It also tells you what to test.
In the case study, “Learn more” likely reduced perceived commitment compared to “Sign up,” which increased sign-ups.
4) Test the smallest change that proves or disproves the hypothesis
One pivot. Not seventeen.
If you test everything at once with no theory, you get results you can’t explain. If you test one lever tied to one hypothesis, you learn something you can reuse.
Also, your future self will thank you, because you will not have to explain to leadership why “Version C” won when “Version C” was fifteen changes duct-taped together.
5) Decide in advance what you will do if you’re wrong
This is a weirdly underrated part of ethical marketing. You are not testing to “win.” You are testing to learn.
If the hypothesis fails, what’s next? Do you test a different friction point? A different value prop? A different audience segment?
When you pre-decide next steps, you stop treating data like a personal attack.
(Your spreadsheet is not a personality test, even if it feels like one.)
Deepening the insight: why “Learn More” can outperform “Sign Up” in lifecycle terms
Let’s pull this into lifecycle marketing and email psychology, because that’s where this gets useful beyond one historical example.
People protect their future selves
Signing up is not just giving you an email address. It’s agreeing to future interruption. It’s allowing the possibility of regret. It’s inviting the risk of “oh no, now they’re going to email me like a brand that just discovered exclamation points.”
“Learn more” is psychologically safer because it keeps the door open. That matters at the top of the funnel, where motivation is fragile, and trust is low.
Small copy changes can shift perceived commitment more than you expect
Most copy discussions focus on persuasion: “How do we make this more compelling?”
But a lot of conversion is about removing invisible friction. “Sign up” can feel like a commitment. “Learn more” can feel like curiosity. The user’s brain often prefers curiosity because it feels reversible.
The first conversion sets the tone for every email that follows
If someone signs up reluctantly, your welcome email has to do the work of rebuilding trust from scratch.
If someone signs up because the experience felt safe and aligned, the welcome email gets to build momentum instead of doing damage control.
This is also why list-building tactics that rely on tricks, pressure, or hidden terms are so costly long-term. You might get the email address, but you lose trust. And trust is what drives clicks that turn into revenue, not just clicks that make a dashboard look alive.
How to apply this without turning your brand into a testing lab full of chaos
A quick guardrail, because experimentation can go off the rails fast.
First, optimize for clarity and consent. If your “winning” variation depends on confusing people, that is not a win. That’s borrowing value from your future deliverability.
Second, protect the customer experience. More testing is not always better if it creates inconsistent journeys or makes your message feel unstable.
Third, don’t get obsessed with what is easy to measure. Some of the most important drivers of email performance are qualitative: trust, relevance, and perceived respect. Data can guide you, but it does not replace actually caring about how it feels to receive your marketing.
Also, if your team starts saying “Let’s just test it” as a substitute for thinking, you have accidentally reinvented chaos with a Jira board.
Key takeaways from this marketing data case study
The point of this marketing data case study is not that Obama’s campaign was uniquely brilliant (though they clearly took testing seriously). The point is what the story reveals about how high-leverage data use actually works.
First, the biggest wins often come from fixing the highest-friction decision point, not from polishing the most visible asset.
Second, “email performance” is a system outcome. If the page after the click is weak, your email will take the blame anyway.
Third, a single experiment can compound for months when it increases list growth, because list size plus trust equals future revenue opportunities (and fewer sad post-mortems).
If you liked this, you’ll probably enjoy The Click Brief, my weekly newsletter about lifecycle marketing, email psychology, measurement that actually predicts revenue, and how to stay effective without getting weird about it.
Subscribe to The Click Brief, or come say hi on LinkedIn or Instagram and tell me what you’re testing right now. If your answer is “my patience,” honestly, fair.