I recall many years ago there was an award given for winning direct mail packages that remained a control for at least 5 years.
I always thought this was curious…was it a control for 5 years because it was so invincible or because the marketer wasn’t testing enough?
[NOTE: For this post, I’m defining “control” as the “best/winning package or promotion” …either because it’s the first one you are testing…or it has stood the test of time against all comers.]
Hopefully you are convinced from my previous ramblings about testing that you are fully committed to testing aggressively…and testing only things that are truly worthwhile (i.e. things that can give you significant lifts in response and profit).
In the spirit of heading into a new year (happy 2024 to my online family!), I am reminded about two rules of thumb (related to testing) which I want to share with you today…both of which caused me some sleepless nights…but no more.
One is about when to violate a golden rule of testing; and the other is about approached the reasons why marketing sometimes gets a bad rap.
When to violate the discipline of single variable testing
Like many of you, I was taught to never test two things at once…and to always isolate “one variable” to prove whether that one change gave us a lift or not.
I think the rule still applies for the most part…especially when the one variable is price.
Even with some of the biggest tests I have ever done, where we hired a new copywriter with a completely different copy platform, we often kept the basic offer the same so we could accurately project the lift of the new package…or how far off it was from winning.
Then we got a little “sloppy” (of sorts) …with good reason…since the best copywriters knew that beating a strong incumbent promotion might involve re-structuring the offer (i.e. changing multiple elements in the control offer), and the format and design, to go along with their new copy platform.
We established that this kind of test was neither single variable nor multi-variable…it was simply a radically new approach altogether.
Well…that’s how we justified it. 🙂
It drove many direct marketing purists in my company crazy…but as long as we knew the game we were playing, I was OK with it.
Note: To keep the purists happy, when we were simply tweaking or editing a current control, we always kept single variable testing discipline in place.
The justification seemed sound to break from that discipline when going for the big winner, however.
The thinking was that to beat something that had been around for a while demanded breakthrough thinking.
And I had no interest in having a control become unbeatable for 5 years, with minimal improvement, no matter what award we could win.
The only award was the reward of a new control.
We lived by the expression, “The control is your enemy.”
I knew that if we needed huge leaps to get a winner, to try to do that one variable or element at a time, while hiring an expensive and scarce resource (i.e. a new “A- list copywriter”), was a daunting task.
And it was much more difficult to get a new blockbuster (winner), especially with a powerful, long-standing control, also written by an A-list copywriter.
It was a very competitive marketplace…and it’s even more so today.
Fast forward…today we can call new controls quickly off any test since we can test faster than ever with actionable results.
When I learned the rule of thumb of single variable testing in the 1980’s, I was living in a world of direct mail only. I get it.
But this is not dinosaur thinking and not a reason to abandon the rule completely.
In today’s world of online testing (often in real time), there is a temptation to keep blowing things up every 15 minutes (because we can).
Just because you can, doesn’t mean you must.
This got even more interesting during a discussion at one of my masterminds:
I made the assertion that if you have large enough universes of names to test online, it might be wise to think even more about the discipline of single variable tests, albeit doing them more often and faster.
The temptation to test multiple variables and then get results that you think are statistically significant (but are not) is the danger I presented.
The discussion then went from interesting to controversial when we were critiquing a sales page and every item we wanted to test by itself seemed too small to create a big lift on its own.
That’s when I realized I needed to be a little more open minded and recalled from my direct mail days “radically new approaches” trumping single variable tests when the situation called for it.
All single variable testing is not created equal.
We ended up with some consensus in a room (of very talented direct response marketers) that big lifts on a page like the one we were looking at would demand big risks and therefore we might need to abandon single variable testing.
And yet, there were some elements that we also agreed should be isolated in their own test panel.
What caused me those sleepless nights was the notion that a golden rule of testing in direct response was just too cumbersome, and abandoning it across the board might make sense, therefore shattering all my hopes and dreams of being a responsible direct marketer.
Realizing I had been in this same place decades earlier and survived being aggressive without being reckless enabled me to sleep easier.
Recalling the Pablo Picasso quote I have shared numerous times in the past also helped:
“Learn the rules like a pro so you can break them like an artist.”
I chalked up the discussion (and the conclusions) as “advanced direct marketing” rather than “sloppy direct marketing.”
The discussion above, which took place at my Titans Mastermind almost five years ago, triggered an idea at the time which I will tell you about in the P.S.— in the spirit of sharing marketing topics like this with a wider audience of the best and brightest direct response marketers and copywriters.
I also became obsessed with doing this kind of analysis on all sorts of promotions in all media with as many marketers I can interact with.
“To single variable test or not to single variable test—that is the question.”
And of course, the answer is exactly what you would expect:
“It depends.”
When the end might justify the means
My second bout with “testing insomnia” occurred when another Titans Mastermind discussion took a turn for the worse when the topic became, “how can we use fear with integrity in our marketing messages.”
In the world I came from—mostly health and finance related promotions—fear was a staple.
As was paranoia.
In fact, we had t-shirts printed that said:
“Paranoia is not a Psychosis…it’s Survival”
We justified using fear…and paranoia…because we had to rise above the noise in a crowded marketplace…and our product was too good for them to ignore.
I admit that sounds a bit arrogant.
So be it. It was our truth.
However, there was always a line I refused to cross…but it was my line, not anyone else’s line.
When I stopped short of what I thought was inappropriate or “too scary,” there were many that thought I had already crossed that line much earlier.
That’s the nature of that beast.
And regardless of where I stopped pushing the envelope on fear, I often passed judgment on what I wouldn’t do that others were doing…they were going too far…but not me.
Our discussions were more about “taste” rather than whether the promotion was too aggressive or bordering on inappropriate.
It was more about how the prospect perceived it more than anything else.
The more competitive the category…and health and finance are as competitive as it gets…the more we found ourselves in a predicament explaining how the end justifies the means.
Having a headline that “carrots might cure cancer” might be a way to guarantee that the prospect will read on…but if the payoff on such a statement didn’t have a study or real evidence to back up the claim, you will quickly lose your reader if they see through the hype.
Not to mention who is policing you…there are as many policemen as there are promotions.
Facebook, Amazon, Google and the good old United States Postal Service each have lines that they won’t let us cross…whether justified, arbitrary or a mixture of both.
That’s why “compliance” has become a marketing term, defined as:
com·pli·ance
noun
- the action or fact of complying with a wish or command.
“they must secure each other’s cooperation or compliance”
Whose wish or command? That depends.
All I know is that while we can’t have lawyers run our businesses, a lawyer who knows the ins and outs of compliance—and marketing—can be your best friend.
And your most valuable marketing partner.
It’s a fine line…and as I said above, the line is different for everyone, whether from the marketer’s perspective or the prospect’s perspective…or the owner of the medium.
The best rule of thumb I can articulate is to begin at the end.
That is, think about how valuable the product or service you are selling is to your audience and how much you believe that getting it into as many hands as possible is a game changer for them…and that you have a moral responsibility on your part to get it to them.
Stated that way, it seems less arrogant. 🙂
With this philosophy, don’t edit on your first draft…or your second…or your third.
You can always scale back from your most dynamic, albeit risky, effort.
Do all this while staying true to your principles and not losing sleep while making claims you feel are necessary to get your genius into the world in the biggest way.
And then check with your attorney when you have your final draft.
This is always the marketer’s dilemma.
I’ve written at length in the past about “congruent marketing” and I guess the best conclusion I can make here is that there is not one standard for how much fear or paranoia is appropriate in the promotions we write.
But there is a standard for each of us…and finding what is most congruent for you (and your audience) is when marketing is always good and never evil.
I once told you the story of how I convinced an audience of 300 Hungarian entrepreneurs who were skeptical about marketing (especially hearing all about it from an ugly American…me!) that marketing is not evil…and that they have the power to dial up or dial down their messaging based on their comfort level and how congruent the messaging is with everything else about their product, service…and mindset (i.e. how aggressive can they be and still sleep at night).
We (me and the Hungarians) agreed that reaching millions with our life mission rather than dozens was worth taking a few risks in terms of being more aggressive with our messaging…but we also agreed that figuring out the line we will not cross under any circumstances is critical to establish at the outset.
And we also agreed that we have a responsibility not to create fear and paranoia, without facts to back everything up…and purpose.
When we go overboard, someone will gut hurt…and maybe even go to jail.
Nothing we do as marketers exists in a vacuum when we create millions of impressions that go out into the world.
The beauty of direct marketing, though, is that we can start with a blank sheet of paper and create anything we want in our own image…remain faithful to what we believe in…pull back when we can’t sleep…and we know that whatever we eventually come up with, it’s measurable.
I will leave you with this:
Test intelligently…test aggressively…and no matter what, look for the biggest rewards, not awards.
Never compromise your integrity.
But never be afraid to be bolder in the spirit of sharing yourself (and your superpowers) with the world.
Warmly,
Brian
P.S. If you have a high tolerance for discussions like the one in the post above…that is, you like disagreements without being disagreeable…with the best marketers and copywriters on the planet…Titans Xcelerator is worthy of your consideration.
Anywhere from two to four times a month, the Xcelerators hop on a live Zoom call for timely discussions on anything and everything happening in media, copy, marketing today…along with guest speakers (only the best-of-the-best) …plus speakers from within the group (“Titan Spotlights”) …and breakout rooms where you can find accountable partners (and friends).
There are a lot more benefits of being a member: Discounts on all the products I sell, a private Facebook Group adding to the live discussions and accountability, and a mailed package with recordings of the calls and the highlights of the month (on a thumb drive).
It is truly the best value in masterminds anywhere.