I remember many years ago there was an award given for winning direct mail packages that remained a control for at least 5 years.
I always thought this was curious…was it a control for 5 years because it was so invincible or because the marketer just wasn’t testing enough?
Hopefully you are convinced from my previous ramblings about testing that you are now fully committed to testing more aggressively…and testing only things that are truly worthwhile (i.e. things that can give you significant lifts).
But two concepts that came up in recent mastermind groups I’ve attended have caused me some sleepless nights on the subject of testing.
One of the discussions revolved around when to violate a golden rule of testing; and the other approached some of the reasons why marketing sometimes gets a bad rap.
When to violate the discipline of single variable testing
Like so many of you, I was always taught to never test two things at once and to isolate “one variable” in order to prove whether that one change gave us a lift or not.
I think the rule still applies for the most part…especially when the one variable is price.
Even with some of the biggest tests I have ever done, where we hired a new copywriter with a completely different copy platform, we often kept the basic offer the same so we could accurately project the lift of the new package…or how far off it was from winning.
Then we got a little “sloppy” (of sorts)…with good reason…since the best copywriters knew that beating a strong incumbent promotion might involve re-structuring the offer (i.e. changing multiple elements in the control offer) to go along with their new copy platform.
We established that this kind of a test was neither single variable or multi-variable…it was simply a radically new approach altogether.
Well…that’s how we justified it at least.
It drove many direct marketing purists in my company crazy…but as long as we knew the game we were playing, I was OK with it.
Note: To keep the purists happy, when we were simply tweaking or editing a current control, we always kept single variable testing discipline in place.
The justification seemed sound to break from that discipline when going for the big winner, however.
The thinking was that to beat something that had been around for a while demanded breakthrough thinking.
And if we were in the situation where the control was unbeatable for 5 years or more, I had no interest in winning any awards.
I just wanted a new control.
I knew that if we needed huge leaps to get a winner, to try to do that one variable or element at a time, while hiring an expensive and scarce resource (i.e a new “A list copywriter”), was a daunting task.
It was so much more difficult to get a new blockbuster (winner), especially with a powerful, long standing control.
Fast forward to today…where we can call new controls every 15 minutes if we want to since we can test faster than ever with actionable results.
Remember–when I learned the rule of thumb of single variable testing in the 1980’s, I was living in a world of direct mail only.
In today’s world of testing (often in real time) online, there is a temptation to keep blowing things up every 15 minutes (because we can).
Where it got interesting during that recent mastermind discussion: I made the assertion that if you have big enough universes of names to test online, it might be wise to think even more about the discipline of single variable tests, albeit doing them more often and faster.
The temptation to test multiple variables and then get results that you think are statistically significant but are not is the danger I presented.
The discussion then went from interesting to a little controversial when we were critiquing a sales page and every item we wanted to test by itself seemed too small for a big lift.
That’s when I realized I needed to be a little more open minded and recalled “radically new approaches” trumping single variable tests when the situation called for it.
All single variable testing is not considered equal.
We ended up with some consensus in a room of very talented direct response marketers that big lifts on a page like the one we were looking at would demand big risks and therefore we might need to abandon single variable tests.
And yet, there were some elements that we also agreed that should be isolated in their own test panel.
What caused me those sleepless nights was the notion that a golden rule of testing in direct response was just too cumbersome, and abandoning it across the board might make sense, therefore shattering all of my hopes and dreams of being a responsible direct marketer.
Realizing I had been in this same place decades earlier and survived being aggressive without being reckless enabled me to sleep easier.
Recalling the Picasso quote I have shared with you numerous times in the past also helped: “Learn the rules like a pro so you can break them like an artist.”
I chalked up the discussion (and the conclusions) as “advanced direct marketing” rather than “sloppy direct marketing.”
This discussion also triggered an idea which I will tell you about in the P.S.—since I am now obsessed with doing this kind of analysis on all sorts of promotions in all media with as many marketers I can interact with.
“To single variable test or not to single variable test—that is the question.”
And of course the answer is exactly what you would expect: “It depends.”
When the end might justify the means
This second recent insomnia-causing mastermind discussion started when multiple people in a room of experienced marketers started arguing about the use of “fear” in marketing messages.
In the world I came from—mostly health and finance related promotions—fear was a staple.
We always justified fear (or paranoia) tactics because we had to rise above the noise in a crowded marketplace…and our product was too good for them to ignore.
I admit that that sounds a little fickle and maybe a little arrogant too.
However there was always a line I refused to cross…but it was my line, not anyone else’s line.
When I stopped short of what I thought was inappropriate or “too scary,” there were many that thought I had already crossed a line much earlier. That’s the nature of that beast.
And regardless of where I stopped pushing the envelope on fear, I often passed judgment on what I wouldn’t do that others were doing…they were going too far but not me!
There was no way I was being hypocritical since they went too far and I didn’t.
Yeah, right.
In reality, we were in a discussion about “taste” and not about whether the promotion was dynamic vs. inappropriate.
The more competitive the category…and health and finance are as competitive as it gets…the more we find ourselves in a predicament explaining how the end justifies the means.
Having a headline that “carrots might cure cancer” might be a way to guarantee that the prospect will read on…but if the payoff on such a statement didn’t have a study or real evidence to back up the claim, you will quickly lose your reader if they see through the hype.
It’s a fine line…and as I said above, the line is different for everyone, either from the marketer’s perspective or the prospect’s perspective.
The best rule of thumb I can articulate is to begin at the end.
That is, think about how valuable the product or service you are selling is to your audience and how much you believe that getting it into as many hands as possible is a game changer for them…and that you have almost a moral responsibility on your part to get it to them.
Do all of this while staying true to your principles and not losing sleep while making claims you feel are necessary to get your genius into the world in the biggest way.
This is always the marketer’s dilemma.
I’ve written at length in the past about “congruent marketing” and I guess the best conclusion I can make here is that there is not one standard for how much fear or paranoia is appropriate in the promotions we write.
But there is a standard for each of us…and finding what is most congruent for you (and your audience) is when marketing is always good and never evil.
I once told you the story of how I convinced an audience of 300 Hungarian entrepreneurs who were skeptical about marketing (especially from an American!) that marketing is not evil…and that they have the power to dial up or dial down their messaging based on their comfort level and how congruent the messaging is with everything else about their product, service…and mindset (i.e. how aggressive can they be and still sleep at night).
We all agreed that reaching millions with our life mission rather than dozens was worth taking a few risks in terms of being more aggressive with our messaging…but we also agreed that figuring out the line we will not cross under any circumstances is critical to establish at the outset.
And we also agreed that we have a responsibility to not simply create fear and paranoia, without facts and purpose to back everything up, in order to sell.
When we go overboard, everyone is hurt.
Nothing we do as marketers exists in a vacuum when we create millions of impressions that go out into the world.
The beauty of direct marketing, though, is that we can start with a blank sheet of paper and create anything we want in our own image…remain faithful to what we believe in…pull back when we can’t sleep…and then whatever we do, it’s all measurable.
I will leave you with this:
Test intelligently…test aggressively…and no matter what, look for the biggest rewards, not awards.
Never compromise your integrity.
But also never be afraid to be bolder in the spirit of sharing yourself (and your super powers) with the world.
Warmly,
Brian
P.S. As I was sitting in these mastermind meetings where I was both losing sleep AND learning valuable lessons at the same time, I realized how valuable this kind of discourse is to make everyone in the room a better marketer.
I also realized (and lamented) that so many people I want to reach (and teach) can’t afford to be in those rooms.
But what if I could expose many more of you, my online family, to learn and be exposed to all I am learning?
And maybe teach a few things along the way too?
Despite doing this direct marketing thing for over 35 years, I remain a student, and will be a student forever.
And even when I am teaching, I am learning.
My belief in the power of masterminding is accentuated by the fact that I am currently a member of 6 mastermind groups and I run two of my own.
My lament is that with the two that I run (Titans Mastermind and Titans Master Class), I can only reach a small group because the model for each involves live events.
With a goal to help a much larger group, I have an idea for a “monthly mastermind” which I am tentatively naming “Titans Accelerator.”
I’m thinking that for around $200 a month, I could create a group where I can share all I am learning through my investment of time and money in the masterminds I attend and host.
My current idea for this monthly program is to offer two group calls a month:
One would involve the kind of critiquing and “hot seats” I described above where everyone in the group has new epiphanies; and the other call would be wisdom (i.e. interviews, presentations) from experts I have direct access to and who are my “Professors” of direct response marketing.
Up until now, only the members of my higher priced Titans Mastermind and Titans Master Class have access to those experts on a regular basis.
There will also be a community (e.g. private Facebook group where we could share resources and ideas in between the calls).
And I am also thinking about a print newsletter which would take a deep dive on a specific direct marketing topic each month.
I would love to gauge interest with you and whether you would enroll in such a program at that price.
If you would like to be a charter member of this “Titans Accelerator Monthly Mastermind” when I open up enrollment, send me an email with a simple “Yes” in the subject line.