One of the most important lessons I learned from Dick Benson, the man who knew more about the science (and art) of direct mail than anyone in history, was to spend more time on lists.
And to this day, the thing I teach the most is the importance of list research and segmentation as the cornerstone to direct response marketing success, online or offline.
“When 41% is a majority” spoke about that in detail.
But another maxim that Dick drilled into me was this:
“No marketer tests enough.”
I think if Dick were alive today he would be excited that technology has enabled us to test more frequently, faster and much more inexpensively than we ever could have dreamed in the days before the Internet.
But with all that excitement, I think he would have identified a new problem…and one that I see a lot in my travels today.
I can even hear him now:
“Do you and all of your clients understand statistical significance enough to call the winners of the tests you are doing accurately?”
“Statistical significance” by itself is not the topic I want to talk about today…I’ll just get agitated if I share the story of the marketer who said to me in the middle of an online launch:
“We’ve got a winner with test A over test B; Panel A has 12 orders and Panel B has 8. This is great!”
On a $97 offer, I can assure you that 12 orders vs. 8 orders does not mean you have a statistically valid winner…
Oops…I just told you the story…now we absolutely should talk about something else.
It is related however.
Today’s topic:
Are we testing enough AND are we testing the right things?
I like to combine Dick Benson’s, “No marketer tests enough” with another maxim from Gordon Grossman, who was very wise too:
“Don’t make tiny tests.”
Gordon was the architect of Reader’s Digest in the 1960’s and one of the originators of the sweepstakes (Groupon and others owe a debt of gratitude to Gordon).
See the P.S. about a special offer on the classic books from Benson and Grossman…I still have a few copies of each in inventory.
Testing a lot but also testing the things that can give us significant lifts in response, revenue or profit (or a combination of all of those) is what it is all about.
Here’s a tip: Whenever you are in a brainstorming meeting discussing things to test, after you get all of the possibilities written on a whiteboard, posted all over the room on sticky notes, or written in crayon on the wall, look to test things that you really think could give you at least a 30% lift (in whatever you see as your key metric).
It will push you harder not to make “tiny tests.”
I talked about this critical concept on stage when I kicked off the epic event “Titans of Direct Response” (I can’t believe that was over two years ago)!
In my opening session at “Titans” while discussing one of the key pillars to becoming extraordinary (“Always Have Insatiable Curiosity”), I put this cartoon up on the screen:
While it looks like I am in favor of putting marketing people in weighted sacks at the bottom of a polluted river, it is quite the opposite.
I used the three marketing geniuses in those sacks to create my “Direct Marketing Success Formula”:
“I could sell this” is Curiosity (and the importance of dreaming)…
“It could work” is Optimism…
“Numbers. I need numbers” is Measurement
Without this formula, companies like the one I helped build (and so many others in direct marketing) would spend more time strategizing than implementing.
Thank goodness that is not the case.
Failing fast and moving on to the next test is my preference over not testing at all because “it’s not ready.”
I’m sure you’ve heard, “Done is always better than perfect.”
Of course you always have to assess the downside risks of faster tests (and more tests); but when there is a chance for a big winner (i.e. it’s NOT a “tiny test”), get it out there quickly.
I believe people who tell you they are perfectionists are really people who are afraid to fail.
In addition, with direct response marketing, where everything is measurable and containing costs is always part of the equation anyway (since everything has to eventually pay out), there is no such thing as a “failure.”
You either win or you learn.
It’s either a breakthrough or it’s education.
My experience has taught me that you can often have one success out of ten and that one winner pays for the nine failures many times over.
And those of you who are new to my list, may have missed it when I spoke about my biggest blunders of the last 35 years in two previous posts.
Click here for one and click here for the other…they were doozies.
In your marketing brainstorming meetings, always have a spirit of ,“There is no No” when it comes to new ideas…and keep the ideas in the air as long as possible when deciding what to test.
Remember:
Curiosity + optimism + measurement = Direct Marketing Success
Since you know you aren’t going too far down the road without numbers that make sense, risks might be lower than you think…that’s why I love direct marketing so much.
Let me give you three quick examples.
=========
Example 1
=========
If we stopped testing “Magalogs” (which were a new format in direct mail in the 1980’s, 16 to 32 page “faux magazines” that were long copy promotions) after our first dismal failure, we never would have figured out how to make the format work.
They were one of the most successful formats in direct mail at the time…still working today…a format my previous company Boardroom has mailed in the hundreds of millions.
In fact, Dick Benson said to me after that first terrible mailing:
“Self-mailers (like Magalogs) don’t usually work in direct mail…but you, Brian, should keep testing Magalogs. You have a longer story to tell about an unknown brand so it’s a logical format. It may be high risk but it is also high reward.”
And if Gordon was part of that conversation, he would have said, “Testing and re-testing a Magalog is NOT a tiny test.”
=========
Example 2
=========
If we thought direct response television wasn’t for us after multiple failures on TV, we would have missed out on creating an incredibly lucrative infomercial business which was one of our greatest success stories ever…I chronicled that entire process in a previous post about how we never gave up on TV.
========
Example 3
=========
If we continued to only create books from our own content, we would have run out of content…and lost an opportunity to create one of the most successful book marketing operations in the history of direct mail.
That story is detailed in “Books are still a perfect product.”
These examples of what I call “relentless testing” not only seemed logical at the time, they also had all of the elements that create direct marketing success:
Curiosity (dream)…Optimism (evidence that it works)…Measurement (numbers that tell you it did work)
When we embark on these tests..and then persisted, there was more than just anecdotal evidence that the tests had the chance to become major breakthroughs…but we had to prove it with flawless execution and numbers that didn’t lie.
Another way to express it:
“Throwing spaghetti up against the wall” (which sounds a bit negative and random) is another way of saying, “How can we test a lot of new things quickly and intelligently… with an eye on hitting a home run every time…but being happy with singles and doubles too?”
It drives me crazy when I see companies spend too much time on planning and not enough time on doing.
Using the disciplines and eternal truths of direct marketing, where everything is testable and measurable, testing aggressively (and not stupidly) is the only game worth playing.
Spaghetti against the wall can get messy…but keeping it in the pot waiting for the water to boil when there’s no fire under it kinda sucks.
And it’s not much fun either.
And you may also go hungry.
When testing, think education (“what will we learn”)…or breakthrough (“what will we create that’s big, beautiful and profitable”)…or both.
Now that’s fun.
Warmly,
Brian
P.S. As I mentioned while quoting Dick Benson and Gordon Grossman above, I still have a few copies of their classic books, Secrets of Successful Direct Mail and Confessions of a Direct Mail Guy respectively.
And if you think these two books are just about old school direct mail, think again.
When you read them you will see how applicable everything these two men talk about is relevant to all we do online today.
Click here for a special offer on both books while I still have copies available.