November 28, 2020

Thinking back on Thanksgiving’s past (don’t worry, this is not a Scrooge flashback!), I recalled the “post-Thanksgiving/before Christmas period” during my  freshman year in college—when I was thinking about my first set of final exams followed by an entire month of vacation.

Unfortunately that led me to that same period after my senior year when I asked my first employer:

“Do we get the entire month of January off for vacation?”

To which he responded: “Yes, of course. As long as you want February off as well. And beyond. Permanently.”

That was the moment I realized I wasn’t in Kansas anymore.

Let’s go back to my freshman year–not in Kansas but in New Jersey–and obviously not in the real world where we have to work in January.

What was on my mind leading up to those first final exams?

Not the exams (unfortunately) but rather these two things:

1. What was I going to major in?

2. What would be the #1 rock song of all time from the survey sponsored by the on-campus radio station? (Yes, I was equally obsessed with this…sadly)

Little did I know…

…that #1 would lead to a weekly blog post titled, Don’t go it alone; but more importantly a blessed career of 40 years (and counting)…

…and #2 would lead to my first lesson in “statistical significance”–without a clue that it would be a fundamental, bedrock principle for that same blessed career, a career that I thought would be one of umpiring baseball in the major leagues, being a film critic or becoming a college English professor…anything but a marketer. Yuck.

My thinking changed about that in a big way (obviously). 

And I bet yours did too.

How many of you “fell into marketing?” I’m positive you were not voted “most likely to be an online funnel builder” in your high school or college yearbook, correct?

Tell the truth now. ☺

It was abundantly clear that I wasn’t going to learn about statistical significance in my English classes nor was I going to learn about it in my marketing classes.

As I recall there was one paragraph in my Introduction to Marketing textbook on anything remotely related.

It was actually a very short paragraph.

To be precise it was one sentence.

The sentence casually mentioned the term “direct marketing”…but nothing about how advertising and marketing could actually have a return on investment, where everything is testable and statistics and numbers in marketing mattered most.

That would need to be part of a second paragraph…or sentence.

It never appeared.

I learned in that textbook, however, all about image advertising, brand advertising and other flavors of non-measurable advertising…even “impulse buying at retail” which helped me as an “Inventory Specialist” (euphemism for “stock boy”) at the liquor store I worked at all through college.

Based on this knowledge, I set up the most kick-ass and strategic displays of the bottles of wine on clearance and quick-sale items in places where there was more store traffic.

But heaven forbid that I had any instruction about creating any metrics to track those impulse sales.

Anyway, the last thing on my mind was “statistical significance” nor was it on the minds of the folks who worked at WRSU-FM, that campus radio station.

I think the only statistic that was on my mind as a freshman was how many unemployed English majors there would be by the time I was a senior; and whether my parents would pay the tuition for my sophomore year once they found out my evil plans.

Those plans included not only to major in English but also to participate fully in the survey of the “the best rock song of all time.”

Those of you who ever participated in voting on the top rock songs of all time (this one was from 1976)  number one was always Led Zeppelin’s Stairway to Heaven and number two was always Layla by Derek and the Dominos (featuring Eric Clapton). 

I can’t even recall what song usually ended up a distant third.

And don’t ask me what the list looks like now. You can tell me if you like.

But back then, I was sick of the same results all the time.

As my protest, I stuffed the ballot box with a song and band I liked at the time: Running Hard by Renaissance.

Renaissance was popular but no one was confusing them with Led Zeppelin or Eric Clapton…or any high profile rock band at the time.

And Running Hard wasn’t even their biggest hit…that would be Carpet of the Sun (which I’m sure you all knew). ☺ 

You probably never heard of the group or either song…which makes this story even more intriguing. Hopefully you’ve at least heard of Stairway To Heaven or Layla

Note about Renaissance: Annie Haslam, the lead singer, has an amazing voice with a five octave range—she’s an opera singer singing rock and roll. 

And she still sings and rocks today—and I was fortunate to see a reunion concert two years ago.

Anyway, I spent the two weeks while “polls were open” voting for Running Hard, as hard as I could (pun intended) as the top rock song of all time.

And I voted a lot…multiple times…and every day.

What else did I have to do except study for finals?

I assumed they would eventually ignore the votes…but illogically, they counted every one as legit. Talk about voter fraud.

I attended a highly respected, and quite large, state university known for mediocre football and basketball teams (actually awful most years)…but not known to be a school full of dummies.

The students running the radio station were an exception and they might have been decent DJ’s…but not much else.

Even with my diminished math skills at the time, I could have figured out this mystery.

But little did I know, I was getting my first lesson in statistical significance—and not from a classroom. 

So what did my college radio station ultimately do?

When they counted down, the number three song of all time was Layla, number two was Stairway to Heaven and number one was, yes, Running Hard.

I remember telling my friends that I felt really good about being at a school with so many Renaissance fans on campus.

What a pleasant surprise!

Now I just had to find them. ☺

Fast forward to the first online launch I ever did (sometime around 2008 or so). 

I recall we were testing two offers in real time, online…which was very exciting since the usual timeline up until then (offline) was that we needed to wait 12 weeks to get actionable results while doing split testing in direct mail.

I remember we were testing a particular variable in the copy and a few hours into the test, we had 12 orders for one offer and 8 for the other which caused our launch expert to exclaim:

“We’ve got a winner!” 

This was on a $97 offer mind you…there was still more time to run the test (in addition to the numbers being so small)…so of course I would not let our expert roll out with his proclaimed “winner.”

I used these two extreme examples to emphasize the need to be educated in how to test effectively and accurately in direct marketing…something we could take a deep dive on here but it’s all in my book, Overdeliver, which many of you have and the rest of you should have.

Sorry…my policy is that there are no “shoulds”…so it’s just a recommendation.

The first rule of thumb I learned about statistical significance was when Dick Benson (the smartest man at the time on the subject) encouraged me to have at least 100 net orders (in our case on a $30 product) on each side of an A/B split test as a starting point to have confidence in the results. 

And that rule can get extrapolated in a variety of ways, online and offline.

(Note: The science of a true “A/B list test”–making sure the two groups are alike in terms of their list profile—is also a key thing to understand to make sure you can rely on your test results. You need statistical significance on the responses and you also need list universes that are demographically similar. Lots more on that in Overdeliver…but I bet you all know that already…let’s call this note an FYI then.)

The other rule of thumb is to have an analyst (with an expertise in statistics) as your right hand person (assuming you are a marketer).

I recall a guest speaker at one of my Titans Mastermind meetings who had built a company that reached over $100 million in revenue at its height–and he said that when he launched his business he assumed his most critical employee/internal partner would be his Chief Financial Officer.

However, after building that nine figure business on the principles of direct response, he said the most important employee in the company was his “Director of Analytics” since he learned that without accurate numbers on the true profitability of every campaign (based on statistically significant, measurable results) the company would have faltered many times and probably failed.

This was the key to their growth and success..

He is not the only CEO/business owner/entrepreneur who figured this out.

And it is always the direct marketers who figure it out first…and best.

Not bragging…but I’m super proud of this particular endorsement of Overdeliver, from direct response marketing icon Dan Kennedy, who said this about the book (and direct response marketing in general):

“Pros will find it fascinating.  Those new to the disciplines will find a vital crash course. Pin-head executives in big, dumb corporations wasting oceans of money on utterly unaccountable brand and image and ego advertising should be forced at gunpoint to read it.

Ogilvy was right when he ranted to his own agency’s staff that only the mail-order people knew what the hell they were doing, and were Ogilvy alive, he would applaud Brian’s work here”

Of course Dan almost made me cry saying David Ogilvy would like my book. ☺

But that is not why I shared his observation with you.

And again, it was not to brag.

I am sharing it to give credence to what I am talking about today and even more broadly.

The lessons are not just for “big dumb corporations” who don’t get it.

What about the liquor store I worked at during college–and so many businesses like it?

Professional service businesses such as dentists, lawyers and the like?

Brick and mortar retail, even with an online presence?

It’s also possible that there are still some college radio stations (and maybe even some online marketers) who don’t get it either…and all of us need to understand direct, measurable, marketing and the principle of statistical significance.

The examples and people I have cited above might not lose “oceans of money” due to their bonehead thinking.

But I would recommend that the radio station should only be looking for the number three song of all time when they ask folks to vote again.

And the online marketers mentioned above, and those like them, should hire a blue chip marketing analyst as a top priority, especially if math and statistics are not their strength (says this English Major).

I know there are many pieces of software that can help with this important function but it’s a role that deserves a lot of personal attention too.

In short, they (and you) should do what David Ogilvy, my new BFF, says about that:

If you ever find a man who is better than you are—hire him.

If necessary pay him more than you pay yourself.

I assume that this was part of his ad for a “Director of Statistical Significance.” ☺



P.S. All this Ogilvy talk reminded me about the classic swipes PDF’s and videos that are part of the resource page which are all free for buyers of my first book, 

The Advertising Solution.

There are ads which should be part of every marketer’s library and there’s even a video of Ogilvy on The David Letterman Show which is priceless.

If you already own the book, and have accessed the resource page, let me know what you think.

If you have bought the book and not accessed the resource page, please go to the URL listed in the back of the book.

And if you haven’t bought the book yet, buy it at  and follow the instructions on that page so you can get the bonuses once you purchase.

I make nothing on the sale of the book but I guarantee it will be the best $16 you will ever spend (that’s what the book goes for on Amazon).

Each of the 6 legends profiled in the book, Ogilvy being one of them, are “direct marketers trapped in general advertiser’s bodies.”

And all of their best thinking and rules of thumb are in the book.

I believe we owe these pioneers a debt of gratitude because they trained us to make our advertising measurable, accountable and always statistically significant.

Read a recent post I sent you that you might have missed entitled The Maddest Pioneers—and there is a link to a 10 minute video in that post telling you more about the 6 legends profiled in The Advertising Solution.

P.P.S. And don’t forget about Overdeliver for a deeper dive into statistical significance.

And it’s got a few other things worth reading about.

At least I think so…☺

About the author 

Brian Kurtz

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}

Title Goes Here

Get this Free E-Book

Use this bottom section to nudge your visitors.