February 17, 2014 Last Updated 7:39 am

Digitally Marketing: No Clicks? No orders? No problem

When marketing your media apps, how long do you wait on a campaign that has no clicks or no orders before you decide to cancel it?

Here’s a common problem all online marketers share. How long do you wait on a campaign that has no clicks or no orders before you decide to cancel it? Are 200 clicks enough? Are 10,000 impressions enough? I mean, your campaign has 10,000 impressions and no clicks? You should pull it, right? You’ve received 200 or 300 clicks and no conversions. Time to pull the plug? How long should you wait? Depending on your threshold for being wrong, the answer is a lot longer than you may imagine.

CPH-Plaza-150-feature

Troy McConnell

To get to the answer, we have to talk about the probabilities of certain events happening.

If you’ve followed Nate Silver, then you’ve been introduced to a type of statistics called Bayesian. It’s a way of using probability to predict future outcomes based on previous information. I’m going to use a very simple case to show how it answers the question of how long you should wait on those campaigns.

To simplify this, I’m going to focus on click-through-rate (CTR) for this column. But the approach is just as valid for conversions. One way to ask the question is:

“What is the probability that the CTR of this campaign is going to be at least average given that for the first 2,000 impressions the CTR has been zero?”

In other words, at this stage of the campaign (2,000 impressions into it), what are the odds that the final CTR is going to be average?

Another question you might ask is “What are the odds of the final CTR being non-zero?” Or, you could ask, “What are the odds that the final CTR is better than average?” I’m going to focus on the question I posed above – but my approach will work for any of these other questions.

Let’s look at the evolution of CTR of a (real) campaign. Below is a chart that shows the cumulative CTR and impressions over the first 40 days of a campaign.  Each day is shown in the x-axis. The CTR is in blue and uses the scale on the right. The number of impressions are in red and use the scale on the left. The red line says that after about 10 days, the campaign had around 60,000 total impressions. After 30 days, it has around 120,000 impressions and by day 40, it had been shown almost 160,000 times. The blue line says that CTR for the first 2 days was around 0.07%. After the first 10 days, the CTR was around 0.035% (with about 60,000 impressions).

Notice that the CTR starts at zero, jumps to 0.07% and then settles in at a little over 0.03%.

Cumulative-CTR-Performance

On the other hand, below is another (real) ad campaign over its first 20 days. Notice that the CTR is 0 for the first 3 days then climbs and levels off to 0.07%. The jump in CTR happens somewhere between about 3,000 impressions and about 47,000 impressions. So, after 3 days, this campaign had 3k+ impressions and no clicks. Could you predict that the eventual CTR on this campaign would be twice the CTR of the above campaign?

Cumulative-CTR-Performance-2

Going back to my original question, is there a way for me to predict that the eventual CTR of the second campaign above was above (or at) average given that the first 3,000 impressions had a 0% CTR? Using Bayesian statistical analysis, I can establish the odds of that happening. Here’s how it works.

Bayes-Table-1The table to the left characterizes a random sampling of the campaigns on our exchange at AudienceFUEL from September 2013 to January 2014.  There are 735 campaigns selected representing over 1.4 billion ad impressions.

For each campaign I looked at the how many impressions the campaign was shown before it got its first click. Sometimes, that was in the first 10 impressions. Other times, a campaign had more than 10,000 impressions before it got its first click. The first column shows how many campaigns had no clicks after at least the first 2,000 impressions.  The second column shows how many campaigns received a click before the campaign got to 2,000 impressions.

The rows of the table show the number of campaigns that had a final CTR of more or less than 0.04%. So, there were 206 total campaigns (the first column) that took at least 2,000 impressions before they received their first click. Of those 206, 137 had final CTR’s of less than 0.04% and 69 had CTR’s of more than 0.04%.  There were 529 campaigns (second column) that got their first click before the first 2,000 impressions.  Notice that the totals of the columns and the rows both equal 735 -which was the total number of campaigns I sampled.

Bayes-Table-2

The table to the right shows the same information but uses percentage of total campaigns instead of total campaigns. So, for example, 18.6% of all campaigns had a final CTR of less than 0.04% and took more than 2,000 impressions to get its first click. By the same rule, 41.8% of all campaigns had a final CTR of more then 0.04% and took less than 2,000 impressions to get the first click.
So what? Well, let’s go back to my question.

“What is the probability that the CTR of this campaign is going to be at least average given that for the first 2,000 impressions the CTR has been zero?”

It turns out the answer is equal to the probability of both (both means the CTR is at least average AND there were more than 2,000 impressions before the first click) divided by the probability that it took more than 2,000 impressions before the campaign got its first click. From the second table above, that number is just  9.4% divided by 28.0%, or 33.5%.

In other words, the odds are about 34% that a campaign that has no clicks after 2,000 impressions will still have a CTR of 0.04% or more. Below is a table that summarized the odds for a campaign to have a CTR of at least 0.04% when the first click comes after 2,000, 5,000 and 30,000 impressions.

Results-of-Bayes-TableSo, if you were comfortable being wrong 1 in 5 times, then you could turn off the campaign after 5,000 impressions. But, if you wanted to be wrong less than 5% of the time, then you would wait until you had 30,000 impressions.

Ok, 30,000 impressions is not a lot of impressions and you might not be micro-managing campaigns (although your automated software might be). But, how long do you wait to pull the plug on a landing page that has clicks but no conversions? The point is that your intuition may have told you to stop sooner than Bayes said you would.  Well, you would if you were making informed marketing decisions.

Troy McConnell is CEO at AudienceFUEL.

  • Wojciech Szywalski 3 years ago

    Impressive article! So often we get “Fooled by Randomness” 😉 Probability was – probably – the hardest part to go through during my whole education process. But yes it’s still impressive, yet very useful, part of mathematics. I would love to read more about use of probability in marketing campaigns.

    • Troy McConnell 3 years ago

      @Wojciech. Thanks for the feedback. I have more technical blog entries about using probabilities to optimize ad campaigns at my company website but I plan on writing more here at TNM for Doug. Feel free to send me an email at troy at audiencefuel dot com