Do Paid Search Ads Work? Negative Evidence Is Growing

'Half the money I spend on advertising is wasted. The trouble is, I dont know which half.'

So said 19th century Philadelphia retailer John Wanamaker, founder of the eponymous store chain.

But what if it's actually worse than that? For instance, what if all the money you spend on advertising is wasted?

eBay is not proposing that all advertising is squandered. But research it conducted and recently released suggests that paid search advertising may add no value to the companies that pay for it, especially when compared to free search listings. Yeah, ouch.

eBay tested both the null and positive cases. In neither did eBay's sales change much when paid search was either increased or cut off. The company has been relatively transparent about its research methodology. It is also willing to concede that paid search may work better for some other companies than it does for eBay.

The results, however, bring to the fore our sometimes delusional reliance on presumption in forming opinions about the meaning of data. The company's scientists also understand that the people buying these ads are neither stupid nor entirely venal. They are looking to perform a client service for which they expect to be paid - and the more successful, the more likely they are to be retained again. But correlation is not causality. What appears to be the reason for a sale - or the lack thereof - may be due to complicated factors and interrelationships between variables that can not be easily ascribed to one particular feature.

The implication is that we are still learning about how the internet works. And just as the net disintermediated certain traditional businesses, so may aspects of its commercial promise turn out to be of greater or less value than initially supposed as we gain more experience applying them. JL

Ray Fisman reports in Harvard Business Review via Slate:

eBay doesnt think so.

Before you read the rest of this article, go to Google and try searching for “Amazon.” You’ll probably notice that the top two listings are both for Amazon’s website, with the first appearing on a light beige background. If you click on the first—a paid search ad—Amazon will pay Google for attracting your business. If you click on the second, Amazon gets your business but Google gets nothing. Try “Macys,” “Walgreens,” and “Sports Authority”—you’ll see the same thing.

If you search for eBay, though, you’ll find only a single listing—an unpaid one. Odds are, after marketers at Amazon, Walgreens, and elsewhere catch wind of a preliminary study released on Friday, their search listings will start to look a lot more like eBay’s. The study—by eBay Research Labs economists Thomas Blake, Chris Nosko, and Steve Tadelis—analyzed eBay sales after shutting down purchases of search ads on Google and elsewhere, while maintaining a control set of regions where search ads continued unchanged.* Their findings suggest that many paid ads generate virtually no increase in sales, and even for ones that do, the sales benefits are far eclipsed by the cost of the ads themselves.

Companies spend enormous sums on marketing their products. Yet it’s notoriously difficult to measure the impact of ad expenditures. Companies advertise heavily at times when they hope to sell a lot—like Christmas Eve and Boxing Day—and in areas where they expect to see their sales grow. So a naïve examination of the relationship between ad expenditures and revenues will of course find they move in sync, even if customers don’t pay the ads any mind.

Advertisement

Advertising has also traditionally produced a lot of waste—I see ads for Brioni suits when I open up the morning paper, even though the last time I wore a suit was on my wedding day. The study’s authors quote 19^th-century retailer John Wannamaker: “I know half the money I spend on advertising is wasted, but I can never find out which half.”

The Internet promised to change all that. Google, Yahoo, Bing, and others gave sellers the opportunity to target their pitches to customers who were plausibly interested in their products. That’s why paid ads for Amazon come up in response to a search for books but not life insurance. Further aligning the interests of companies and consumers, advertisers only get charged for paid search listings that actually get clicked on, ensuring that they pay for attracting genuinely interested customers.

But what do companies actually get for the billions they now spend on search marketing? The eBay team began by examining whether there’s any benefit to buying search ads that contain the word “ebay.” In these cases, it’s possible that in the absence of paid listings, customers would simply click on the unpaid—or “natural”—listing, which would appear at the top of the search anyway. So in March 2012, eBay conducted a controlled trial to see what would happen if they shut off this “branded keyword advertising” by halting their purchases of search ads containing the word “ebay” on Microsoft and Yahoo search engines, while continuing to purchase search ads on Google as a control. There was no change in eBay sales via Yahoo and Bing, relative to those that came through Google—consumers simply substituted clicks on the unpaid search listing for the now-absent paid ones.

Encouraged by these findings, eBay management agreed to run a controlled experiment in which they shut off all Google search ads in a third of the country, while continuing to buy ads everywhere else. In contrast to branded keywords—where it’s inevitable that the company will end up as one of the top unpaid listings—there’s a good chance that if you try searching on “used les paul guitar,” a guitar reseller will appear ahead of eBay’s search listing. So in order to drive a customer to eBay for his guitar purchase rather than, say, to Guitar Center, it might be worth the cost of placing a carefully targeted ad.

But in aggregate, that’s not what the eBay team found—overall, there was no appreciable decline in sales of eBay listings in the part of the country where Google ad purchases were shut off. People who thought to buy guitars via eBay were finding their way to the site anyway, either by clicking on natural listings or by going directly to eBay’s site without using a search engine at all. Search ads did generate a modest increase in the likelihood that Internet surfers with little recent history of eBay transactions would end up making purchases on eBay. So paid search ads serve an informational function, letting a sliver of potential eBay customers know that they’re in the guitar business. But by the time you get to customers who have had three prior eBay transactions in the last year, the effect of paid search on sales drops almost to zero. Overall, paid search turns out to be a very expensive way of attracting new business: The study’s authors estimate that, at least in the short-run, paid ads generate only about 25 cents in extra revenues for each dollar of ad expenditures. (For branded keyword searches, the additional revenues are close to zero.)

People buying search ads aren’t idiots—they’ve looked at the correlation between keyword purchases and subsequent sales and no doubt found it to be strong. But this study suggests that marketing departments should be more careful in confusing causation and correlation in assessing the returns to their ad expenditures, to avoid the equivalent of concluding that marketing works because you advertise and sell a lot in December.

The study’s authors note that paid search may be more profitable for other companies than it’s been for eBay. For example, as Stop & Shop tries to get a foothold in the crowded New York online grocery marketplace, they might sensibly buy some ads to compete with FreshDirect. Paid search may also be worth it for smaller companies that lack the name recognition and high Google page rank that make paid searches less valuable for the eBays and Amazons of the world—some of Google’s own research indicates that this is likely the case. Caveats aside, eBay’s experiences suggest that all companies should look carefully at how much bang they’re getting for their search marketing dollars.

The larger lesson from eBay’s experiment is about the importance of questioning conventional marketing wisdom. As much as the Internet has given companies opportunities to target their ads, it’s also given them a ready testing ground to experiment with different business practices to see what really works.