Solving The Click Fraud Conundrum
A firestorm recently touched off across the 'net when a published article revealed a study that found up to 50 percent of the click-throughs on pay-per-click (PPC) engines were fraudulent. Even though the study pointed out that the majority of fraud occurs in highly competitive and upper-priced keywords, it was still a major clarion call for marketers who run any type of PPC campaign.
The study was performed by web analytics provider ClickLab – a company that markets software designed to automatically track and diagnose click fraud. While the source may raise red flags suggesting a conflict of interest, numerous other independent studies have found that, at the very least, click fraud accounts for between 10 and 20 percent of PPC traffic. And we're fairly certain that the percentage is significantly higher within hyper-competitive arenas such as gambling, adult, and pharmacy – just to name a few.
Simply put, this latest study is just another indication of what's been strongly suspected for some time now – that click fraud is taking a significant bite out of PPC marketing budgets.
Up to now, PPC marketers have viewed click fraud as an unavoidable cost of doing business online and most companies have just factored the cost into the equation. After all, the time and resources one must commit to effectively ferret out instances of click fraud can easily add up to more expense than the cost of the fraud itself. Then, when you combine the hassle of haggling with PPC providers over details and refunds, you have a genuine headache that only a good dose of denial can mask.
But, the headache is getting bigger and denial seems to be exacerbating it. So, not surprisingly, a handful of services offering solutions have emerged. Their objective is to help companies deal with the problem in ways that are time and cost effective. The question is, of course, do the "solutions" solve the problem? ...or are the "cures" more problematic than the disease? Let's examine your options.
Should You Even Be Worried?
Determining if click fraud is a problem for you requires that you get a rough idea of how much money you might be losing. That will give you some basis upon which to decide whether it's worth the effort trying to recoup your losses. Bear in mind that attempts to extract refunds can require the kind of effort that depletes the best of us.
Therefore, when making this decision be sure to factor in the return-on-investment (ROI) of your, uh, solution. For instance, if you're running, say, less than $500 a month on PPC ads, then a comprehensive anti-fraud program costing $295 a month hardly makes sense. Even spending more than a few hours of your time every month tracking click fraud wouldn't be financially wise in this case since you probably aren't experiencing more that $50 to $100 dollars a month in fraud to begin with. Surely, your time is worth more than that.
As a starting point, figure in 10% (a conservative number) of your overall PPC advertising budget (20% if you're in a competitive field where clicks routinely cost $10 or more). This number should represent the absolute maximum you're willing to commit to your click fraud prevention program and still expect a positive or break-even fraud prevention ROI (not to be confused with your overall advertising ROI).
Remember also to factor in the time required to manage the software that monitors the clicks and the effort involved in providing documentation to support your negotiations with PPC provider fraud and refund departments.
Why Click Fraud?
Click fraud typically occurs for two reasons:
Presenting, Click Fraud for Hire! (What will they think of next?)
the most alarming news about click fraud is that it's now being
conducted on a large scale by companies that are hired specifically for
that purpose. Your competitors can now hire companies in India, China,
and elsewhere who have workers trained specifically to drain PPC
advertising accounts while avoiding detection by PPC providers' fraud
The Many Faces of Click Fraud
The most obvious kind of click fraud can be easily detected and filtered out by PPC providers. Whenever a perpetrator clicks an ad multiple times in succession from the same computer it's almost always discredited without the advertiser ever being charged for the fraudulent clicks. Surely, we should all expect protection from such a rudimentary attempt to defraud our advertising accounts.
However, a sophisticated fraudster can employ an array of techniques that are much more difficult for PPC providers to detect. For instance:
How To Detect Click Fraud
Your first and most important line of protection against click fraud is effective tracking. You should become intimately familiar with your server logs so that you can recognize trends within your click-through traffic. This will make it much easier to identify unusual patterns such as traffic spikes or excessive clicks originating from the same IP address.
There are several software programs designed to make tracking your pay-per-click traffic easier. Most come with features that will track all clicks by IP address, PPC affiliate, time spent on your site, geographic origin, and a host of other data that will aid you in determining if a click is fraudulent or not.
Unfortunately, however, there is no fool-proof way to prevent an unscrupulously determined competitor from clicking on your pay-per click ads. What you can do is generate and maintain a detailed record of all incoming click-traffic. This will arm you with the evidence you may need to prove your case should it become necessary to provide documentation for a refund request.
You can also use such evidence to confront any perpetrator you happen to identify within your records. This will give you teeth when pursuing recourse with the proper authorities and in most cases scare the fraudster enough to make them at least stop clicking your ads.
One of the easiest ways to monitor for click fraud is by assigning a unique tracking marker to every URL you're using in your PPC campaign. Doing so will make it relatively simple to identify traffic sources through your server logs.
For example, if you're running an Overture campaign for the phrase hawaii timeshare condo, you could assign a tracking URL such as:
This greatly simplifies the process of sorting through your PPC data while viewing your server logs thereby making it easier to identify which phrases and campaigns are attracting questionable clicks. Such descriptive URLs also make it easier to employ your log analysis software (WebTrends, ClickTracks, Sawmill, etc.) to help you crunch the numbers. Such tools will help you track your average daily clicks, number of page views, and conversion rates per click for each PPC engine as well as each individual keyword/keyphrase.
Be aware that you'll need to employ some sort of mechanism, such as cookies or session URLs, that attaches the tracking data associated with the URL to the customer for the entire time they're on your site. This can be done using a web scripting language such as PHP or ASP. Of course, this involves a bit of programming so it's handy if you have experience in this arena. If not, you should find someone who can help you accomplish this task or, in the event you're feeling up to the challenge yourself, review these
The most obvious form of click fraud to look for when analyzing your logs is a pattern of repeated clicks from the same IP address coupled with a very low or non-existent sales conversion rate. Such activity indicates that the same person is clicking your ad over and over again with no intention of making a purchase. As mentioned previously, this will usually be caught by your PPC provider but it's important to keep an eye out for it yourself. Also be aware that many of the smaller PPC providers do a poor job of catching even this most basic form of click fraud (remember, the fox is guarding the hen house - so you'd better keep an eye on the fox, too).
Pay particular attention to the average number of click-throughs you're getting per impression (i.e. the number of times someone clicks your ad relative to how many times it's viewed). Impression tracking is information that must be provided by your PPC provider – your site logs can't show you this. But providers such as Google AdWords can provide you with tools that enable impression tracking so be sure to utilize that resource in conjunction with your in-house tracking tools. If you notice that nearly everyone who sees your ad is clicking through, but few, if any, sales are occurring, this could be an indicator of click fraud.
As indicated above, using cookies to track visitors who click your ads can also be an effective way to help identify click fraud. Let's examine the example previously mentioned where the fraudster is clicking your ad in Yahoo, then AltaVista, then Go2.net and so forth. If you cookie them after the first ad-click, then you can track them across all of these PPC affiliates. You can even track them as they move to other PPC providers such as from Google AdWords over to one of the Overture partner sites.
Bear in mind, however, that a sophisticated fraudster will delete their cookies between ad-clicks and this will effectively defeat your cookie tracking. On the other hand, cookie-tracking does give you a fighting chance to detect and snag the less diligent or knowledgeable fraudsters.
If the fraudster is using a hitbot (automated clicking software, aka, clickbot), then detection becomes much more difficult. One clue to look for is proxy server IP addresses. If you notice a significant activity coming from proxy server IP addresses, this could indicate click fraud.
Occasionally, you may find that one of the proxy servers being rotated through isn't anonymous. In such a case the fraudster's actual IP address will slip through revealing the true origin of the click. When this happens you may be able to determine the identity of the perpetrator. However, this can be difficult to do and requires a very close analysis of your log files. Furthermore, much of the traffic coming through proxy servers is not fraudulent – so, it can be difficult to distinguish legitimate clicks from fraudulent clicks.
Again, that underscores why your first line of defense is to become intimately familiar with your site's traffic logs so you can see the difference between normal and abnormal traffic patterns, conversions ratios, page views, and so forth. You must know how real PPC traffic typically behaves before you can identify the fake PPC traffic.
By the way, you can find a list of proxy servers by searching Google. If you find the IP address of any of these servers showing up frequently in your logs, this may indicate a fraudster is using that server to avoid detection while clicking on your ads.
If you suspect that your competition is hiring a foreign company's service to click your ads, you might want to configure your ads so that they're only shown in countries where you have buyers. For instance, if your hot tub company is located in California, the chances that someone in Nigeria, India, or China will be purchasing from you is somewhere between zero and none. So, the risk of losing sales by eschewing clicks from these countries is negligible
Remember that if you do choose this option, Google will detect the IP address of potential clickers and display your ads only to visitors with IP addresses originating within your specified countries. To learn more, visit Google's information site at: https://adwords.google.com/support/bin/answer.py?answer=6277&topic=21.
Unfortunately, this "solution" can again be subverted by the savvy fraudster. That's because using an anonymous proxy server will usually show the clicker's country of origin as unknown or no country, no region. In such cases, Google will display ads selected mostly from the U.S. Furthermore, if the fraudster is using an AOL account, it's impossible to track their geographic location since AOL generates a new, US-based, IP address for each page view.
Currently, Google is the only major PPC provider that allows you to restrict ads to certain countries based on IP address.
Detection—Your Best Line Of Defense
You or someone within your company must become intimate with your site's traffic logs in the same way that someone within your company attends to income, receivables, taxes, and bank accounts. Now is the time to add site logs to that list. Otherwise, you stand to leak money playing the PPC game as surely as if you'd leak money by trusting an embezzler to do your books. Knowing how to interpret your traffic logs is critical to operating any business online!
Secondly, set your systems up to detect click-fraud. Even the fox will refrain from raiding the hen house if he knows the farmer is watching, gun-in-hand. Common thieves are cowards and rarely act if they perceive the crime as being too risky. By seeing to it that you're not an easy target, you'll have the battle half-won before it even starts.
From there it's a matter of proving to the engines where, when, and how you've been victimized by click fraud. They will need to see documentation that your clicks are originating from the same user and that the clicks are not generating sales. Again, that's why tracking and monitoring your PPC traffic is critical.
Whenever you have data to document your click fraud claim, contact the PPC provider by phone and explain that you've been defrauded. Then request a refund. Because PPC providers get many of these requests daily, don't waste your time or theirs until you have the documentation in hand and ready to present. Also bear in mind that lower level service reps may not even be familiar with the issue of click fraud. Therefore you should either ask for the Click Fraud Department or speak with a manager or other higher-up before you can expect to move forward toward a favorable settlement.
Services That Can Help
While your raw log data may be enough to establish proof of click fraud, a detailed web analytics service such as ClickLab or a dedicated fraud detection service such as WhosClickingWho will generate more detailed, easy-to-read reports that expedite the refund process.
Again, the mechanics and sophistication of your detection scheme should depend on your expected return-on-investment (ROI). Just starting out, while testing a conservative, supplemental or limited PPC ad campaign, might warrant that you personally eye-ball your log data manually. However for medium to large PPC ad campaigns – or when aggressively testing large numbers of variables – the enormous amount of time and effort required by a manual approach quickly renders such a labor intensive endeavor unrealistic.
Unless your PPC campaign is spending more than $500 a month, it's hard to justify purchasing an anti-fraud solution such as ClickLab or WhosClickingWho since such services typically run between $50—$100 monthly.
Regardless, our preliminary tests with both of these programs have found them to be useful tools. ClickLab provides a fairly comprehensive web analytics program with detailed reports and an easy-to-use interface. However, it should be noted that ClickLab's click fraud solution is currently in beta testing, so expect to experience a few minor bugs.
WhosClickingWho also comes highly recommended by a number of online marketers. Although their interface is less intuitive and they don't provide the detailed graphs or web analytic reports that ClickLab does, their click fraud solution is functional and more mature.
Both services come with a number of useful features. They are able to:
In short, they automate and simplify the heavy lifting that would otherwise take you an enormous amount of time and energy if you were to analyze your log files manually.
A medium to large PPC campaign (spending more than $50,000 a month) would probably justify a larger, more robust anti-fraud solution. Companies managing large PPC campaigns may also be able to justify hiring a dedicated firm such as Alchemist Media or ClickAssurance to handle both the detection of click fraud as well as the process of recovering refunds from the PPC providers.
Alchemist Media has emerged as an authority on the topic of click fraud and provides an all-in-one click fraud detection package for the corporate enterprise – at price-levels that place them out of the range for most small online businesses.
ClickAssurance is a relative newcomer to the field. Their approach is currently unique – they work on a contingency basis taking only a percentage of the refund they're successful in recovering on your behalf from the PPC provider. In theory, this minimizes your risk as an advertiser since they only get paid if you do. They also offer to analyze your past logs to determine if you've been previously defrauded and will work to recover any money you may have lost before you were aware that click fraud was a problem.
Besides employing a click fraud detection service, there are a number of other ways to limit potential damage.
Where Will the Arms Race End?
Over the past year, the click fraud issue has risen from relative obscurity in the online advertising world to a sharp thorn in the side of every PPC advertiser that has watched bid costs skyrocket while sales remain constant. Some people speculate that increasingly sophisticated and rampant click fraud will eventually dilute the market for paid search advertising. Many others believe that the major search engines will eventually step in and eliminate the fraud problem with expensive new technology systems.
Regardless of the long term result, it seems clear the PPC providers are not motivated to tackle the click fraud problem any time soon – since even an incremental solution would mean a substantial cut in their revenues. And, until a comprehensive fraud solution is developed, there will be two classes of PPC advertisers: those that pay an inflated price for click traffic each month controlled by their competitors’ use of click fraud, and those that pay much less in bid costs by actively managing their click fraud and demanding refunds.
Presently, you cannot prevent it – you can only detect it. And the sophistication of your log analysis and click fraud detection systems are what will determine your ability to compete as well as to acquire reimbursement whenever it should happen that you fall victim to the growing scourge of pay-per-click fraud.
Just our two clicks,
Esoos Bobnar is a researcher and technical analyst for Planet Ocean Communications
Jim Gilbert is President of Position Concepts, a specialty service search engine marketing firm providing SEM, SEO, PPC, and research services. Jim has performed research and search engine optimization since the inception of public access Internet search engines in 1993. Before his involvement with the Internet, Jim spent twelve years providing analysis support in statistics, simulation, and numerical analysis for several fortune 500 companies.
Planet Ocean ® is a registered trademark of Planet Ocean Communications, Inc.