Far too often I hear people quoting Alexa data. Even last week, at the 2007 Omniture Summit I witnessed Tim O’Reilly using Alexa charts to prove Web 2.0 success in front of 1,000 smart web analytics professionals. I know I couldn’t have been the only person in the crowd to notice. For Tim’s benefit, and anyone else who uses Alexa Data, please take note:
ALEXA DATA IS TREMENDOUSLY FLAWED
I touched on this in a competitive intelligence metrics post back in October, showing that Alexa’s data is less accurate to determining true site traffic then the # of characters the domain name, but now I’d like to really illustrate how far off Alexa’s data is.
Many people have pointed out Alexa’s data is biased towards a certain crowd and can be manipulated (see the links at the bottom of this post), but none have illustrated the margin of error that I’m about to. Below I take a look at two very different sites with very different web traffic stats.
Site 1: Allrecipes – Allrecipes is a leading food site – as you might expect, Allrecipes users are similar to what you might see on the Internet as a whole, though slightly more female.
Site 2: SEOMoz – SEOMoz is a site that caters to the SEO and online marketing community – a crowd more likely to install the Alexa toolbar.
Using Alexa, you might conclude that SEOMoz receives more traffic than Allrecipes:
Alexa Reach Chart:
Alexa Rank Chart:
Both sites are very popular within their target audience, but despite what Alexa may show, Allrecipes has much more traffic. Let’s face it, more people cook food, then perform SEO! In fact, if you were to populate the above charts with actual data, SEOMoz would be a flat sliver near the x-axis. Here’s some real data from Dec. ’06:
Allrecipes Unique Visitors: 11,023,187
SEOMoz Unique Visitors: 102,523
If you were to use Alexa charts to draw conclusions about either site based off real numbers for one site, your traffic estimates would be off by approximately 11,842%. Numbers that big are often difficult to grasp, so I like to put it in perspective. A mistake of that magnitude is the equivalent of:
- The CIA mixing up the population of Ohio for China.
- Your accountant saying you owe $1,000 to the IRS, when you really owe $119,417.
- A cop pulling you over for doing 60 in a 30, when you were really going half-a-mile-per-hour.
- Telling your spouse you’ll be home in three hours, then showing up 15 days later.
These are mistakes that none of us could get away with, so why should we let Alexa?
I’m not the first to prove Alexa data is flawed. Here are links to other Alexa skeptics:
Peter Norvig, Paul Stamatiou, Josh Pigford, Matt Cutts, Rand Fishkin (thanks for the data!), Greg Linden, Bruce Stewart, Alex Iskold, John Chow, and Markus Frind.
