I found this search engine blind comparison site via Jeremy Zawodny’s blog earlier today. Jeremy linked to Seth Godin’s post about it, ‘Can more than 60% be wrong’. Am I missing something, or is Google solidly winning that in comparison, not losing as Seth implies. Yes, more people preferred Yahoo OR MSN over Google, but significantly more people preferred Google over either of the other two alone.

Generating search results is fundamentally a statistical process, and giving ‘not google’ two chances makes the whole thing very unfair if you’ve going to compare it in this way. You might as well give the worlds best marksman a single shot at a target, then say he failed if any of the second through hundredth best managed to get closer. How about we start electing politicians like that?

Looking at the results as they currently stand, about 10% more people judged Google better than Yahoo (in second place) and that’s pretty significant result if the number can be trusted. It would be interesting to extend the experiment and have users mark which results they considered relevant to their query to get the traditional search engine measures (like MRR and S@3). I suspect Google would still win on more queries than any of the other whole web search engines.