A lot of noise is being generated about Wikia Search: “Google’s Nightmare” according to Fast Company latest issue. Of course, this is the usual journalistic hype about anything to do with Web. Big names, like Wikipedia’s founder, Jimbo Wales, will get a lot of media attention, although time and again it is some complete unknown who will pop up to give us the “next big thing”.

But is search broken, as Wikiasearch claims? Surely it depends on a whole raft of things, like what you’re searching for, how good you are at describing it, why you’re looking for it (i.e. context) and who you are (i.e. more context again - “personalisation”). Obviously, a single set of algorithms seems unlikely to solve a potentially complex and wide set of problems.

User behaviour now seems a more promising attribute to explore than the legendary “back link” attribution that supposedly made Google so effective in the first place, which the article in Fast Company claims Google has more or less abandoned (as it’s so prone to link spamming to boost a site’s ranking).

Interestingly, an operator or ISP has a potentially far better chance of understanding user behaviour than Google or any other web player, or so claimed BT at the recent Telco 2.0 Industry Brainstorm. Click behaviour clearly spans a number of websites - it isn’t only what users do on Google (or affiliates). An operator can monitor a user’s entire clickstream, not just the “cookie crumb” trail. However, whilst that sounds wow-wee - “think of all that power” - it isn’t as useful as it first seems. It is one thing to know every link that a user ever clicks, but quite another to mine the links for meaning. A link isn’t called http://”click this if your next car is probably going to be a BMW 520″, it is called http://”some obscure string”. Until the Semantic Web arrives, figuring out meaning from a clickstream is a big big problem.

The Wikiasearch premise, as mentioned on their homepage, is to use human intelligence, not just algorithms. There is clearly merit in this approach. A community of experts on a particular subject are probably more likely to be able to identify which sites on the web are useful when searching for terms within that community’s domain of expertise. Of course, the community could be biased, but an open search framework at the very least allows the bias to be countered by a “democratization” of the knowledge base underlying the search. That’s essentially how Wikipedia works - users can contend with each other and the output is hopefully balanced or, at the very least, the contentions are transparent.

The challenge is the size of the community participation versus the size of the knowledge base on the Web, which is clearly vast and growing rapidly. It would therefore seem that a degree of automated “link mining” is always going to be required, but the promise of “democratisation” is made through the use of open software where the algorithms are collectively generated and transparent.

However, for mobile search, I think that the potential for community-supported search is far greater because I don’t think the “long tail” is so important. Within the mobile context, which here means when the user is compelled to find something quickly that is of real-time value (as opposed to the “killing time” browsing in a coffee shop waiting for a friend), I think the number of categories of search will be relatively small and so will the desire for results: users won’t care for 1,564,000 results - they only want 3, 5 or “something reasonably useful within 8 seconds.”

In the mobile situation, depending on the search criteria, there is a distinct cost/benefit trade off. Users don’t necessarily want the “optimum” answer in the sense that they know is accessible via a more concerted research effort, but they will make do with an answer that is “safe” in some sense, hence why it ought to be possible to make do with 5 useful/safe (”appropriate”) results, rather than ten million. Of course, one could argue that a good search engine should do that anyway, but I’m contending that this is in some ways easier to do if we know the user is mobile and in a “need an appropriate answer within 8 seconds” mode.

There are numerous ways to exploit the mobile “search intention”, which is why it seems more likely that some unknown bright spark not involved with Web search will crack this nut rather than the Googles and Yahoos trying to shoehorn their search fatwares into mobiles. What I’m saying here is that mobile search ain’t “web search on the mobile”, just like mobile TV ain’t TV on the mobile and that the Wikiasearch idea to use “the community” has potentially more promise to get us to where we need to be.

Technorati Tags: Wikiasearch, mobile search