Searching & Finding, part II

So there is Maltego, the coolest tool for finding information and there are machines that find lots of data. Of course Google uses some very smart alogrithims and Udi Manber really knows what he's talking about. On April 16, 2008 he answered the question "When I come to a Google in the future the context of my social network could be folded into the search?" with "I can imagine if you give us permission to do that, and we find that that’s useful for some queries. The question is, what percentage of queries and what kind of queries? When should you use it and when should you not use it?"

This had me completly baffeld. WHAT? I was saying to myself WHAT IS WRONG WITH THIS DUDE? I mean, after one look at the concept of Maltego I knew that that is the only way forward. Maybe he drank a little too much Google Gulp? Maybe he was trying to hide something since Google does not do pre-annoucements? Or maybe, he'd seen Maltego of Delver too and was just trying to surpress their market value so the goog's could snatch it up for little money in a little time?

"We have no intention of competing with the Googles of the world, because Google is doing a very good job of indexing the Web and bringing you the Wikipedia page of every search query you're looking for," says Liad Agmon, CEO of Delver. But we've been there, seen it, and even do it ourselfs now.

But that does not satisfy anymore. You know the procedure yourself: go to google.com, type a couple of keywords, check the first listing, alter the keywords [order even], check the listing and on and on. Most of the listings you get will be actively manipulated by crooks and link spammers.

So we need something else. As Anand Rajaraman puts it: if you have limited resources, add more data rather than fine-tuning the weights on your fancy machine-learning algorithm. Of course, you have to be judicious in your choice of the data to add to your data set. And this is exactly the point I am trying to drive home. More data sources [and some very decent post processing] enhances the results in amazing ways. [he works on his own SE too, called kosmix].

Some say, it is a terrible idea, like KublaiKhan. "This sort of searching will result in information from 'opposingsides' of controversies or arguments being deprecated, resulting inskewed information being available--because people tend to associatethemselves with other people of the same opinion."He goes on: "This new search engine will be wildly popular amongst thetype of person who enjoys violent flamewars, and will be useless forany person who wishes to consider both sides of a situation beforeforming an opinion... so it's going to be an enormous success and if I had the cash I'd invest in it. :-/"

Personally I would like to quote merreborn in reply to that remark:
"Sorry, I can't friend you, you'll screw up my search results"

Update:
Seems there is much much more going on and wrong between google and social websites...