Friday, April 06, 2007

The World Needs Open Source Mega-Search

Open Source, Mega - Search.

Proprietary Search Engines are just that, Proprietary. Calacanis highlights the issue with regard to Metasearch, asking if it's even legal or not. (it appears by many Big Search TOS's that MetaSearch is NOT legal, btw)

The famous Page Rank (tm) Algorithm is proprietary and still largely secret. Remembering that fact helps to put in perspective that the emphasis in, "Google's Index of All the World's Information" in reality is, "Google's Index of all the World's Information."

This comment goes even further:

"Why is it then that our current most modern Meaning Economy is a text box dictatorship? Why in such an advanced civilization have we become Knowledge Peasants whom are so easily placated by the black magic of our Goovernor? Am I the only one wondering why these commercial boxes own such an important social function: what everything means?"


Is anyone completely comfortable with closed & proprietary systems determining how information is accessed and what it means to us?


As the Network and the Computer converge, the concept of Open Source should scale to Search. The idea of "Open Source + Search" is a broad one. It's about Networks of Information & Content Aggregation sharing their Indexes and literally "what runs under the hood" with each other.

Powerful shared and distributed computing systems like Amazon's Elastic Cloud can be leveraged to deploy deep spidering applications, or Mega Crawlers, to instantaneously tease-out very tailored & specific information from the vastness of the entire web (by the web, for the web)

Further out, with the acceleration of Moore's Law combined with the adoption of "consumer information trapping" technologies, this distributed search index could integrate smaller nodes, end users, much like a P2P -or- BitTorrent Network.

Thus, in this hybrid network of networks and eventually peer-able grid, nodes will share & mine each others databases, normalize the collective results, and deliver massive amounts of relevant content to the public by way of an algorithmically open source search architecture under the GPL.

(and of course, the costs of this not-for-profit endeavor could be defrayed by monetizing the search engine results contextually with say...an Adsense account? ;-)

No comments: