Jeremie Miller, the project's technology chief, hopes an "alpha" version of the engine will be running by Christmas. As well as search, it will offer "wiki-style tools to improve search and basic social networking"
"Unlike Google, Search Wikia will not share search data with advertisers, nor invade privacy by storing users' search terms...The effort's architecture is similar in fashion to the SETI@Home project...500 volunteers are running web-crawlers to compile Search Wikia's web index, which so far totals 100 million pages"
Wikia's Search is smartly boot-strapping off of the established open source distributed web crawler software/project that it purchased in July, Grub. An introduction from Grub's site:
Search is part of the fundamental infrastructure of the Internet. And, it is currently broken.
Why is it broken? It is broken for the same reason that proprietary software is always broken: lack of freedom, lack of community, lack of accountability, lack of transparency. Here, we will start to change all that.
Grub started back in 2000 with a simple concept of distributing part of the search process pipeline: crawling. In a way, we were a bit ahead of our time, but our intention then was what it is now. We want to help fix search.
Now, with the help of Wikia, community members, contibuters, and Open Source developers our time has come again. Come be part of something greater. Come help us change the World.
This is a very interesting project with altruistic motives. Google and Yahoo do good job at search, but as pointed out here previously, having an open source, and non-commercial democratic search index/application would be a welcome addition to the proprietary ones offered by the web giants. Time will tell if the Search Wikia can deliver the same kind of quality, relevant results.
digg the New Scientist story here