Download List

Project Description

Yioop! is a PHP search engine. Yioop! can be configured as either a general purpose search engine for the whole Web or it can be configured to provide search results for a set of URLs or domains. Yioop can crawl pages or can directly index archives such as ARC and WARC. It supports indexing several file formats such as HTML, Atom, PDF, DOC, PPT, RTF, RSS, XML, SVG, PNG, JPG, BMP, GIF, and sitemaps. The Yioop! crawler can be deployed on one or many machines. It supports having one or more to crawl scheduler processes, as well as multiple fetchers and mirrors. Crawling respects robots.txt including Crawl-delay. Yioop! crawls are stored in a Web archive format that is easy to move around. Crawling can be done on one machine and the results deployed elsewhere. Yioop! supports mixing of crawls. Yioop! comes with a search front end that can be localized as desired using a GUI. This GUI supports RTL languages. Management of crawls can also be done using this GUI. Yioop! can be configured in a straightforward manner to make use of file caching or memcache if available.

System Requirements

System requirement is not defined
Information regarding Project Releases and Project Resources. Note that the information here is a quote from Freecode.com page, and the downloads themselves may not be hosted on OSDN.

2013-01-05 11:37
0.92

This release supports materializing as new indexes query-based combinations (crawl mixes) of
old search indexes.

This should make query performance of crawl mixes much better.

Cache pages of search results now have a new history UI which allows you to search cache pages
in all indexes you have,
much like the way Internet Archive does.

Yioop now supports spell corrections on searches after they have been performed,
and it has an
API for transliterating between roman and other scripts.

Query performance has been improved over previous versions, and lots of minor bugs have been
fixed.
Tags: Minor

2012-02-04 11:54
0.82

This release improved scalability by allowing multiple machines to maintain portions of the "to crawl next" queue. Query processing can also be split amongst machines, with different machines being responsible for documents of a given hash. Yioop! now supports mirroring of machines. Two word phrases as determined by an XML file such as Wikipedia URL dump can now be treated as a logical unit. The Yioop! model-view-controller framework has been made easier to extend and documentation for it has been added to the website.
Tags: Minor

2011-12-08 08:57
0.80

This version supports starting, stopping, and viewing log files of the queue server and fetchers from a Web interface. One can now inject new URLs into an active crawl via a Web interface. This version of Yioop! supports re-crawling of pages after a fixed number of days. Also, the file extensions that are crawled, the number of bytes downloaded per page, and how Yioop! weighs different page components can now all be controlled through a Web interface rather than just the config.php file. Improvements have also been made to how HTML Processor extracts text to index.
Tags: Minor

2011-10-29 11:40
0.78

Character n-grams are now supported for many languages that did not have a stemmer. Language detection was improved and better UTF-8 preparation was provided for downloads. Yioop!'s ability to following redirects, including bit.ly redirects, was improved. Proximity scoring of text in documents has also been enhanced.
Tags: Minor

2011-10-03 00:53
Version 0.76

This version adds a function API to get search results out of Yioop! It also improves the Open RSS Responses that Yioop! generates and allows them to contain images. The online documentation has been enhanced to describe in more detail how to incorporate Yioop! into a site. This release also simplifies the arc_tool syntax and adds the ability within arc tool to reindex a corrupt IndexArchiveBundle dictionary. This version fixes a bug in output buffering of the dictionary that could cause the dictionary to become corrupted on large indexes.
Tags: Minor

Project Resources