PHP Classes

Blue Crawler: Crawl and retrieve links of Web pages

Recommend this page to a friend!
     
  Info   View files Files   Install with Composer Install with Composer   Download Download   Reputation   Support forum   Blog    
Ratings Unique User Downloads Download Rankings
StarStarStar 56%Total: 1,710 All time: 2,301 This week: 57Up
Version License PHP version Categories
bluecrawler 1GNU General Publi...4.0Searching, Web services
Description 

Author

kai

This class can be used to crawl and retrieve links of Web pages

It can retrieve a given Web pages and retrieve the URL of all linked pages recursively.

The class can also crawl all linked pages until it finds given search keywords.

Picture of kai
Name: kai <contact>
Classes: 2 packages by
Country: South Africa South Africa

Details

Crawling a complete site:

construct the Crawler class with a string representing the URL you want to crawl

call getAllLinks(); on the object to return an array of found links



To crawl the site for a specific word or phrase construct the Crawler class with with a string representing the URL you want to crawl and the phrase you are looking for.

Reference the “thelink” variable owned by the Crawler object for the link the phrase were found on.

  Files folder image Files (4)  
File Role Description
Plain text file Crawler.php Class main class
Accessible without login Plain text file Readme.txt Doc. readme
Accessible without login Plain text file test.php Example example
Accessible without login Plain text file WordSeperators.php Conf. support code

The PHP Classes site has supported package installation using the Composer tool since 2013, as you may verify by reading this instructions page.
Install with Composer Install with Composer
Downloadbluecrawler-2010-04-15.zip 2KB
Downloadbluecrawler-2010-04-15.tar.gz 2KB
Install with ComposerInstall with Composer
 Version Control Unique User Downloads Download Rankings  
 0%
Total:1,710
This week:0
All time:2,301
This week:57Up
 User Ratings  
 
 All time
Utility:70%StarStarStarStar
Consistency:70%StarStarStarStar
Documentation:66%StarStarStarStar
Examples:70%StarStarStarStar
Tests:-
Videos:-
Overall:56%StarStarStar
Rank:1867
  

  Applications that use this package  
No pages of applications that use this class were specified.

Add link image If you know an application of this package, send a message to the author to add a link here.


  Related pages  
Home Site