To search, Click below search items.

 

All Published Papers Search Service

Title

Self Adjusting Refresh Time Based Architecture for Incremental Web Crawler

Author

A.K. Sharma, Ashutosh Dixit

Citation

Vol. 8  No. 12  pp. 349-354

Abstract

Due to the deficiency in their refresh techniques [12], current crawlers add unnecessary traffic to the already overloaded Internet. Moreover there exist no certain ways to verify whether a document has been updated or not. In this paper, an efficient approach is being proposed for building an effective incremental web crawler [13]. It selectively updates its database and/ or local collection of web pages instead of periodically refreshing the collection in batch mode thereby improving the “freshness” of the collection significantly and bringing new pages in more timely manner. It also detects web pages which frequently undergo up-dation and dynamically calculates the refresh time of the page for its next update.

Keywords

World Wide Web, Search engine, Incremental Crawler, Hypertext, Browser

URL

http://paper.ijcsns.org/07_book/200812/20081250.pdf