To search, Click below search items. |
| All Published Papers Search Service |
|
|
|
Title |
Self Adjusting Refresh Time Based Architecture for Incremental Web Crawler |
|
Author |
A.K. Sharma, Ashutosh Dixit |
| Citation |
Vol. 8 No. 12 pp. 349-354 |
|
Abstract |
Due to the deficiency in their refresh techniques [12], current crawlers add unnecessary traffic to the already overloaded Internet. Moreover there exist no certain ways to verify whether a document has been updated or not. In this paper, an efficient approach is being proposed for building an effective incremental web crawler [13]. It selectively updates its database and/ or local collection of web pages instead of periodically refreshing the collection in batch mode thereby improving the “freshness” of the collection significantly and bringing new pages in more timely manner. It also detects web pages which frequently undergo up-dation and dynamically calculates the refresh time of the page for its next update. |
|
Keywords |
World Wide Web, Search engine, Incremental Crawler, Hypertext, Browser |
|
URL |
