A Critical Review of Migrating Parallel Web Crawler

摘要: The size of the internet is very large and it has grown enormously, search engines are tools for World Wide Web navigation. In order to provide powerful facilities, maintain comprehensive indices documents their contents on by continuously downloading pages processing, known as web crawling. this paper we reviewed various crawlers performance attributes. We study mobile parallel crawling approach that makes system more effective efficient. major advantage analysis portion process done locally where data resides rather than remotely inside engine. This can significantly reduce net- work load which, in turn, improve process. grows, becomes imperative parallelize a process, finish reasonable amount time. identify fundamental issues related migrating also propose metrics evaluate crawler. Lastly, summarize attributes effects