PUBCRAWL: protecting users and businesses from CRAWLers
作者： Engin Kirda , Christopher Kruegel , Giovanni Vigna , Gregoire Jacob
DOI:
关键词: Computer security 、 Web crawler 、 World Wide Web 、 Computer science 、 Spamming 、 Phishing 、 Containment (computer programming) 、 Crawling 、 Service provider 、 Technology transfer 、 Block (data storage)
摘要: … to identify crawlers with high accuracy, including crawlers that were previously-unknown to the social networking site. We also identified interesting campaigns of distributed crawlers. …
ucsb.edu参考文章(17)
A. Stassopoulou, M.D. Dikaiakos, Crawler Detection: A Bayesian Approach international conference on image and signal processing. pp. 16- 16 ,(2006) , 10.1109/ICISP.2006.7
R. B. Cleveland, STL : A Seasonal-Trend Decomposition Procedure Based on Loess Journal of Office Statistics. ,vol. 6, pp. 3- 73 ,(1990)
Kamal Nigam, Andrew McCallum, A comparison of event models for naive bayes text classification national conference on artificial intelligence. pp. 41- 48 ,(1998)
Pascal Thubert, Eric Levy-Abegnoli, Marc Lamberton, System and method for enabling a web site robot trap ,(2001)
Pang-Ning Tan, Vipin Kumar, Discovery of Web Robot Sessions Based on their Navigational Patterns Data Mining and Knowledge Discovery. ,vol. 6, pp. 9- 35 ,(2002) , 10.1023/A:1013228602957
Anália Lourenço, Orlando Belo, Applying Clickstream Data Mining to Real-Time Web Crawler Detection and Containment Using ClickTips Platform GfKl. pp. 351- 358 ,(2007) , 10.1007/978-3-540-70981-7_39
Eamonn Keogh, Shruti Kasetty, On the need for time series data mining benchmarks Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '02. pp. 102- 111 ,(2002) , 10.1145/775047.775062
William S. Cleveland, Susan J. Devlin, Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting Journal of the American Statistical Association. ,vol. 83, pp. 596- 610 ,(1988) , 10.1080/01621459.1988.10478639
Marios D. Dikaiakos, Athena Stassopoulou, Loizos Papageorgiou, An investigation of web crawler behavior: characterization and metrics Computer Communications. ,vol. 28, pp. 880- 897 ,(2005) , 10.1016/J.COMCOM.2005.01.003
Derek Doran, Swapna S. Gokhale, Web robot detection techniques: overview and limitations Data Mining and Knowledge Discovery. ,vol. 22, pp. 183- 210 ,(2011) , 10.1007/S10618-010-0180-Z