Search engine behavior towards automated querying

摘要: Automated querying systems can sometimes form a large portion of the overall traffic search engine, thus degrading its performance. A serious challenge to engine designers is ensure that while third-party automated agents are blocked by their defense mechanisms, human users not put inconvenience. We propose study how mechanisms commercial, large-scale engines respond querying, using proprietary agent. These responses quantified, and presented in state models.