AI-Assisted Human Labeling: Batching for Efficiency without Overreliance

摘要: Human labeling of training data is often a time-consuming, expensive part machine learning. In this paper, we study "batch labeling", an AI-assisted UX paradigm, that aids labelers by allowing single action to apply multiple records. We ran large scale on Mechanical Turk with 156 participants investigate labeler-AI-batching system interaction. the efficacy when compared single-item interface (i.e., one record at-a-time), and evaluate impact batch accuracy time. further AI algorithm quality its effects labelers' overreliance, as well potential mechanisms for mitigating it. Our work offers implications design systems practices focusing