Improving the Reproducibility of PAN's Shared Tasks: - Plagiarism Detection, Author Identification, and Author Profiling.

摘要: This paper reports on the PAN 2014 evaluation lab which hosts three shared tasks plagiarism detection, author identification, and profiling. To improve reproducibility of in general, PAN’s particular, Webis group developed a new web service called TIRA, facilitates software submissions. Unlike many other labs, asks participants to submit running softwares instead their run output. deal with organizational overhead involved handling submissions, TIRA experimentation platform helps significantly reduce workload for both organizers, whereas submitted are kept state. year, we addressed matter responsibility successful execution order put back charge executing at our site. In sum, 57 have been lab; together 58 submissions last this forms largest collection date, all readily available further analysis. The report concludes brief summary each task.