Systems and Methods for the Determining Annotator Performance in the Distributed Annotation of Source Data
作者： Peter Welinder , Pietro Perona
DOI:
关键词: Metadata 、 Source data 、 Computer science 、 Set (abstract data type) 、 Information retrieval 、 Training set 、 Cluster analysis 、 Annotation
摘要: Systems and methods for determining annotator performance in the distributed annotation of source data accordance embodiments invention are disclosed. In one embodiment invention, a method clustering annotators includes obtaining set data, training representative sets annotations from portion set, each recall metadata based on provided by precision grouping into groups metadata.
lens.org 参考文章(87)
Matthew E. Kulick, Adam D. Bursey, Search result annotations ,(2011)
Simon Tong, Daphne Koller, Support vector machine active learning with applications to text classification Journal of Machine Learning Research. ,vol. 2, pp. 45- 66 ,(2002) , 10.1162/153244302760185243
Panagiotis G. Ipeirotis, Joshua M. Attenberg, Foster J. Provost, Methods, systems, and media for identifying errors in predictive models using annotators ,(2012)
Namejs Reinbachs, Carl Suttle, Sam Abuzalaf, Methods and systems for facilitating online collaboration and distribution of geospatial data ,(2005)
Lubomir Bourdev, Jitendra Malik, Poselets: Body part detectors trained using 3D human pose annotations international conference on computer vision. pp. 1365- 1372 ,(2009) , 10.1109/ICCV.2009.5459303
Zoubin Ghahramani, David A. Cohn, Michael I. Jordan, Active learning with statistical models Journal of Artificial Intelligence Research. ,vol. 4, pp. 129- 145 ,(1996) , 10.5555/1622737.1622744
H. Mannila, A. Gionis, P. Tsaparas, Clustering Aggregation (long version) ACM Transactions on Knowledge Discovery From Data. ,vol. 1, ,(2007)
Kevin Bache, Moshe Lichman, UCI Machine Learning Repository University of California, School of Information and Computer Science. ,(2007)
A. P. Dawid, A. M. Skene, Maximum Likelihood Estimation of Observer Error‐Rates Using the EM Algorithm Journal of The Royal Statistical Society Series C-applied Statistics. ,vol. 28, pp. 20- 28 ,(1979) , 10.2307/2346806
Serge Belongie, Peter Welinder, Florian Schroff, Pietro Perona, Steve Branson, Takeshi Mita, Catherine Wah, Caltech-UCSD Birds 200 California Institute of Technology. ,(2010)