Toshiba BRIDJE at NTCIR-6 CLIR: The Head/Lead Method and Graded Relevance Feedback

作者: Tatsuya Izuha , Akira Kumano , Toshihiko Manabe , Tomoharu Kokubu , Tetsuya Sakai

DOI:

关键词:

摘要: At NTCIR-6 CLIR, Toshiba participated in the Monolingual and Bilingual IR tasks covering three topic languages (Japanese, English Chinese) one document language (Japanese). For Stage 1 (which is usual ad hoc task using new NTCIR6 topics), we submitted two DESCRIPTION runs TITLE for each language. Our first search strategy Selective Sampling with Memory Resetting, our second Head/Lead method, which uses run as of components data fusion. According to Relaxed Rigid Mean Average Precision statistics released by organisers, are top performer all six subtasks. 2 reused NTCIR-3, 4 5 test collections), repeated strategies order enable analysis across four collections. Moreover, conducted some unofficial true relevance feedback experiments exploiting graded provided automatic results show that method slightly but consistently improves performance, while “interactive” suggest graded-relevance metrics favour favours binary feedback. In addition, significance tests Japanese collection “harder” than previous

参考文章(16)
Kiduk Yang, Robert G. Sumner, William M. Shaw, Roger Akers, Interactive retrieval using IRIS : TREC-6 experiments text retrieval conference. pp. 711- 733 ,(1997)
Emine Yilmaz, Javed A. Aslam, Estimating average precision with incomplete and imperfect judgments Proceedings of the 15th ACM international conference on Information and knowledge management - CIKM '06. pp. 102- 111 ,(2006) , 10.1145/1183614.1183633
Stephen Robertson, On GMAP Proceedings of the 15th ACM international conference on Information and knowledge management - CIKM '06. pp. 78- 83 ,(2006) , 10.1145/1183614.1183630
TETSUYA SAKAI, Advanced technologies for information access International Journal of Computer Processing of Languages. ,vol. 18, pp. 95- 114 ,(2005) , 10.1142/S0219427905001274
Tetsuya Sakai, On the reliability of factoid question answering evaluation ACM Transactions on Asian Language Information Processing. ,vol. 6, pp. 3- ,(2007) , 10.1145/1227850.1227853
Tetsuya Sakai, Evaluating evaluation metrics based on the bootstrap Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '06. pp. 525- 532 ,(2006) , 10.1145/1148170.1148261
Tetsuya Sakai, Alternatives to Bpref international acm sigir conference on research and development in information retrieval. pp. 71- 78 ,(2007) , 10.1145/1277741.1277756
Gabriella Kazai, Mounia Lalmas, eXtended cumulated gain measures for the evaluation of content-oriented XML retrieval ACM Transactions on Information Systems. ,vol. 24, pp. 503- 542 ,(2006) , 10.1145/1185877.1185883
Tetsuya Sakai, Toshihiko Manabe, Makoto Koyama, Flexible pseudo-relevance feedback via selective sampling ACM Transactions on Asian Language Information Processing. ,vol. 4, pp. 111- 135 ,(2005) , 10.1145/1105696.1105699
Chris Buckley, Ellen M. Voorhees, Retrieval evaluation with incomplete information Proceedings of the 27th annual international conference on Research and development in information retrieval - SIGIR '04. pp. 25- 32 ,(2004) , 10.1145/1008992.1009000