Latest News
Cross Language Evaluation Form: CLEF 2009 Call for Participation
Registration for CLEF 2009 is now... - Link INFILE: INFILE : Information, Filtering, Evaluation Evaluation campaign for information... - Link
|
|
TrebleCLEF Flyer
TrebleCLEF Deliverables & Reports
D6.1 - Dissemination Plan
D2.2 - Operational Scientific Digital Library
D5.1.1 - Evauation Resources for CLEF
D2.1.1 - CLEF 2008 Evaluation Campaign Workshop
D3.1 - System Developers Workshop
D3.2 - MLIA User Communities Workshop
D2.3.1 - Analysis of Evaluation Campaign Results
by Mark Sanderson of the Workshop on Novel Technologies for Evaluation in Information Retrieval - held at ECIR 2008, Glasgow, UK, 30 March, 2008
TrebleCLEF Released Publications
- Amigó E., Gonzalo J., Artiles J., Verdejo F., 2008. Comparison of extrinsic clustering evaluation metrics based on formal constraints. Information Retrieval Journal.
-
Braschler M., Di Nunzio G., Ferro N., Gonzalo J., Peters C., Sanderson M., 2007. From CLEF to TrebleCLEF: Promoting Technology Transfer for Multilingual Information Retrieval. In: Thanos C., Borri F., Launaro A. editors, Working Notes of the Second DELOS Conference on Digital Libraries, Pisa, Italy.
paper
-
Braschler M., Ferro N., Gonzalo J., Sanderson M., 2008. Workshop on Novel Methodologies for Evaluation in Information Retrieval. In: Advances in Information Retrieval. 30th European Conference on IR Research, ECIR 2008, Glasgow, UK, March 30-April 3, 2008. Proceedings. Lecture Notes in Computer Science (LNCS) 4956, Springer Berlin/Heidelberg, p. 713.
Link to the Springer page
- Clough P., Gonzalo J., Karlgren J., Barker E., Artile J., Peinado V., 2008. Large-Scale Interactive Evaluation of Multilingual Information Access Systems - the iCLEF Flickr Challenge. In Workshop on Novel Methodologies for Evaluation (ECIR 2008).
google scholar
-
Dussin M., Ferro N., 2008. Direct: Applying the DIKW Hierarchy to Large-scale Evaluation Campaigns. In ACM Press, 2008, Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2008), NY, USA, p. 424.
Link to the ACM portal
-
Dussin, M. and Ferro, N., 2008. The Design of the User Interface of a Scientific DLS in the context of the Data, Information, Knowledge and Wisdom. In Agosti, M. and Thanos, C., editors, Proceedings of the Fourth Italian Research Conference on Digital Library Systems (IRCDL 2008).
- Dussin M., Ferro N. (2008). The Role of the DIKW Hierarchy in the Design of a Digital Library System for the Scientific Data of Large-scale Evaluation Campaigns. In ACM Press, 2008, Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2008), NY, USA, p. 450.
Link to the ACM portal
-
Ferro, N. and Harman, D., 2008. Dealing with MultiLingual Information Access: Grid Experiments at TrebleCLEF. In Agosti, M. and Thanos, C., editors, 2008, Proceedings of the Fourth Italian Research Conference on Digital Library Systems (IRCDL 2008).
- Herrera J., Peñas A., Verdejo F., LNAI, 2006. Textual Entailment Recognition Based on Dependency Analysis and WordNet. In: J. Quinonero-Candela, I. Dagan, B. Magnini, F. dAlché-Buc editors. MLCW 2005; . 231.
google scholar
- J. Herrera, A. Peñas, F. Verdejo, LNAI, 2006. Techniques for Recognizing Textual Entailment and Semantic Equivalence. In: Postproceedings of the XI Conference of the Spanish Association for Artificial Intelligence. Santiago de Compostela, Spain. 2006.
google scholar
- Herrera J., Peñas A., Verdejo M., 2007. Paraphrase Extraction from Validated Question Answering Corpora in Spanish. In Working Notes of the XXIII Conference of the Spanish Association for Natural Language Processing, (SEPLN 2007).
google scholar
- López-Ostenero F., Peinado V., Gonzalo J., Verdejo F., 2008. Interactive Question Answering: Is Cross-Language Harder than Monolingual Searching? In Information Processing and Management. Special topic issue on User-centered Evaluation of Information Retrieval Systems. 66-81.
google scholar
- Peñas A., Rodrigo Á., Verdejo F., 2006. SPARTE, a Test Suite for Recognising Textual Entailment in Spanish. In Alexander F. Gelbukh editor, 2006, Computational Linguistics and Intelligent Text Processing, 7th International Conference, CICLing 2006, Mexico City, Mexico, February 19-25, 2006, Proceedings, Lecture Notes in Computer Science. p. 275.
google scholar
- Peñas A., Sutcliffe R., 2006. Multilingual Question Answering - MLQA'06. EACL 2006 Workshop Proceedings.
google scholar
- Peñas A., Rodrigo Á., Sama V., Verdejo F., 2008. Testing the Reasoning for Question Answering Validation. Journal of Logic and Computation, Special Issue on Natural Language and Knowledge Representation,. To appear (draft version).
- Rodrigo Á., Peñas A., Verdejo F., 2008. Evaluating Answer Validation in multi-stream Question Answering. In Proceedings of The Second International Workshop on Evaluating Information Access (EVIA-NTCIR 2008).
- Téllez-Valero A., Montes-y-Gómez M., Villaseñor-Pineda L., Peñas A., 2008. Improving Question Answering by Combining Multiple Systems via Answer Validation. In Computational Linguistics and Intelligent Text Processing (CICLing 2008).
Related Literature
Multilingual Information Access / Information Retrieval System Evaluation
CLEF Publications
Pubications of Interest
- De Groot & van Laer “The quality of legal dictionaries: an assesment” (full text on http://arno.unimaas.nl/show.cgi?fid=13383 (pdf); abstract on http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1287603 ) [Maastricht Faculty of Law working paper & Social Science Research Network respectively].
Not only criteria for reliable bilingual dictionaries have been developed (16 pages), but these criteria have been applied on more than 200 bilingual legal dictionaries as well (bibliography, 45 pages). The final, maybe provocative, conclusion is that most legal dictionaries are of dubious quality and therefore not useful to professional translators.
|
