CFP: CLEF 2010

Note that CLEF is now a conference, not just an evaluation workshop:

---

Call for Papers

CLEF 2010 - Conference on Multilingual and Multimodal Information Access Evaluation

20–23 September 2010, Padua, Italy

http://www.clef2010.org/


Background

CLEF2010 is the continuation of the popular CLEF campaigns that have run for the past ten years. It will cover a broad range of issues in the fields of multilingual and multimodal information access evaluation. It will consist of two main parts: a peer-reviewed conference (20-21
September) on experimental evaluation, which will innovate the CLEF tradition, and a series of labs (22-23 September), which will continue the CLEF tradition of community-based evaluation and discussion on evaluation issues.


Aim and Scope

The CLEF2010 conference aims at advancing the evaluation of complex multimodal and multilingual information systems in order to support individuals, organizations, and communities who design, develop, employ, and improve such systems.

The growth of the Internet has been exponential with respect to the number of users, media, and languages used regularly for global information dissemination. Language and media barriers are no longer seen as inviolable and they are constantly crossed and mixed to provide content that can be accessed on a global scale within a multicultural and multilingual setting.

Users need to be able to co-operate and communicate across language and media boundaries, going beyond separate search in diverse media/languages and exploiting interactions between different languages and media.

Experimental evaluation - both laboratory and interactive - is a key to fostering the development of multilingual and multimodal information systems that address increasingly complex information needs.

The CLEF2010 labs will continue the CLEF tradition of community-based benchmarking and will complement these with workshops on emerging issues in evaluation methodology. Lab proposals were sollicited at an earlier stage and and the 2010 lab program can be found at http://www.clef2010.org/.

At this time we invite submissions for presentation at the CLEF2010 conference. We welcome submissions on all aspects of multilingual and multimodal information access evaluation. All submissions will be reviewed on the basis of relevance, originality, importance, and clarity.

The conference proceedings will be published in the Springer Lecture Notes in Computer Science (LNCS) series.


Topics for Discussion

Relevant topics for the CLEF 2010 include, but are not limited to:

- Novel methodologies for the design of evaluation tasks, especially user-centric ones;
- Analysis of the impact of multilingual/multicultural/multimodal
differences in interface and search design;
- Assessing multilinguality and multimodality in relevant application communities, e.g. digital libraries, intellectual property, medical, music, video, and social media.
- Alternative methods for improving and automating ground-truth creation, for example crowd-sourcing or clicklog-based;
- Prediction of success and satisfaction rates;
- Task-oriented metrics of success and failure;
- Evaluation of technology vs testing of scientific theories;
- Innovative and easy to communicate techniques for analysing the experimental results, including statistical analyses, data mining, and information visualization;
- Alternatives for and comparison of item-based, list-based, set-based, and session-based evaluation;
- Simulation (of queries, sessions, users) and information retrieval;
- Infrastructures for bringing automation and collaboration in the evaluation process;
- Component-based evaluation approaches;
- Evaluation and analysis using private or anonymized test data;
- Living laboraties and evaluating live systems.


Important Dates

- Submission Deadline: 2 May 2010
- Notification of Acceptance: 11 June 2010
- Camera Ready: 25 June 2010
- Conference: 20-21 September 2010.


Format

Authors are invited to submit electronically original papers, which have not been published and are not under consideration elsewhere, using the LNCS proceedings format (http://www.springer.com/computer/lncs?SGWID=0-164-7-72376-0).

Two types of papers are solicited:
- long papers: 12 pages max;
- short papers: 6 pages max.

Papers will be peer-reviewed by at least 3 members of the program committee. Selection will be based on originality, clarity, and technical quality.

Papers should be submitted in PDF format to the following address:
http://www.easychair.org/conferences/?conf=clef2010


Organization

Honorary Chair
- Carol Peters, ISTI-CNR Pisa, Italy

General Chairs
- Maristella Agosti, University of Padua, Italy
- Maarten de Rijke, University of Amsterdam, The Netherlands

Programme Chairs
- Nicola Ferro, University of Padua, Italy
- Alan Smeaton, Dublin City University, Ireland

Organization Chair
- Emanuele Pianta, CELCT Trento, Italy


Program Committee

- Eneko Agirre, University of the Basque Country, Spain
- Giambattista Amati, Fondazione Ugo Bordoni, Italy
- Julie Berndsen, University COllege Dublin, Ireland
- Pia Borlund, Royal School of Library and Information Science, Denmark
- Pavel Braslavski, Yandex and Ural State Tech. University, Ural State University, Russia
- Chris Buckley, Sabir Research, USA
- Pável Calado, Superior Technical Institute, Portugal
- Tiziana Catarci, Sapienza, University of Rome, Italy
- Stefano Ceri, Politecnico di Milano, Italy
- Charlie Clark, University of Waterloo, Canada
- Bruce Croft, University of Massachusetts, Amherst, USA
- Franca Debole, Institute of Information Science and Technologies (ISTI), Italian National Research Council (CNR) , Italy
- J. Stephen Downie, University of Illinois at Urbana-Champaign, USA
- Susan Dumais, Microsoft Research, USA
- Floriana Esposito, University of Bari, Italy
- Marcello Federico, Fondazione Bruno Kessler, Italy
- Norbert Fuhr, University of Duisburg, Germany
- Fredric Gey, University of California, Berkeley, USA
- Marcos André Gonçalves, Federal University of Minas Gerais, Brasil
- Julio Gonzalo, National Distance Learning University, Spain
- Gregory Grefenstette, Exalead, France
- Allan Hanbury, Information Retrieval Facility (IRF), Austria
- Peter Ingwersen, Royal School of Library and Information Science, Denmark
- Perla Innocenti, University of Glasgow, UK
- Kalervo Järvelin, University of Tampere, Finland
- Gareth Jones, Dublin City University, Ireland
- Theodore Kalamboukis, Athen University of Economics and Business, Greece
- Leonid Andreevich Kalinichenko, Institute for Problems of Informatics of the Russian Academy of Sciences, Russia
- Noriko Kando, National Institute of Informatics (NII), Japan
- Sarantos Kapidakis, Ionian University, Greece
- Jussi Karlgren, Swedish Institute of Computer Science (SICS), Sweden
- Kazuaki Kishida, Keio University, Japan
- Mikko Kurimo, Helsinki University of Technology, Finland
- Mounia Lalmas, University of Glasgow, UK
- Ray Larson, University of California, Berkeley, USA
- Chin-Yew Lin, Microsoft Research Asia, China
- Saturnino Luz, Trinity College Dublin, Ireland
- Bernardo Magnini, Fondazione Bruno Kessler, Italy
- Prasenjit Majumder, Dhirubhai Ambani Institute of Information and Communication Technology, India
- Thomas Mandl, University of Hildesheim, Germany
- Paul McNamee, Johns Hopkins University, USA
- Mandar Mitra, Indian Statistical Institute, India
- Stefano Mizzaro, University of Udine, Italy
- Alistair Moffat, University of Melbourne , Australia
- Viviane Moreira Orengo, Federal University of Minas Gerais, Brasil
- Henning Müller, University of Applied Sciences Western Switzerland, Switzerland
- Jian-Yun Nie, University of Montreal, Canada
- Doug Oard, University of Maryland, USA
- Nicola Orio, University of Padua, Italy
- Christos Papatheodorou, Ionian University, Greece
- Gabriella Pasi, University of Milan Bicocca, Italy
- Anselmo Peñas, National Distance Learning University, Spain
- Vivien Petras, Humboldt University, Germany
- Jean-Michel Renders, Xerox Research Centre (XRCE), France
- Seamus Ross, University of Toronto, Canada
- Ian Ruthven, University of Strathclyde, UK
- Tetsuya Sakai, Microsoft Research Asia, China
- Diana Santos, SINTEF Information and Communication Technology, Norway
- Giuseppe Santucci, Sapienza, University of Rome, Italy
- Frederique Segond, Xerox Research Centre (XRCE), France
- Giovanni Semeraro, University of Bari, Italy
- Paraic Sheridan, Centre for Next Generation Localisation, Ireland
- Ian Soboroff, National Institute of Standards and Technology (NIST), USA
- Alexander Sychov, Voronezh State University, Russia
- John Tait, Information Retrieval Facility (IRF), Austria
- Letizia Tanca, Politecnico di Milano, Italy
- Elaine Toms, Dalhousie University, Canada
- Andrew Trotman, University of Otago, New Zealand
- Natalia Vassilieva, HP Labs Russia, Russia
- Felisa Verdejo, National Distance Learning University, Spain
- Ellen Voorhees, National Institute of Standards and Technology (NIST), USA
- Gerhard Weikum, Max-Planck Institute for Informatics (MPII), Germany
- Christa Womser-Hacker, University of Hildesheim, Germany
- Justin Zobel, University of Melbourne , Australia