First Call for Papers - MultiLing 2017
----------------------------------------------

MultiLing 2017 Workshop in EACL 2017
Summarization and summary evaluation across source types and genres

April 3/4, 2017
Valencia, Spain

Website: http://multiling.iit.demokritos.gr/pages/view/1616/multiling-2017

Topics:
=======
We invite submissions on the following (non-exhaustive) topics:
- Multilingual and cross-lingual summarization (main focus, horizontal across all other topics)
- Single-document summarization
- Multi-document summarization
- Summarization evaluation
- Headline generation
- Cross-domain/cross-topic summarization
- Summarization in different genres and sources (news, social media, fora, transcripts, literature, etc.)
- Sentiment/opinion summarization
- Multilingual/cross-lingual summarization corpus creation
- Machine learning for summarization
- Research works building upon previous and current MultiLing community tasks (elaborated below in the Community Tasks section)

In all the above topics we encourage authors to:
- support repeatability of experiments (through sharing data, tools, algorithm implementations);
- demonstrate the reusability and scope of methods by applying them on different datasets;
- provide insights related to strengths and weaknesses of methods;
- build upon existing MultiLing tasks and datasets (http://multiling.iit.demokritos.gr/search?q=dataset&search_type=tags), but not limit themselves to these.

What is new:
============
We stress that, this year, MultiLing introduces novelty:
- we have expanded the scope, accepting contributions beyond the MultiLing community tasks
- we have added the headline generation community task (elaborated below)

Organizers:
===========
    George Giannakopoulos - NCSR Demokritos (General Chair, Summary Evaluation Task)
    Benoit Favre - LIF (CCCS, Headline Generation Task)
    Elena Lloret - University of Alicante (Headline Generation Task)
    John M. Conroy - IDA Center for Computing Sciences (Single Document Summarization Task)
    Josef Steinberger - University of West Bohemia (OnForumS task)
    Marina Litvak - Sami Shamoon College of Engineering (Headline Generation Task)
    Peter Rankel - Elder Research Inc. (Single Document Summarization Task)


Programme Committee:
====================
    Udo Kruschwitz - University of Essex
    Horacio Saggion - Universitat Pompeu Fabra
    Katja Filippova - Google
    John M. Conroy - IDA Center for Computing Sciences
    Vangelis Karkaletsis - NCSR Demokritos
    Laura Plaza - UNED
    Francesco Ronzano - Universitat Pompeu Fabra
    Mark Last - Ben-Gurion University of the Negev
    George Petasis - NCSR Demokritos
    Elena Lloret - University of Alicante
    Ahmet Aker - Universität Duisburg-Essen
    Josef Steinberger - University of West Bohemia
    Benoit Favre - LIF
    Marina Litvak - Sami Shamoon College of Engineering
    Mijail Alexandrov Kabadjov - University of Essex
    Natalia Vanetik - Sami Shamoon College of Engineering, Ben-Gurion University of the Negev
    Florian Boudin - University of Nantes
    Mahmoud El-Haj - Lancaster University
    George Giannakopoulos - NCSR Demokritos

Submissions and Important Dates:
================================
We accept short paper (4 pages max) and long paper (8 pages max) submissions, according to the EACL formatting guidelines (http://eacl2017.org/index.php/calls/call-for-papers, "SUBMISSION REQUIREMENTS" and "SUBMISSION FORMAT" sections). As per the EACL guidelines, short papers are expected to hold a self-sufficient, specific point, which can fit in a few pages: a small, focused contribution; work in progress; early findings; a negative result; an opinion piece. Long paper submissions, on the other hand, are expected to describe substantial, original, completed and unpublished work.

All works should comply with a double blind approach, i.e. (based on the EACL 2017 website):
- papers must not include authors' names and affiliations.
- self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ..." must be avoided. Instead, use citations such as "Smith previously showed (Smith, 1991) ..."
Papers that do not conform to these requirements will be rejected without review.

For the camera-ready submissions, the accepted papers at all the workshops will have to be made available both as LaTeX sources and PDF files.

The important dates related to the submissions are:
- Jan 16, 2017: Workshop Paper Due Date
- Feb 11, 2017: Notification of Acceptance
- Feb 21, 2017: Camera-ready papers due

Make sure you revisit the MultiLing 2017 website frequently for updated information (http://multiling.iit.demokritos.gr/pages/view/1616/multiling-2017).

About Multiling:
================
The first MultiLing was implemented as a pilot in TAC 2011, aiming to focus on the multilingual aspect of summarization. The momentum of the community gave birth to the biennial cycle of MultiLing workshops with MultiLing 2013 (in ACL 2013), MultiLing 2015 (in SIGDIAL 2015) and, now, MultiLing 2017 in EACL 2017. The number of interested and participating research groups in MultiLing has grown from 8 (in 2011) to more than 20 in 2015, while the current community members in the MultiLing website are around 50.

Originally, MultiLing built upon community tasks in a multilingual setting: single- and multi-document summarization, summary evaluation, online fora summarization, transcript summarization. This year we are opening the workshop to submissions beyond the community tasks, aiming to increase the scope and impact of the workshop. This decision was supported by the feedback of the community members and allows contributors beyond the task participants to submit their work, submissions regarding experiments on existing results, as well as extensions of previous works with significant added value (new experiments, new findings, deeped study).


Community Tasks:
================
The following tasks will run in the MultiLing community (before, but also beyond MultiLing 2017):

* Single Document Summarization.
Following the task of 2015, the multi-lingual single-document summarization task will be to generate a single document summary for all the given Wikipedia feature articles from one of about 38 languages provided. The provided training data will be the Single-Document Summarization Task data from MultiLing 2015. A new set of data will be generated based on additional Wikipedia feature articles. The summaries will be evaluated via automatic methods and participants will be required to perform some limited summarization evaluations. The manual evaluation will consist of pairwise comparisons of machine(-generated) summaries. Each evaluator will be presented the human(-generated) summary and two machine summaries. The evaluation task is to read the human summary and judge if the one machine summary is significantly closer to the human summary information content (e.g. system A > system B) or if the two machine summaries contain comparable quantity of information as the human summary.

* Headline Generation Task.
The objective of the Headline Generation (HG) task is to
explore some of the challenges highlighted by current state of the art approaches on creating informative headlines to news articles: non-descriptive headlines, out-of-domain training data, and generating headlines from long documents which are not well represented by the head heuristic. We propose to make available a large set of training data for headline generation, and create evaluation conditions which emphasize those challenges. We will also rerun the task in DUC 2004 conditions in order to create comparable results.

* Summary Evaluation Task.
This task aims to examine how well automated systems can evaluate summaries from different languages. This task takes as input the summaries generated from automatic systems and humans in the Summarization Tasks of MultiLing 2015, but also in the Single document summarization tasks of 2015 and 2017 (when the latter is completed). The output should be a grading of the summaries. Ideally, we would want the automatic evaluation to maximally correlate to human judgement, thus the evaluation will be based on correlation measurement between estimated grades and human grades.

* Online Forum Summarization (OnForumS) task. Further to the successful pilot of
OnForumS at MultiLing 2015, we are organizing the task again in 2017 with a brand new dataset. The OnForumS task investigates how the mass of comments found on news providers web sites (e.g., The Guardian) can be summarized. We posit that a crucial initial step towards that goal is to determine what comments link to, be that either specific news snippets or comments by other users. Furthermore, a set of labels for a given link may be articulated to capture phenomena such as agreement and sentiment with respect to the comment target. Solving this labelled linking problem can enable recognition of salience (e.g., snippets/comments with most links) and relations between comments (e.g., agreement). The evaluation will focus on how many of the links and labels were correctly identified, as in the previous OnForumS run.

* Call Centre Conversation Summarization (CCCS) task.
The Call Centre Conversation Summarization (CCCS) task - run for the first time as a pilot task in 2015 - consists in automatically generating summaries of spoken conversations in the form of textual synopses that shall inform on the content of a conversation and might be used for browsing a large database of recordings. As in CCCS 2015, participants to the task shall generate abstractive summaries from conversation transcripts that inform a reader about the main events of the conversations, such as the objective of the participants and how they are met. Evaluation will be performed by ROUGE-like measures based on human-written summaries as in CCCS 2015, and - if possible - will be coupled by manual evaluation, depending on the funding we can secure for the task.

For more information, visit the MultiLing community website (http://multiling.iit.demokritos.gr/).