Invited talk:

Title: Mining and Enriching Multilingual Scientific Text Collections

Speaker: Horacio Saggion

See additional information here.

Programme:

Regular workshop papers:
Papers submission deadline: 11 July 2019 21 July 2019
Papers acceptance notification: 28 July 2019

Community tasks:
Participant registration deadline: 20 May 2019
Release of test data: 4 July 2019
Submissions deadline:  11 July 2019 21 July 2019
Evaluations announced: 28 July 2019

Along with the community task roadmap above, please make sure to visit the dedicated page for each task of intereset for potential updates of deadlines and important dates.

(2019-09-06) Final workshop proceedings published.
(2019-09-06) Workshop notebook proceedings published.
(2019-09-03) Workshop program announced.
(2019-07-10) Submission deadline extended to July 21st.
(2019-07-10) Community task submission details updated.
(2019-07-5) Community task test data released.
(2019-05-20) The deadline for registration to the community tasks is today.
(2019-05-13) Training data for the Summary Evaluation community task published.
(2019-05-09) Community task important dates and information updated.
(2019-05-09) Second call for papers published, submission and notification dates updated.
(2019-05-01) Training data for the Headline Generation community task published.
(2019-05-01) Training data for the Financial Narrative Summarization community task published.
(2019-04-03) Workshop date fixed to the 6th of September.
(2019-03-26) Call for participation in the community tasks published.
(2019-03-26) First call for papers published.
(2019-02-05) MultiLing accepted as a RANLP 2019 workshop.

Main Info:

Multiling 2019 is an accepted RANLP 2019 workshop and will take place at the 6th of September at Varna, Bulgaria.
Be sure to check the roadmap page for important dates.

Invited speakers:
Horacio Saggion - Universitat Pompeu Fabra

MultiLing 2019 covers a variety of topics on Natural Language Processing, focused on the multi-lingual aspect of summarization, but also its value across different settings:

Multilingual summarization across genres and sources:

Summarization has been receiving increasing attention during the last years. This is mostly due to the increasing volume and redundancy of available online information but also due to the user created content. Recently, more and more interest arises for methods that will be able to function on a variety of languages and across different types of content and genres (news, social media, transcripts).

This topic of research is mapped to different community tasks, covering different genres and
source types: Multilingual single-document summarization [Giannakopoulos et al., 2015]; news
headline generation (new task in MultiLing 2019); financial narrative summarization (new task
in MultiLing 2019, under a view of synergy and complementary to other workshops as in FNP
2018 at LREC’18 [El-Haj et al., 2018]); user-supplied comments summarization (OnForumS
task [Kabadjov et al., 2015]); conversation transcripts summarization (see also [Favre et al.,
2015]). The spectrum of the tasks covers a variety of real settings, identifying individual re-
quirements and intricacies, similarly to previous MultiLing endeavours [Giannakopoulos et al.,
2011, Giannakopoulos, 2013, Elhadad et al., 2013, Giannakopoulos et al., 2015, Giannakopoulos
et al., 2017].

Multilingual summary evaluation:

Summary evaluation has been an open question for several years, even though there exist methods that correlate well to human judgement, when called upon to compare systems. In the multilingual setting, it is not obvious that these methods will perform equally well to the English language setting. In fact, some preliminary results have shown that several problems may arise in the multilingual setting [Giannakopoulos et al., 2011].

The same challenges arise across different source types and genres. This section of the workshop
aims to cover and discuss these research problems and corresponding solutions.

Community tasks:

MultiLing is supported by a number of community tasks (check the related links on the top menu of the website for information on previous runs). This year we plan to implement 3 main tasks:

• Headline generation
• Financial narrative summarization
• Summary evaluation

Stay tuned for more information on the tasks.

Programme Committee Members: (to be finalized)

 John M. Conroy - IDA Center for Computing Sciences
 Marina Litvak - Sami Shamoon College of Engineering
 Udo Kruschwitz - University of Essex
 Horacio Saggion - Universitat Pompeu Fabra
 Elena Lloret - University of Alicante
 Vangelis Karkaletsis - NCSR Demokritos
 Mark Last - Ben-Gurion University of the Negev
 Natalia Vanetik - Sami Shamoon College of Engineering; Ben-Gurion University of the Negev
 Josef Steinberger - University of West Bohemia
 Laura Plaza - UNED
 Florian Boudin - University of Nantes
 Iraklis Varlamis - Harokopio University
 George Petasis - NCSR Demokritos
 Peter Rankel - Stratus Solutions
 George Giannakopoulos - NCSR Demokritos
 Mahmoud El-Haj - Lancaster University
 Ahmet Aker - University of Sheffield
 Giuseppe Riccardi - University of Trento
 Corina Forascu - University "Al. I. Cuza" Iasi
 Michalis Vazirgiannis - Ecole Polytechnique; Athens University of Economics and Business

To stay updated on MultiLing2019, you can join the mailing list: multiling19 at scify dot org.

MultiLing covers a variety of topics on Natural Language Processing, focused on the multi-
lingual aspect of summarization:

- Multilingual summarization across genres and sources: Summarization has been receiving increasing attention during the last years. This is mostly due to the increasing volume and redundancy of available online information but also due to the user created content. Recently, more and more interest arises for methods that will be able to function on a variety of languages and across different types of content and genres (news, social media, transcripts).
This topic of research is mapped to different community tasks, covering different genres and source types: Multilingual single-document summarization [Giannakopoulos et al., 2015]; news headline generation (new task in MultiLing 2017); user-supplied comments summarization (OnForumS task [Kabadjov et al., 2015]); conversation transcripts summarization (see also [Favre et al., 2015]). The spectrum of the tasks covers a variety of real settings, identifying individual requirements and intricacies, similarly to previous MultiLing endeavours [Giannakopoulos et al., 2011, Giannakopoulos, 2013, Elhadad et al., 2013, Giannakopoulos et al., 2015].
- Multilingual summary evaluation: Summary evaluation has been an open question for
several years, even though there exist methods that correlate well to human judgement, when
called upon to compare systems. In the multilingual setting, it is not obvious that these methods
will perform equally well to the English language setting. In fact, some preliminary results have
shown that several problems may arise in the multilingual setting [Giannakopoulos et al., 2011].
The same challenges arise across different source types and genres. This section of the workshop
aims to cover and discuss these research problems and corresponding solutions.
The workshop will build upon the results of a set of research community tasks, which are
elaborated on in the following paragraphs.

Single document summarization

Following the pilot task of 2015, the multi-lingual single-document summarization task will be to generate a single document summary for all the given Wikipedia feature articles from one of about 40 languages provided. The provided training data will be the  Single-Document Summarization Task data from MultiLing 2015. A new set of data will be generated based on additional Wikipedia feature articles. The summaries will be evaluated via automatic methods and participants will be required to perform some limited summarization evaluations.
The manual evaluation will consist of pairwise comparisons of machine(-generated) summaries. Each evaluator will be presented the human(-generated) summary and two machine summaries. The evaluation task is to read the human summary and judge if the one machine summary is significantly closer to the human summary information content (e.g. system A > system B) or if the two machine summaries contain comparable quantity of information as the human summary.

Headline Generation

The objective of the
Headline Generation (HG) task is to explore some of the challenges
highlighted by current state of the art approaches on creating informative headlines to news
articles: non-descriptive
headlines, out-of-domain training data, and generating headlines from
long documents which are not well represented by the head heuristic.
We propose to make available a large set of training data for headline
generation, and create evaluation conditions which emphasize those
challenges. We will also rerun the task in DUC 2004 conditions in
order to create comparable results.

Summary Evaluation

This task aims to examine how well automated systems can evaluate summaries from
different languages. This task takes as input the summaries generated from automatic systems
and humans in the Summarization Tasks of MultiLing 2015, but also in the Single document summarization tasks of 2015 and 2017 (when the latter is completed).
The output should be a grading of the summaries. Ideally, we would want the automatic evaluation to maximally correlate to human
judgement, thus the evaluation will be based on correlation measurement between estimated grades and human grades.

Online Forum Summarization (OnForumS)

Further to the successful pilot of OnForumS at MultiLing 2015, we are organizing the task again
in 2017 with a brand new dataset.  The OnForumS task investigates
how the mass of comments found on news providers web sites (e.g., The Guardian)
can be summarized. We posit that a crucial initial step towards that goal is to determine
what comments link to, be that either specific news snippets or comments by other users.
Furthermore, a set of labels for a given link may be articulated to capture phenomena
such as agreement and sentiment with respect to the comment target. Solving this labelled
linking problem can enable recognition of salience (e.g., snippets/comments with most links)
and relations between comments (e.g., agreement). The evaluation will focus on how many of
the links and labels were correctly identified, as in the previous OnForumS run.

Call Centre Conversation Summarization (CCCS)

The Call Centre Conversation Summarization (CCCS) task --- run for the first time as a pilot task
in 2015 --- consists in automatically
generating summaries of spoken conversations in the form of textual
synopses that shall inform on the content of a conversation and might
be used for browsing a large database of recordings.
As in CCCS 2015, participants to the task shall generate abstractive
summaries from conversation transcripts that inform a reader about the
main events of the
conversations, such as the objective of the participants and how they are met.
Evaluation will be performed by ROUGE-like measures based on
human-written summaries as in CCCS 2015,
and --- if possible --- will be coupled by manual evaluation, depending on
the funding we can secure for the task.

• (2017-03-13) Schedule updated with camera-ready paper versions available in Event Information page.
• (2017-03-06) Proposed schedule and more information on invited talk now available in Event Information page.
• (2017-03-06) List of accepted papers now available in Event Information page.
• (2017-02-22) Deadline for final submissions (camera-ready) today (23:59, GMT-12).
• (2017-02-16) Anti-harassment Policy of ACL supported by MultiLing 2017. See more here.
• (2017-02-16) Decisions of papers will be sent out within the day.
• (2016-12-17) Submission deadline extended to Monday, Jan 30, 2017 (end of day GMT-12, i.e. end of day wherever you are in the world).
• (2016-12-16) Extension to the submission deadline (at least a week). Finalization of updated deadline pending.
• (2016-12-13) Paper submissions should be made at the following webpage: https://www.softconf.com/eacl2017/multiling2017/
• (2016-12-13) Second call for papers published
• (2016-12-05) Date of workshop fixed to April, 3rd 2017
• (2016-11-10) First call for papers published. Check the corresponding subpage.
• (2016-11-08) First call for papers in preparation.
• (2016-11-02) MultiLing accepted as an EACL 2017 Workshop.

Main Info:

MultiLing 2017 is an accepted, one-day EACL 2017 Workshop with this year's focus on

Summarization and summary evaluation across source types and genres

MultiLing 2017 will be collocated with EACL 2017 in Valencia, Spain on April 3, 2017.

For important dates check the Roadmap sub-page.

Below we provide a quick view of the Workshop information. Extended information (overview and tasks, roadmap, calls, per-task pages) can be found in the links advertised as sub-pages in the "Navigation" index on the right of this page.

Invited speakers (Finalization pending)

• Enrique Alfonseca - Google

Organizers

• George Giannakopoulos - NCSR Demokritos (General Chair, Summary Evaluation Task)
• Benoit Favre - LIF (CCCS, Headline Generation Task)
• Elena Lloret - University of Alicante (Headline Generation Task)
• John M. Conroy - IDA Center for Computing Sciences (Single Document Summarization Task)
• Josef Steinberger - University of West Bohemia (OnForumS task)
• Marina Litvak - Sami Shamoon College of Engineering (Headline Generation Task)
• Peter Rankel - Elder Research Inc. (Single Document Summarization Task)

Programme Committee (PC) members (finalization pending):

• Udo Kruschwitz - University of Essex
• Horacio Saggion - Universitat Pompeu Fabra
• Katja Filippova - Google
• John M. Conroy - IDA Center for Computing Sciences
• Vangelis Karkaletsis - NCSR Demokritos
• Laura Plaza - UNED
• Francesco Ronzano - Universitat Pompeu Fabra
• Mark Last - Ben-Gurion University of the Negev
• George Petasis - NCSR Demokritos
• Elena Lloret - University of Alicante
• Ahmet Aker - Universität Duisburg-Essen
• Josef Steinberger - University of West Bohemia
• Benoit Favre - LIF
• Marina Litvak - Sami Shamoon College of Engineering
• Mijail Alexandrov Kabadjov - University of Essex
• Natalia Vanetik - Sami Shamoon College of Engineering
• Florian Boudin - University of Nantes
• Mahmoud El-Haj - Lancaster University
• George Giannakopoulos - NCSR Demokritos

Additional information will be made available on this page (and subpages), as they are available.

(2015-08-30) The session program can be found in the Schedule page.

(2015-08-30) System reports and overviews are now available in the Addendum page.

(2015-04-20) MultiLing accepted as SIGDIAL session. NOTE: Submission deadlines and format will be aligned to the corresponding SIGDIAL requirements (cf. here and here for more info).

(2015-04-10) CCCS test data released. See CCCS page.

(2015-03-14) Updated dates in all tasks. Please check corresponding pages (on the right).

(2015-02-18) MSS test data made available. Check the task page for more information.

(2015-02-14) MMS test data made available. Check the task page for more information.

(2015-01-17) Sample data on MMS have been published. Follow the links to the right for more information.

(2015-01-07) Sample data for OnForumS and training data on MSS have been published. Follow the links on the right for more information.

(2014-12-07) 1st call for participation is out. Check it here.

NOTE: On the right you will find several links covering different aspects of the MultiLing 2015 event.

Overview

MultiLing 2013 is a community effort, a set of research tasks and a corresponding workshop which covers three subdomains of Natural Language Processing, focused on the multilingual aspect of summarization. Each domain is allocated a separate section of the workshop.
The three domains are:

• Multilingual multi-document summarization: Summarization, especially from multiple documents, has received increasing attention during the last years. This is mostly due to the increasing volume and redundancy of available online information. Recently, more and more interest arises for methods that will be able to function on a variety of languages. Multilingual multi-document summarization is the domain that researches such methods and studies their requirements and intricacies.
• Multilingual summary evaluation: Summary evaluation has been an open question for several years, even though there exist methods that correlate well to human judgement, whencalled upon to compare systems. In the multilingual setting, it is not obvious that these methods will perform equally well to the English language setting.
  In fact, some preliminary results have shown that several problems may arise in the multilingual setting  [Giannakopoulos et al., 2011]. This section of the workshop aims to cover and discuss these research problems and corresponding solutions.
• Multilingual summarization data collection and exploitation: The collection of multi-lingual corpora for summarization and summarization evaluation offers a challenge in itself.
  This section of the workshop works towards well-defined practices for the collection of such data, as well as the implementation and use of community tools for the support of the collection process. Furthermore, this section will include a discussion on how we can maximize the effect of the generated corpora in favor or the scientific community.

References:

[Giannakopoulos et al., 2011] Giannakopoulos, G., El-Haj, M., Favre, B., Litvak, M., Steinberger, J., and Varma,
V. (2011). TAC2011 MultiLing Pilot Overview.

The workshop will build upon the results of a set of research community tasks, described here.

Venue

MultiLing 2013 will be held as part of ACL 2013 (Sofia, Bulgaria's capital, August 4-9, 2013).

MultiLing will take place on the last day of ACL (August 9, 2013).

Acknowledgements

The MultiLing 2013 Workshop has been partially supported by the NOMAD Project.

Address of the bookmark: http://www.nist.gov/tac/