Download

NEW - You can download test data by clicking here.

You can download the sample and training data by clicking here.

Related information

Multilingual Single-document Summarization (MSS)

Following the pilot task of 2013, the multi-lingual single-document summarization task will be to generate a single document summary for all the given Wikipedia feature articles from one of about 40 languages provided. The provided training data will be the 2013 Single-Document Summarization Pilot Task data from MultiLing 2013. A new set of data will be generated based on additional Wikipedia feature articles. For each language 30 documents are given. The documents will be UTF-8 without mark-ups and images. For each document of the training set, the human-generated summary is provided. For MultiLing 2015 the character length of the human summary for each document will be provided, called the target length. Each machine summary should be as close to the target length provided as possible. For the purpose of evaluation all machine summaries greater than the target length will be truncated to the target length. The summaries will be evaluated via automatic methods and participants will be required to perform some limited summarization evaluations.

The manual evaluation will consist of pairwise comparisons of machine-generated summaries. Each evaluator will be presented the human-generated summary and two machine-generated summaries.  The evaluation task is to read the human summary and then judge if the one machine-generated summary is significantly closer to the human generated summary information content (e.g.  system A > system B or system B > system A) or if the two machine-generated summaries contain comparable quantities of information as the human-generated summary.

Information

For any additional information, please contact Jeff Kubina (jeff.kubina @ gmail.com)