ICT Doctoral School

Course: Machine Translation (Official page)

Description: The course introduces the foundations and the recent advancements in machine translation (MT), probably the most prolific application sector in computational linguistics. Machine translation deals with the automatic translation of speech or text between two languages. This technology is considered strategic for the integration of Europe as well as for the global market. Most internet companies (Google, Microsoft, Facebook, eBay, ...) own now MT research teams in order to support their geographic expansion. This course focuses on the statistical or machine learning approach which, after a long hegemony of phrase-based models, recently moved to neural network based models. Hence, the course will focus on neural MT, in particular modelling, training, and decoding. We will also look at use cases of MT in the translation industry, ranging from enterprise dedicated MT to the support of professional translators. The following is a tentative content table of the course: 1. Introduction to MT, problem, approaches, phrase-based versus neural MT, quality evaluation. 2. Fundamentals of neural networks, feed-forward networks, activation functions, loss functions, training criteria, stochastic gradient descent, learning rate policies, computation graph, back propagation algorithm. 3. Recurrent neural networks, time unfolded representation, back-propagation through time, vanishing and exploding gradient problems, long-short term memory units, gated recurrent units; 4. Neural MT, encoder-decoder architecture, attention model, beam search, model variations, large vocabulary methods, beam search, ensamble decoding. 5. MT Evaluation, manual and automatic MT evaluation, in depth comparison of neural vs phrase-based MT. 6. Training criteria for NMT, crosse-entropy , data as demonstrator, reinforcement learning, bandit learning, minimum risk training, curriculum learning, adversarial learning; 7. Convolutional NMT, convolutional networks, convolutional and, transformer models; 8. MT deployment, post-editing and online learning, interactive MT, topic and domain adaptation, automatic post-editing, multilingual NMT, the ModernMT project.

Academic Year: 2018-2019

Lecturer: Marcello Federico

Room: Molveno, Povo 1, University of Trento

Schedule:

Exam

What: 15' talk + 10' QA session

Mark will be depend 40% on the presentation, and 60% on the QA session.

The talk will either present a machine translation engine or a long research paper. In the first case, the engine can be developed with some of the available tools (see below) and tested on a public benchmark agreed among the participants (see below). The student and the lecturer should agree beforehand on the content of the presentation.

Venue: all presentations will be given within a few webinars open to all students.

Date: 5 July, 17:30-19:00 and 19 July, 17:30-19:00

Please register to the exam from this page.

Software

  • ModernMT, adaptive neural MT toolkit, written in Python, on github

  • Nematus, neural MT toolkit, written in Python, on github

  • TensorFlow, deep learning framework, written in Python

  • OpenNMT-py, neural MT toolkit, written in Python

  • Sockeye, neural MT toolkit, written in Python

  • Marian, neural MT, written in C++

Benchmark

  • The WIT3 website hosts the official benchmarks used for the IWSLT evaluation campaigns.