Course: Machine Translation (Official page)
Description: The course introduces the foundations and the recent advancements in machine translation (MT), probably the most prolific application sector in computational linguistics. Machine translation deals with the automatic translation of speech or text between two languages. This technology is considered strategic for the integration of Europe as well as for the global market. Most internet companies (Google, Microsoft, Facebook, eBay, ...) own now MT research teams in order to support their geographic expansion. This course focuses on the statistical or machine learning approach which, after a long hegemony of phrase-based models, recently moved to neural network based models. Hence, the course will focus on neural MT, in particular modelling, training, and decoding. We will also look at use cases of MT in the translation industry, ranging from enterprise dedicated MT to the support of professional translators. The following is a tentative content table of the course: 1. Introduction to MT, problem, approaches, phrase-based versus neural MT, quality evaluation. 2. Fundamentals of neural networks, feed-forward networks, activation functions, loss functions, training criteria, stochastic gradient descent, learning rate policies, computation graph, back propagation algorithm. 3. Recurrent neural networks, time unfolded representation, back-propagation through time, vanishing and exploding gradient problems, long-short term memory units, gated recurrent units; 4. Neural MT, encoder-decoder architecture, attention model, beam search, model variations, large vocabulary methods, beam search, ensamble decoding. 5. MT Evaluation, manual and automatic MT evaluation, in depth comparison of neural vs phrase-based MT. 6. Training criteria for NMT, crosse-entropy , data as demonstrator, reinforcement learning, bandit learning, minimum risk training, curriculum learning, adversarial learning; 7. Convolutional NMT, convolutional networks, convolutional and, transformer models; 8. MT deployment, post-editing and online learning, interactive MT, topic and domain adaptation, automatic post-editing, multilingual NMT, the ModernMT project.
Academic Year: 2018-2019
Lecturer: Marcello Federico
Room: Molveno, Povo 1, University of Trento
Schedule:
June 12, 2019 10:00-12:30; [slides_1 ]
June 13, 2019 10:00-12:30; [slides_2]
June 14, 2019 10:00-12:30;[slides_3]
June 17, 2019 10:00-12:30;[slides_4]
June 18, 2019 10:00-12:30; [slides_5 Transformer Animation]
June 19, 2019 10:00-12:30;[slides_6]
June 20, 2019 10:00-12:30; [slides _7]
June 21, 2019 10:00-12:30; [slides_8]
Exam
What: 15' talk + 10' QA session
Mark will be depend 40% on the presentation, and 60% on the QA session.
The talk will either present a machine translation engine or a long research paper. In the first case, the engine can be developed with some of the available tools (see below) and tested on a public benchmark agreed among the participants (see below). The student and the lecturer should agree beforehand on the content of the presentation.
Venue: all presentations will be given within a few webinars open to all students.
Date: 5 July, 17:30-19:00 and 19 July, 17:30-19:00
Please register to the exam from this page.
Software
ModernMT, adaptive neural MT toolkit, written in Python, on github
Nematus, neural MT toolkit, written in Python, on github
TensorFlow, deep learning framework, written in Python
OpenNMT-py, neural MT toolkit, written in Python
Sockeye, neural MT toolkit, written in Python
Marian, neural MT, written in C++
Benchmark