Seminar "Selected Topics in Human Language Technology and Machine Learning"

Registration for the seminar

Registration for the seminar is only possible online via the central registration page.

Prerequisites for Participation in the Seminar

• Einführung in das wissenschaftliche Arbeiten (Proseminar)
• Attendance of the lectures Statistical Classification and Machine Learning, Automatic Speech Recognition, and/or Statistical Methods in Natural Language Processing, or evidence of equivalent knowledge is highly recommended.
• For successful participants of the above lectures, seminar participation is guaranteed.
  

General Goals of the Seminar

• Performance of a literature review based on the initial references provided to attain on overview of the assigned seminar topic
• Reading, comprehending and critically analyzing the assigned and found articles.
• Covering relevant publications within the scope of the assigned topic and the seminar format.
• Describing the topic on this basis in a written report.
• Preparing slides and presenting the topic in an oral presentation.
• NewsKeeping the ethical guidelines for the authoring of academic work, which especially includes that all work used to prepare the seminar report and presentation is correctly cited.

Seminar Format and Important Dates

• Proposals: initial proposals will be accepted up until (October 26, 2020) by email to the seminar topic's supervisor. At this time, participants must arrange an appointment with the relevant supervisor. Revised proposals will be accepted up until two weeks after the proposal date.
• Article: PDF must be submitted until January 15, 2021 by email to the seminar topic's supervisor.
  
• Presentation slides: PDF must be submitted at least 1 week prior to the trial presentation date by email to the seminar topic's supervisor.
  
• Trial presentations: at least 2 weeks prior to the actual presentation date; refer to the topics section.
• Seminar presentations: date will be announced during lecture period.
• Final (possibly corrected) articles and presentation slides: PDF must be submitted 4 weeks after the presentation date at the latest by email to the seminar topic's supervisor.
• Compulsory attendance: in order to pass, participants must attend all presentation sessions.
• Ethical Guidelines:The Computer Science Department of RWTH Aachen University has adopted ethical guidelines for the authoring of academic work, such as seminar reports. Each student has to comply with these guidelines. In this regard, you, as a seminar attendant, have to sign a declaration of compliance, in which you assert that your work complies with the guidelines, that all references used are properly cited, and that the report was done autonomously by yourself. We ask you do download the guidelines and submit the declaration together with your seminar report and talk to your supervisor. You also find a German version of the guidelines and a German version of the declaration you may use as well.

Note: failure to meet deadlines, absence without permission from compulsory sessions (presentations and preliminary meeting as announced by email to each participating student), or dropping out of the seminar after more than 3 weeks after the preliminary meeting/topic distribution results in the grade 5.0/not appeared.

The deadline for de-registration from the seminar is Monday, October 12, 2020, 23:59h, i.e. within three weeks after the distribution of the topics. After this deadline, seminar participation is confirmed and will be graded.

Topics, Initial References Defining the Topics, Participants, and Supervisors

• Automatic Speech Recognition;
• Machine Translation;
• Natural Language Understanding;
• Machine Learning.

1. Methods for Speaker Diarization (Tran; Supervisor: Wilfried Michel; Introduction Video)
   Initial References:
   
   • M.H. Moattar, M.M. Homayounpour, "A review on speaker diarization systems and approaches", in Speech Communication, Vol. 54, No. 10, 2012.
   • Xavier A. Miro, S. Bozonnet, N. Evans, C. Fredouille, G. Friedland, O. Vinyals , "Speaker Diarization: A Review of Recent Research", in IEEE Transactions on Audio, Speech, and Language Processing, Vol. 20 , No. 2 , Feb. 2012 )
   • A. Zhang, Q. Wang, Z. Zhu, J. Paisley, C. Wang, "Fully Supervised Speaker Diarization", in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, Apr. 2019.
   • T. L. Nwe, H. Sun, H. Li, S. Rahardja, "Speaker diarization in meeting audio," in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Taipei, Taiwan, Apr. 2009.
   • K. Church, W. Zhu, J. Vopicka, J. Pelecanos, D. Dimitriadis, P. Fousek, "Speaker diarization: A perspective on challenges and opportunities from theory to practice," in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, Mar. 2017.


2. Permutation-Invariant Training (Hoffbauer; Supervisor: Peter Vieting; Introduction Video)
   Initial References:
   
   • D. Yu, M. Kolbaek, Z.-H. Tan, J. Jensen: "Permutation Invariant Training of Deep Models for Speaker-Independent Multi-Talker Speech Separation," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, Mar. 2017, arXiv:1607.00325.
   • Y. Qian, X. Chang, D. Yu: "Single-Channel Multi-talker Speech Recognition with permutation Invariant Training", submitted to IEEE/ACM Transactions on Audio, Speech and Language Processing, arXiv:1707.06527, Jul. 2017.


3. Speaker-Dependent Speaker Separation (Liu; Supervisor: Peter Vieting; Introduction Video)
   Initial References:
   
   • Yan-Hui Tu, Jun Du, Chin-Hui Lee: "A Speaker-Dependent Approach to Single-Channel Joint Speech Separation and Acoustic Modeling Based on Deep Neural Networks for Robust Recognition of Multi-Talker Speech," Journal of Signal Processing Systems, pp. 1-11, 2018.
   • Tian Gao, Jun Du, Li-Rong Dai, Chin-Hui Lee: "A unified DNN approach to speaker-dependent simultaneous speech enhancement and speech separation in low SNR environments," Speech Communication, pp. 28-39, Vol. 95, 2017.


4. Encoder-decoder Attention based ASR (Alotbah; Supervisor: Wei Zhou; Introduction Video)
   Initial References:
   
   • Chorowski, Jan K and Bahdanau, Dzmitry and Serdyuk, Dmitriy and Cho, Kyunghyun and Bengio, Yoshua: "Attention-based models for speech recognition ," in PAdvances in neural information processing systems, pp. 577-585, 2015.
   • William Chan and Navdeep Jaitly and Quoc V. Le and Oriol Vinyals: "Listen, Attend and Spell: A Neural Network for Large Vocabulary Conversational Speech Recognition," in International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2016.
   • Zeyer, Albert and Irie, Kazuki and Schlüter, Ralf and Ney, Hermann: "Improved training of end-to-end attention models for speech recognition," in Proc. of INTERSPEECH , Hyderabad, India, Sep. 2018.
   • Tüske, Zoltan and Saon, George and Audhkhasi, Kartik and Kingsbury, Brian: "Single headed attention based sequence-to-sequence model for state-of-the-art results on {Switchboard-300}," in International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020.


5. Neural Transducer based ASR (Wu; Supervisor: Wei Zhou; Introduction Video)
   Initial References:
   
   • Alex Graves: " Sequence Transduction with Recurrent Neural Networks," in Proc. of the International Conference of Machine Learning (ICML) 2012 Workshop on Representation Learning, Edinburgh, Scotland, Jun. 2012.
   • Anshuman Tripathi and Han Lu and Hasim Sak and Hagen Soltau: "Monotonic Recurrent Neural Network Transducer and Decoding Strategies," in IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) ,Sentosa, Singapore, Dec. 2019.
   • Ehsan Variani and David Rybach and Cyril Allauzen and Michael Riley: "Hybrid Autoregressive Transducer (HAT)," in 45th International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , Virtual, May 2020.
   • Mohammadreza Ghodsi and Xiaofeng Liu and James Apfel and Rodrigo Cabrera and Eugene Weinstein: "Rnn-Transducer with Stateless Prediction Network," in 45th International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , Virtual, May 2020.


6. Speech Enhancement for ASR (Kusak; Supervisor: Nick Rossenbach; Introduction Video)
   Initial References:
   
   • F. Weninger, H. Erdogan, S. Watanabe, E. Vincent, J. Roux, J. R. Hershey, B. Schuller: "Speech Enhancement with LSTM Recurrent Neural Networks and Its Application to Noise-Robust ASR," in Proc. 12th International Conference on Latent Variable Analysis and Signal Separation (LVA/ICA), Vol. 9237, pp. 91-99, Liberec, Czech Republic, Aug. 2015.
   • T. Ochiai, S. Watanabe, S. Katagiri: "Does speech enhancement work with end-to-end ASR objectives?: Experimental analysis of multichannel end-to-end ASR," in Proc. IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), no. 26280063, pp. 1-6, Tokyo, Japan, Sep. 2017.


7. Pre-training Language Representation Models for NLP (Kemper; Supervisor: David Thulke; Introduction Video)
   Initial References:
   
   • J. Devlin, M.-W. Chang, K. Lee, K. Toutanova: "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in Proc. Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pp. 4171-4186, Minneapolis, MN, USA, USA, Jun. 2019.
   • Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. Salakhutdinov, Q. V. Le: "XLNet: Generalized Autoregressive Pretraining for Language Understanding," arXiv preprint arXiv:1906.08237.
   • . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov: "RoBERTa: A Robustly Optimized BERT Pretraining Approach," arXiv preprint arXiv:1907.11692.


8. Joint Intent Classification and Slot Filling (Burdorf; Supervisor: David Thulke; Introduction Video)
   Initial References:
   
   • C. Li, L. Li, J. Qi: "A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding", in Proc. Conf. on Empirical Methods in Natural Language Processing (EMNLP), pp. 3824-3833, Brussels, Belgium, Oct. 2018.
   • Z. Zhang, Z. Zhang, H. Chen, and Z. Zhang: "A Joint Learning Framework With BERT for Spoken Language Understanding," in IEEE Access 7, Nov. 2019.


9. Cross-lingual Word and Sentence Embedding (Feucht; Supervisor: Yingbo Gao; Introduction Video)
   Initial References:
   
   • T. Mikolov, Q. Le, I. Sutskever: "Exploiting Similarities among Languages for Machine Translation," Sep. 2013.
   • M. Artetxe, G. Labaka, E. Agirre: "Learning Bilingual Word Embeddings with (Almost) No Bilingual Data," in Proceedings of 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 451-462, Vancouver, Canada, Jul. 2017.
   • G. Glavas, R. Litschko, S. Ruder, I. Vulic: "How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions," in Proceedings of 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, Jul. 2019.
   • H. Schwenk, M. Douze: "Learning Joint Multilingual Sentence Representations with Neural Machine Translation" , in Proceedings of the ACL 2nd Workshop on Representation Learning for NLP (Repl4NLP) , pp. 157-167, Vancouver, Canada, Aug. 2017.
   • M. Artetxe, H. Schwenk: "Massively Multilingual Sentence Embeddings for Zero-Shot Cross-lingual Transfer and Beyond", in Proceedings of 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, Jul. 2019.
   • G. Lample, A. Conneau: "Cross-lingual Language Model Pre-training," arXiv:1901.07291, Jan. 2019.


10. Multilingual Automatic Speech Recognition (Scherer; Supervisor: Christoph Lüscher; Introduction Video)
    Initial References:
    
    • Toshniwal, Shubham, Tara N. Sainath, Ron J. Weiss, Bo Li, Pedro Moreno, Eugene Weinstein, and Kanishka Rao: "Multilingual speech recognition with a single end-to-end model,"In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4904-4908. IEEE, 2018.
    • Tong, Sibo, Philip N. Garner, and Herve Bourlard: "An investigation of deep neural networks for multilingual speech recognition training and adaptation," in Proc. of INTERSPEECH, no. CONF. 2017.
    • Sercu, Tom, Christian Puhrsch, Brian Kingsbury, and Yann LeCun: "Very deep multilingual convolutional neural networks for LVCSR," in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4955-4959. IEEE, 2016.


11. Domain Adaptation/Expansion (NN; Supervisor: Christoph Lüscher; Introduction Video)
    Initial References:
    
    • Meng, Zhong, Zhuo Chen, Vadim Mazalov, Jinyu Li, and Yifan Gong: "Unsupervised adaptation with domain separation networks for robust speech recognition, in 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 214-221. IEEE, 2017 .
    • Ghorbani, Shahram, Soheil Khorram, and John HL Hansen: "Domain Expansion in DNN-based Acoustic Models for Robust Speech Recognition," in arXiv preprint arXiv:1910.00565 (2019)


12. Vocoders (NN; Supervisor: Yingbo Gao; Introduction Video)
    Initial References:
    
    • WaveNet: Oord A, Dieleman S, Zen H, et al: "Wavenet: A generative model for raw audio," arXiv preprint arXiv:1609.03499, 2016.
    • WaveRNN: Kalchbrenner, Nal, et al:"Efficient neural audio synthesis, arXiv preprint arXiv:1802.08435 (2018).
    • WaveGAN, Donahue, Chris, Julian McAuley, and Miller Puckette: "Adversarial audio synthesis, arXiv preprint arXiv:1802.04208 (2018).


13. Neural Text-to-speech (Schümann; Supervisor: Nick Rossenbach; Introduction Video)
    Initial References:
    
    • Y. Taigman, L. Wolf, A. Polyak, E. Nachmani: "VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop," arXiv:1707.06588, Jul. 2017.
    • J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. J. Skerry-Ryan, R. A. Saurous, Y. Agiomyrgiannakis, Y. Wu: "Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions, arXiv:1712.05884, Dec. 2017.
    
    
14. End-to-end Speech-to-text Translation (Schmitt; Supervisor: Christian Herold; Introduction Video)
    Initial References:
    
    • A. Berard, O. Pietquin, C. Servan, L. Besacier: "Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation", in 30th Conference on Neural Information Processing Systems (NIPS) , Barcelona, Spain, Dec. 2016. arXiv:1612.01744.
    • R. J. Weiss, J. Chorowski, N. Jaitly, Y. Wu, Z. Chen: "Sequence-to-Sequence Models Can Directly Translate Foreign Speech," in Proc. 18th Annual Conference of the International Speech Communication Association (Interspeech) , Stockholm, Sweden, Aug. 2017, arXiv:1703.08581.


15. Minimum Expected Loss Training (NN; Supervisor: Mohammad Zeineldeen; Introduction Video)
    Initial References:
    
    • R. Prabhavalkar et al, "Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models", ICASSP 2018.
    • M. Shannon, "Optimizing expected word error rate via sampling for speech recognition", arXiv 2017.
    • D. Bahdanau et al, An Actor-Critic Algorithm for Sequence Prediction, ICLR 2017.
    • S. Sabour et al, Optimal Completion Distillation for Sequence Learning, ICLR 2019.


16. Modern Policy Learning Methods for Games (Schüller; Supervisor: Mohammad Zeineldeen; Introduction Video)
    Initial References:
    
    • D. Silver et al, Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm, arXiv 2017.
    • M. Jaderberg et al, Human-level performance in first-person multiplayer games with population-based deep reinforcement learning, Science 2019.


17. Extractive and Abstractive Text Summarization (Mansouri; Supervisor: Jan Rosendahl; Introduction Video)
    Initial References:
    
    • Mehdi M. Allahyari, S. Pouriyeh, M. Assefi, S. Safaei, E. D. Trippe, J. B. Gutierrez, K. Kochut: "Text Summarization Techniques: A Brief Survey," arXiv:1707.02268, Jul. 2017.
    • J.-M. Torres-Moreno: Automatic Text Summarization, John Wiley & Sons, Inc. , Hoboken, NJ, Sep. 2014, 348 pages. (available within RWTH Aachen Network).
    • R. Nallapati, B. Zhou, C. dos Santos, C. Gulcehre, B. Xiang: "Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond," in Proc. 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pp. 280-290, Berlin, Germany, Aug. 2016.
    • A. See, P. J. Liu, C. D. Manning: "Get To The Point: Summarization with Pointer-Generator Networks," in Proc. 55th Annual Meeting of the Association for Computational Linguistics, pp. 1073-1083, Vancouver, Canada, Jul./Aug. 2017.


18. Named Entity Recognition (Rasaratnam; Supervisor: Weiyue Wang; Introduction Video)
    Initial References:
    
    • G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, C. Dyer: "Neural Architectures for Named Entity Recognition," in Proc. of NAACL-HLT 2016, San Diego, California, June, 2016.
    • J. Strakova, M. Straka, J. Hajic: "Neural Architectures for Nested NER through Linearization," in Proc. of ACL 2019, Florence, Italy, July, 2019.
    • A. Akbik, D. Blythe, R. Vollgraf: "Contextual String Embeddings for Sequence Labeling," in Proc. of COLING 2018, Santa Fe, New Mexico, August, 2018

Article and Presentation Format

• Online LaTeX-Documentation:

• LaTeX: "A document preparation system"
• LaTeX help 1.1
• The Not So Short Introduction to LaTeX
  
• LaTeX 2e for authors

• Document Templates:

• Article Template (51kB),  contains the template and all necessary files in tar format (or here 10kB in zip format).
• New Presentation Slide Template, a zip file containing the template and all necessary graphics as well as the institute’s style template. Note: We deactivated the RWTH and i6 logos in this version of the template since the seminar content is produced by students outside of i6.

Detailed Guidelines:

• Take care to stay within your own topic. To this end participants should be aware of the other topics in the seminar. If applicable, cross-reference other articles and presentations.
• Important: As part of the introduction, a slide should outline the most important literature used for the presentation. In addition, the presentation should clearly indicate which literature the particular elements of the presentation refer to.
  
• Take notice of references to other topics in the seminar and discuss topics with one another!
  
• Participants are expected to seek out additional literature on their topic. Assistance with the literature search is available at the faculty’s library. Access to literature is naturally also available at the Lehrstuhl Informatik 6 library.
• Notation/Mathematical Formulas: consistent, correct notation is essential. When necessary, differing notation from various literature sources is to be modified or standardized in order to be clear and consistent. The lectures held by the Lehrstuhl Informatik 6 should provide a guide as to what appropriate notation should look like.
  
• Tables must have titles (appearing above the table).
• Figures must have captions (appearing below the figure).
  
• The use of English is recommended and mandatory for the presentation slides. Nevertheless, the article and oral presentation may be done in German.
  
• In the case that no adequate translation of an English technical term is available, the term should be used unchanged.
• Completeness: acknowledge all literature and sources, thus following the ethical guidelines for the authoring of academic work.
  
• Referencing must conform to the standard described in the article template.
• Examples should be used to illustrate points.
• Examples should be as complex as necessary but as simple as possible.
• Slides should be used as presentation aids and not to replace the role of the presenter; specifically, slides should:
• illustrate important points and relationships;
• remind the audience (and the presenter) of important aspects and considerations;
  
• give the audience an overview of the presentation.
  
• Slides should not contain chunks of text or complicated sentences; rather they should consist of succinct statements and use consistent terminology.
• Use illustrations where appropriate - a picture says a thousand words!
• Abbreviations should be defined at the first usage in the manner demonstrated in the following example: "[...] at the Rheinisch-Westfälischen Technischen Hochschule (RWTH) there are [...]".
  
• Usage of fonts, typefaces and colors in presentation slides must be consistent and appropriate. Such means should serve to clarify points or relationships, not be applied needlessly or at random.
  
• Care should be taken when selecting font sizes for presentation slides (also within diagrams) to ensure legibility on a projector even for those seated far from the screen.

Some Tips:

• The draft of the article and the trial presentation slides to be sent before the corresponding deadlines must be complete (in particular, they must contain a required number of pages). In principle, supervisors will not give any iterative feedback if an updated version is submitted after the deadlines. Do not miss the opportunities!

• Define the task clearly upfront (what is the problem? what are the input/output of the system?).
• Give a short overview of the existing works on the topic.
• Provide detailed descriptions of the state-of-the-art approach(es) with mathematical definitions using correct notations.
• Include meaningful experimental results (extracted from the papers) with clear definitions of the datasets and the evaluation metrics.

• Why is this paper relevant for my topic: a historical piece of work? the state-of-the-art method? Does the paper help me understanding the topic better?
• Do I really understand the paper? Can I describe how the method works without doubt? Can I explain the nature/dimension of all quantities in equations? Is this paper self-contained? or should I further read cited papers to get more background?
• Is the paper content correct and consistent with other publications on the topic? If not: try to resolve discrepancies and broach such issues when discussing the paper in your article and presentation.
• Are the experiments meaningful and convincing? Can I clearly describe the experiment and the evaluation metric?
• How does this paper relate to other papers I have read? Is the same dataset used for evaluation?

Contact

Yingbo Gao
RWTH Aachen University
Lehrstuhl Informatik 6
Ahornstr. 55
52074 Aachen

Room 6125
Tel: 0241 80 21611

E-Mail: gao@cs.rwth-aachen.de

Last modified: 2020-10-29 13:09:24 Disclaimer.