Seminar "Selected Topics in Human Language Technology and Pattern Recognition"

Registration for the seminar

Registration for the seminar is only possible online via the central registration page.

Prerequisites for participation in the seminar

• Bachelor students: Einführung in das wissenschaftliche Arbeiten (Proseminar)
• Master students: Bachelor degree
• Attendance of the lectures Pattern Recognition and Neural Networks, Speech Recognition or Statistical Methods in Natural Language Processing, or evidence of equivalent knowledge is highly recommended.
• For successful participants of the above lectures, seminar participation is guaranteed.
  

Seminar format and important dates

• Proposals: initial proposals will be accepted up until the start of the term's lecture period (October 9, 2017) by email to the seminar topic's supervisor. At this time, participants must arrange an appointment with the relevant supervisor. Revised proposals will be accepted up until two weeks after the start of the term.
• Article: PDF must be submitted at least 1 month prior to the trial presentation date by email to the seminar topic's supervisor.
  
• Presentation slides: PDF must be submitted at least 1 week prior to the trial presentation date by email to the seminar topic's supervisor.
  supervisor.
• Trial presentations: at least 2 weeks prior to the actual presentation date; refer to the topics section.
• Seminar presentations: the exact dates and plan for the presentation block will be arranged and announced for the individual topics.
• Final (possibly corrected) articles and presentation slides: PDF must be submitted at the latest 4 weeks after the presentation date by email to the seminar topic's supervisor.
• Compulsory attendance: in order to pass, participants must attend all presentation sessions.
• Ethical Guidelines:The Computer Science Department of RWTH Aachen University has adopted ethical guidelines for the authoring of academic work, such as seminar reports. Each student has to comply with these guidelines. In this regard, you, as a seminar attendant, have to sign a declaration of compliance, in which you assert that your work complies with the guidelines, that all references used are properly cited, and that the report was done autonomously by yourself. We ask you do download the guidelines and submit the declaration together with your seminar report and talk to your supervisor. You also find a German version of the guidelines and a German version of the declaration you may use as well.

Note: failure to meet deadlines, absence without permission from compulsory sessions (presentations and preliminary meeting as announced by email to each participating student), or dropping out of the seminar after more than 3 weeks after the preliminary meeting/topic distribution results in the grade 5.0/not appeared.

Schedule

Topics, relevant references and participants

1. Deep Learning: Introduction
   
   
   1. Feedforward Deep Networks (ID: 01) (Freiny; Supervisor: Harald Hanselmann)
      Initial References:
      
      • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 6.
      Date of presentation: Monday: 8th January
   
   
   2. Regularization of Deep or Distributed Models (ID: 02) (Jansen; Supervisor: Jan Rosendahl)
      Initial References:
      
      • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 7.8-7.15.
      Date of presentation: Monday, 8th January
   
   
   3. Optimization for Model Training (ID: 03) (Pförtner; Supervisor: Julian Schamper)
      Initial References:
      
      • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 8.
      Date of presentation: Monday, 8th January
   
   
   4. Convolutional Networks (ID: 04) (Sharma; Supervisor: Harald Hanselmann)
      Initial References:
      
      • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 9.
      Date of presentation: Tuesday, 9th January
   
   
   5. Recurrent Neural Network and Long Term Dependencies (ID: 05) (Lauschke; Supervisor: Julian Schamper)
      Initial References:
      
      • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 10.1-10.6.
      Date of presentation: Tuesday, 9th January
   
   
   6. Practical Methodology (ID: 06) (Jonalik; Supervisor: Wilfried Michel)
      Initial References:
      
      • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 11.
      Date of presentation: Wednesday, 10th January
   
   
   7. Representation Learning (ID: 08) (Kleine-Tebbe; Supervisor: Eugen Beck)
      Initial References:
      
      • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 15.
      Date of presentation: Wednesday, 10th January
   
   
   8. Deep Generative Models Part 1 (ID: 09) (Scholkemper; Supervisor: Eugen Beck)
      Initial References:
      
      • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 20.1-20.4.
      Date of presentation: Wednesday, 10th January
   
   Deep Generative Models Part 2 (ID: 10) (NN; Supervisor: Tobias Menne)
   Initial References:
   
   • I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, Book in preparation for MIT Press, Feb. 2016, Sec. 20.10.
   Date of presentation: No (dropped out)
   
   
   
2. Deep Learning: Advanced Models
   
   
   9. Neural Turing Machines and Related (ID: 11) (Yang; Supervisor: Albert Zeyer)
      Initial References:
      
      • A. Graves, G. Wayne, I. Danihelka "Neural Turing Machines," arXiv:1410.5401, Oct. 2014
      • I. Danihelka, G. Wayne, B. Uria, N. Kalchbrenner, A. Graves, "Associative Long Short-Term Memory," arXiv:1602.03032, Feb. 2016.
      Date of presentation: Monday, 15th January
   
   
   10. Generative Adversarial Networks (ID: 12) (Faber; Supervisor: Albert Zeyer)
       Initial References:
       
       • Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio: "Generative Adversarial Nets" Advances in Neural Information Processing Systems 27 (NIPS 2014), Montréal, Canada, 2014.
       • Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Arjovsky, Aaron Courville: "Adversarially Learned Inference" arXiv, 2016.
       Date of presentation: Monday, 15th January
   
   
   11. Generative Auto-Regressive Models (ID: 13) (Kapoor; Supervisor: Mirko Hannemann)
       Initial References:
       
       • Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, Koray Kavukcuoglu: "WaveNet: A Generative Model for Raw Audio" arXiv, 2016.
       • Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aaron van den Oord, Alex Graves, Koray Kavukcuoglu: "Neural Machine Translation in Linear Time" arXiv, 2016.
       • Aaron van den Oord, Nal Kalchbrenner, Oriol Vinyals, Lasse Espeholt, Alex Graves, Koray Kavukcuoglu: "Conditional Image Generation with PixelCNN Decoders" arXiv, 2016.
       Date of presentation: Monday, 15th January
   
   
   
3. Deep Learning: Machine Translation
   
   
   12. Automatic Evaluation of Machine Translation (ID: 20) (Stanchev; Supervisor: Weiyue Wang)
       Initial References:
       
       • Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu: "BLEU: a Method for Automatic Ev aluation of Machine Translation" Annual Meeting of the Association for Computational Linguistics (ACL 2002), Philadelphia, PA, USA, 2002.
       • Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, John Makhoul: "A Study of Translation Edit Rate with Targeted Human Annotation" The Conference of the Association for Machine Translation in the Americas Annual Meeting of the Association for Computational Linguistics (AMTA 2006), Cambridge, MA, USA, 2006.
       • Weiyue Wang, Jan-Thorsten Peter, Hendrik Rosendahl, Hermann Ney: "CharacTER : Translation Edit Rate on Character Level" ACL 2016 First Conference on Machine Translation (WMT 2016), Berlin Germany, 2016.
       Date of presentation: Tuesday, 16th January
   
   
   13. Alignment Based Neural Machine Translation (ID: 19) (Lee; Supervisor: Weiyue Wang)
       Initial References:
       
       • Tamer Alkhouli, Gabriel Bretschner, Jan-Thorsten Peter, Mohammed Hethnawi, Andreas Guta, Hermann Ney: "Alignment-Based Neural Machine Translation" ACL 2016 First Conference on Machine Translation (WMT 2016), Berlin, Germany, 2016.
       • Weiyue Wang, Tamer Alkhouli, Derui Zhu, Hermann Ney: "Hybrid Neural Network Alignment and Lexicon Model in Direct HMMfor Statistical Machine Translation" Annual Meeting of the Association for Computational Linguistics (ACL 2017), Vancouver, Canada, 2017.
       Date of presentation: Tuesday, 16th January
   
   
   14. Attention-based Neural Machine Translation (ID: 14) (Drechsel; Supervisor: Jan Rosendahl)
       Initial References:
       
       • D. Bahdanau, K. Cho, Y. Bengio, "Neural Machine Translation by Jointly Learning to Align and Translate," Int. Conf. on Learning Representations (ICLR), San Diego, CA, USA, May 2015.
       • M. T. Luong, H. Pham, C. D. Manning, "Effective Approaches to Attention-based Neural Machine Translation," Conf. on Empirical Methods in Natural Language Processing (EMNLP), Lisbon, Portugal, Sep. 2015.
       Date of presentation: Monday, 29th January
   
   
   15. Character-based Translation (ID: 15) (Tran; Supervisor: Parnia Bahar)
       Initial References:
       
       • Wang Ling, Isabel Trancoso, Chris Dyer, Alan W Black: "Character-based Neural Machine Translation," arXiv:1511.04586, Nov. 2015.
       Date of presentation: Monday, 29th January
   
   
   16. Convolutional Neural Machine Translation (ID: 16) (Nickels; Supervisor: Parnia Bahar)
       Initial References:
       
       • Jonas Gehring, Michael Auli, David Grangier, Yann N. Dauphin: "A Convolutional Encoder Model for Neural Machine Translation" Association for Computational Linguistics (ACL 2017), Vancouver, Canada, 2017.
       • Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N. Dauphin: "Convolutional Sequence to Sequence Learning" Proceedings of the 34th International Conference on Machine Learning, (ICML), Sydney, Australia, 2017.
       Date of presentation: Monday, 29th January
   
   
   17. Semi-supervised Learning for Neural Machine Translation (ID: 18) (Lopatin; Supervisor: Yunsu Kim)
       Initial References:
       
       • R. Sennrich, B. Haddow, A. Birch: "Improving Neural Machine Translation Models with Monolingual Data" ACL 2016.
       • Y. Cheng, W. Xu, Z. He, W. He, H. Wu, M. Sun, Y. Liu: "Semi-supervised Learning for Neural Machine Translation" ACL 2016.
       • D. He, Y. Xia, T. Qin, L. Wang, N. Yu, T. Liu, W. Ma: "Dual Learning for Machine Translation" NIPS 2016.
       Date of presentation: Tuesday, 30th January
   
   
   
   
   
4. Statistical Machine Translation
   
   
   Domain Adaptation in Machine Translation (ID: 21) (NN; Supervisor: Andreas Guta)
   Initial References:
   
   • Philipp Koehn, Josh Schroeder: "Experiments in Domain Adaptation for Statistical Machine Translation" the Second Workshop on Statistical Machine Translation, Prague, Czech Republic, 2007.
   • Hal Daume III, Jagadeesh Jagarlamudi: "Domain Adaptation for Machine Translation by Mining Unseen Words" the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT2011), Portland, OR, USA, 2011.
   • Rui Wang, Andrew Finch, Masao Utiyama, Eiichiro Sumita: "Sentence Embedding for Neural Machine Translation Domain Adaptation" Annual Meeting of the Association for Computational Linguistics (ACL 2017), Vancouver, Canada, 2017.
   Date of presentation: No (dropped out)
   
   
   
5. Deep Learning: Automatic Speech and Handwriting Recognition
   
   
   18. Multi Target Learning (ID: 26) (Bieschke; Supervisor: Markus Kitza)
       Initial References:
       
       • Y. Shinohara: "Adversarial Multi-task Learning of Deep Neural Networks for Robust Speech Recognition", in Interspeech, 2016, pp. 2369-2372.
       • G. Pironkov, S. Dupont, and T. Dutoit: "Speaker-aware long short-term memory multi-task learning for speech recognition" in 2016 24th European Signal Processing Conference (EUSIPCO), 2016, pp. 1911-1915.
       Date of presentation: Tuesday, 30th January
   
   
   19. Segmental Recurrent Neural Networks (ID: 30) (Raissi; Supervisor: Mirko Hannemann)
       Initial References:
       
       • Lingpeng Kong, Chris Dyer, Noah A. Smith: "Segmental Recurrent Neural Networks" Int. Conf. on Learning Representations (ICLR), Puerto Rico, May 2, 2016.
       • Liang Lu, Lingpeng Kong, Chris Dyer, Noah A. Smith, Steve Renals: "Segmental Recurrent Neural Networks for End-to-end Speech Recognition" Interspeech, California, Sep 16, 2016.
       Date of presentation: Tuesday, 30th January
   
   
   
6. Deep Learning: Speech Signal Processing
   
   
   20. ANN Supported Source Separation (ID: 35) (Lenßen; Supervisor: Tobias Menne)
       Initial References:
       
       • John R. Hershey, Zhuo Chen, Jonathan Le Roux, Shinji Watanabee: "Deep Clustering: Discriminative Embeddings for Segmentation and Separation," IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, March 20-25, 2016.
       Date of presentation: Wednesday, 31st January
   
   
   
7. Deep Learning: Discriminative Training
   
   
   21. Sequence Discriminative Training (ID: 24) (Behrens; Supervisor: Wilfried Michel)
       Initial References:
       
       • Dong Yu, Li Deng: "Automatic Speech Recognition - A Deep Learning Approach" , New York, NY, USA: Springer, pp. 137-153, Oct. 2014.
       Date of presentation: Wednesday, 31st January
   
   
   
8. Deep Learning: Natural Language Understanding
   
   
   22. Sentence Embedding (ID: 22) (Gelmez; Supervisor: Yunsu Kim)
       Initial References:
       
       • Q. Le, T. Mikolov: "Distributed Representations of Sentences and Documents" ICML 2014.
       • R. Kiros, Y. Zhu, R. Salakhutdinov, R. Zemel, A. Torralba, R. Urtasun, S. Fidler: "Skip-thought Vectors" NIPS 2015.
       Date of presentation: Wednesday, 31st January
   
   
   
   

Guidelines for the article and presentation

• Online LaTeX-Documentation:

• LaTeX: "A document preparation system"
• LaTeX help 1.1
• The Not So Short Introduction to LaTeX
  
• LaTeX 2e for authors

• Document Templates:

• Article Template (51kB),  contains the template and all necessary files in tar format (or here 10kB in zip format).
• Presentation Slide Template (11.3MB), a zip file containing the template and all necessary graphics as well as the institute’s style template. Note: We deactivated the RWTH and i6 logos in this version of the template since the seminar content is produced by students outside of i6.

• Guidelines for articles and presentation slides:

• The aim of the seminar for the participants is to learn the following:
• to tackle a topic and to expand knowledge
• to critically analyze the literature
  
• to hold a presentation
  
• Take notice of references to other topics in the seminar and discuss topics with one another!
  
• Take care to stay within your own topic. To this end participants should be aware of the other topics in the seminar. If applicable, cross-reference other articles and presentations.
  

• Important: As part of the introduction, a slide should outline the most important literature used for the presentation. In addition, the presentation should clearly indicate which literature the particular elements of the presentation refer to.
  
• Take notice of references to other topics in the seminar and discuss topics with one another!
  
• Participants are expected to seek out additional literature on their topic. Assistance with the literature search is available at the faculty’s library. Access to literature is naturally also available at the Lehrstuhl Informatik 6 library.
• Notation/Mathematical Formulas: consistent, correct notation is essential. When necessary, differing notation from various literature sources is to be modified or standardized in order to be clear and consistent. The lectures held by the Lehrstuhl Informatik 6 should provide a guide as to what appropriate notation should look like.
  
• Tables must have titles (appearing above the table).
• Figures must have captions (appearing below the figure).
  
• The use of English is recommended and mandatory for the presentation slides. Nevertheless the article and oral presentation might be German.
  
• In the case that no adequate translation of an English technical term is available, the term should be used unchanged.
• Completeness: acknowledge all literature and sources.
  
• Referencing must conform to the standard described in the article template.
• Examples should be used to illustrate points.
• Examples should be as complex as necessary but as simple as possible.
• Slides should be used as presentation aids and not to replace the role of the presenter; specifically, slides should:
• illustrate important points and relationships;
• remind the audience (and the presenter) of important aspects and considerations;
  
• give the audience an overview of the presentation.
  
• Slides should not contain chunks of text or complicated sentences; rather they should consist of succinct words and terms.
• Use illustrations where appropriate - a picture says a thousand words!
• Abbreviations should be defined at the first usage in the manner demonstrated in the following example: "[...] at the Rheinisch-Westfälischen Technischen Hochschule (RWTH) there are [...]".
  
• Take care to stay within your own topic. To this end participants should be aware of the other topics in the seminar. If applicable, cross-reference other articles and presentations.
• Usage of fonts, typefaces and colors in presentation slides must be consistent and appropriate. Such means should serve to clarify points or relationships, not be applied needlessly or at random.
  
• Care should be taken when selecting fonts for presentation slides (also within diagrams) to ensure legibility on a projector even for those seated far from the screen.

Contact

Julian Schamper
RWTH Aachen University
Lehrstuhl Informatik 6
Ahornstr. 55
52074 Aachen

Room 6129
Tel: 0241 80 21615

E-Mail: schamper@cs.rwth-aachen.de

Last modified: 2018-01-29 16:18:14 Disclaimer.