Seminar "Selected Topics in Human Language Technology and Pattern Recognition"

In the Winter Term 2016 / 2017 the Lehrstuhl Informatik 6 will host a seminar entitled "Selected Topics in Human Language Technology and Pattern Recognition".

Registration for the seminar


Registration for the seminar is only possible online via the central registration page from Friday, Jun. 24 to Sunday, Jul. 10, 2016. A link can also be found on the Computer Science Department's homepage.

Prerequisites for participation in the seminar

Seminar format and important dates

Please note the following deadlines:

Note: failure to meet deadlines, absence without permission from compulsory sessions (presentations and preliminary meeting as announced by email to each participating student), or dropping out of the seminar after more than 3 weeks after the preliminary meeting/topic distribution results in the grade 5.0/not appeared.

Topics, relevant references and participants

The general topic for this semester's seminar will be "Deep Learning for Human Language Technology and Pattern Recognition." The follwoing topics will be introduced at the preparatory meeting in the seminar room at the Lehrstuhl Informatik 6. The date of the meeting has been annouced individually to the seminar's participants as decided in the central registration (see above).


  1. Introduction to Deep Learning

    1. Feedforward Deep Networks (Liang; Supervisor: Kazuki Irie)
      Initial References:
      Estimated date for presentation: 18.01
    1. Regularization of Deep or Distributed Models (Diehl; Supervisor: Kazuki Irie)
      Initial References:
      Estimated date for presentation: 18.01
    1. Optimization for Model Training (Ungerechts; Supervisor: Pavel Golik)
      Initial References:
      Estimated date for presentation: 19.01
    1. Convolutional Networks (Lorang; Supervisor: Harald Hanselmann)
      Initial References:
      Estimated date for presentation: 19.01
    1. Recurrent Neural Network and Long Term Dependencies (Ouannani; Supervisor: Parnia Bahar)
      Initial References:
      Estimated date for presentation: 19.01
    1. Practical Methodology (Bähr; Supervisor: Markus Kitza)
      Initial References:
      Estimated date for presentation: 20.01
    1. Autoencoders (Bahloul; Supervisor: Markus Kitza)
      Initial References:
      Estimated date for presentation: 20.01
    1. Representation Learning (Brix; Supervisor: Albert Zeyer)
      Initial References:
      Estimated date for presentation: 20.01
    1. Deep Generative Models Part 1 (Bonot; Supervisor: Tobias Menne)
      Initial References:
      Estimated date for presentation: 01.02
    1. Deep Generative Models Part 2 (Merboldt; Supervisor: Tobias Menne)
      Initial References:
      Estimated date for presentation: 01.02
    1. Neural Turing Machines and Related (Nix; Supervisor: Albert Zeyer)
      Initial References:
      Estimated date for presentation: 01.02

  2. Deep Learning for Image Recognition

    1. Similarity Learning using Convolutional Neural Networks (Konar; Supervisor: Harald Hanselmann)
      Initial References:
      Estimated date for presentation: 02.02
    1. Spatial Transformer Networks (NN; Supervisor: Harald Hanselmann)
      Initial References:
      • M. Jaderberg, K. Simonyan, A. Zisserman and K. Kavukcuoglu: "Spatial transformer networks," Proc. Advances in Neural Information Processing Systems (NIPS), Montreal, Canada, pp. 2008-2016, Dec. 2015.

  3. Deep Learning for Language Modeling

    1. Neural Network-based Language Models (Khuurkhunkhuu; Supervisor: Parnia Bahar)
      Initial References:
      Estimated date for presentation: 02.02
    1. Convolutional Neural Networks for Language Processing (NN; Supervisor: Kazuki Irie)
      Initial References:
    1. Character-based Language Processing with Recurrent Neural Networks (NN; Supervisor: Kazuki Irie)
      Initial References:

  4. Deep Learning for Automatic Speech and Handwriting Recognition

    1. End-to-end Recurrent Neural Network Systems (Schnathmeier; Supervisor: Patrick Doetsch)
      Initial References:
      Estimated date for presentation: 02.02
    1. Decoding in Recurrent Neural Networks (Titgemeyer; Supervisor: Patrick Doetsch)
      Initial References:
      • Alex Graves and Santiago Fernández and Faustino Gomez. "Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks," International Conference on Machine Learning (ICML), pp. 369-376, New York City, NY, Jun. 2016.
      Estimated date for presentation: 03.02
    1. DNN Adaptation (Hein; Supervisor: Pavel Golik)
      Initial References:
      • G. Saon, H. Soltau, D. Nahamoo, M. Picheny, "Speaker adaptation of neural network acoustic models using i-vectors," IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 55-59, Olomouc, Czech Republic, Dec. 2013.
      Estimated date for presentation: 03.02

  5. Deep Learning for Speech Signal Processing

    1. Bottleneck Features (NN; Supervisor: Markus Kitza)
      Initial References:
    1. Neural Network Based Speech Enhancement (NN; Supervisor: Tobias Menne)
      Initial References:
      • Y. Xu, J. Du, Z. Huang, L.-R. Dai, C.-H. Lee: Multi-objective Learning and Mask-based Post-processing for Deep Neural Network based Speech Enhancement. Proc. Interspeech 2015, Dresden, Germany, pp. 1508-1512, Sep. 2015.
      • F. Weninger, H. Erdogan, S. Watanabe, E. Vincent, J. Le Roux, J. R. Hershey B. Schuller: "Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR" Proc. 12th Int. Conf. on Latent Variable Analysis and Signal Separation (LVA/ICA), Liberec, Czech Republic, pp. 91-99, Aug. 2015.
    1. Using Deep Learning to Support Conventional Signal Processing (NN; Supervisor: Tobias Menne)
      Initial References:
      • J. Heymann, L. Drude, A. Chinaev, R. Haeb-Umbach: BLSTM Supported GEV Beamformer Front-End for the 3rd Chime Challenge. Proc. Automatic Speech Recognition and Understanding Workshop (ASRU), Scottsdale, AZ, pp. 444-451, Dec. 2015.

  6. Deep Learning for Machine Translation

    1. Attention-based Neural Machine Translation (NN; Supervisor: Parnia Bahar)
      Initial References:
    1. Multi-Task Learning (Brocker; Supervisor: Jan-Thorsten Peter)
      Initial References:
      • Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, Haifeng Wang: Multi-Task Learning for Multiple Language Translation. Proc. Association for Computational Linguistics (ACL), Beijing, China, pp. 1723-1732, Jul. 2015.
      Estimated date for presentation: 03.02
    1. Character-based Translation (NN; Supervisor: Jan-Thorsten Peter)
      Initial References:
      Estimated date for presentation: 03.02

Guidelines for the article and presentation

The roughly 20-page article together with the slides (between 20 & 30) for the presentation should be prepared in LaTeX format. Presentations will consist of 30 to 40 minutes presentation time & 15 minutes discussion time. Document templates for both the article and the presentation slides are provided below along with links to LaTeX documentation available online. The article and the slides should be prepared in LaTeX format and submitted electronically in pdf format. Other formats will not be accepted.

             General:
             Specific:

Contact

Inquiries should be directed to the respective supervisors or to:

Julian Schamper
RWTH Aachen University
Lehrstuhl Informatik 6
Ahornstr. 55
52074 Aachen

Room 6129
Tel: 0241 80 21615

E-Mail: schamper@cs.rwth-aachen.de