Seminar "Selected Topics in Human Language Technology and Pattern Recognition"

Registration for the seminar

Prerequisites for participation in the seminar

• Bachelor students: Einführung in das wissenschaftliche Arbeiten (Proseminar)
• Master students: Bachelor degree
• Diploma students: Vordiplom
  
• Attendance of at least one of the lectures Pattern Recognition and Neural Networks, Introduction to Statistical Classification, Automatic Speech Recognition, or Statistical Methods in Natural Language Processing, or evidence of equivalent knowledge.
• For successful participants of the above lectures, the possibility of a seminar talk is guaranteed.
  

Seminar format and important dates

• Mo, Feb. 10, 2014, 09:30-13:00h and 14:00-17:30h (5 presentations)
• Mo, Feb. 17, 2014, 09:00-12:30h (3 presentations)
• Tu, Feb. 18, 2014, 09:00-12:30h (3 presentations)
• Thu, Feb. 20, 2014, 09:00-12:30h (3 presentations)

• Proposals: initial proposals (report's content page) have to be submitted by email to your supervisors by Nov. 3, 2013. At this time participants must arrange an appointment with the individual supervisor.
• Article: must be submitted at least 1 month prior to the trial presentation date, but not later than Dec. 20, 2013 to the individual supervisor in electronic form (PDF).
  
• Presentation slides: must be submitted at least 1 week prior to the trial presentation date to the individual supervisor in electronic form (PDF).
• Trial presentations: at least 2 weeks prior to the actual presentation date. Please refer to your individual supervisor to schedule your trial presentation.
• Seminar presentations:As discussed during the kick-off meeting, the seminar will take place weekly during the semester, and will start in the first week of November. We will prepare a poll, once L2P is available, to find a day and time, where the presentations can be done during the semester.
• Final (corrected) articles and presentation slides: must be submitted within 4 weeks after the presentation date at the latest to the individual supervisor in electronic form (PDF).
• Compulsory attendance: in order to receive a certificate participants must attend all presentation sessions.
• Ethical Guidelines:The Computer Science Department of RWTH Aachen University has adopted ethical guidelines for the authoring of academic work such as seminar reports. Each student has to comply with these guidelines. In this regard, you, as a seminar attendant, have to sign a declaration of compliance, in which you assert that your work complies with the guidelines, that all references used are properly cited, and that the report was done autonomously by yourself. We ask you do download the guidelines and submit the declaration together with your seminar report and talk to your individual supervisor. You also find a German version of the guidelines and a German version of the declaration you may use as well.

Note: failure to meet deadlines, absence without permission from compulsory sessions (presentations and preliminary meeting as announced by email to each participating student), or dropping out of the seminar after more than 3 weeks after the kick-off meeting results in the grade 5.0/not appeared.

Topics, relevant references and participants

    Topics for Bachelor Students only:

1. Linear Models for Classification (NN; Betreuer: Patrick Dötsch)
   
• Christopher M. Bishop: Pattern recognition and machine learning, 738 pages, Springer, New York, NY, October 2007, Sec. 4.

2. Introduction to Neural Networks (Stutz; Betreuer: Pavel Golik)
   Vortrag: 10.02.2014
   
• Christopher M. Bishop: Pattern recognition and machine learning, 738 pages, Springer, New York, NY, October 2007, Sec. 5.

3. Sparse Kernel Machines (incl. SVM) (Pohlen; Betreuer: Simon Wiesler)
   Vortrag: 10.02.2014
   
• Christopher M. Bishop: Pattern recognition and machine learning, 738 pages, Springer, New York, NY, October 2007, Sec. 7.

4. Introduction to Log-Linear Modeling (NN; Betreuer: Simon Wiesler)
   
• Christopher M. Bishop: Pattern recognition and machine learning, 738 pages, Springer, New York, NY, October 2007, Sec. 8.

5. Mixture Models and EM (NN; Betreuer: Pavel Golik)
   
• Christopher M. Bishop: Pattern recognition and machine learning, 738 pages, Springer, New York, NY, October 2007, Sec. 9.

6. Hidden Markov Models (Duda, Hart & Storck, pp. 128-138) (Durmaz; Betreuer: Pavel Golik)
   Vortrag: 10.02.2014
   
• Richard O. Duda, Peter E. Hart & David G. Storck: Pattern Classification, 2nd edition, 654 pages, Wiley & Sons, New York, 2001, pp. 128-138.

7. Introduction to Discriminative Training in ASR (Göringer; Betreuer: Muhammad Tahir)
   Vortrag: 10.02.2014
   
• Xiaodong He, Li Deng: Discriminative Learning for Speech Recognition: Theory and Practice, 112 pages, Morgan & Claypool, October 2008, Sec. 1 & 2.

8. Unified View of Discriminative Training (Hamm; Betreuer: Muhammad Tahir)
   Vortrag: 10.02.2014
   
• Xiaodong He, Li Deng: Discriminative Learning for Speech Recognition: Theory and Practice, 112 pages, Morgan & Claypool, October 2008, Sec. 3.

9. Discriminative Learning for ASR (Gündüz; Betreuer: Muhammad Tahir)
   Vortrag: 17.02.2014
   
• Xiaodong He, Li Deng: Discriminative Learning for Speech Recognition: Theory and Practice, 112 pages, Morgan & Claypool, October 2008, Sec. 4 & 5.

10. Implementation of Discriminative Learning for ASR (Kryvosheya; Betreuer: Muhammad Tahir)
    Vortrag: 17.02.2014
    
• Xiaodong He, Li Deng: Discriminative Learning for Speech Recognition: Theory and Practice, 112 pages, Morgan & Claypool, October 2008, Sec. 6 & 7.

    Topics for Master and Bachelor Students, priority for Master students and students with prior knowledge of the field:

11. Multilingual Modeling for ASR (incl. NN) (Schwittlinsky; Betreuer: Zoltan Tüske)
    Vortrag: 17.02.2014
    
• G. Heigold, V. Vanhoucke, A. Senior, P. Nguyen, M. Ranzato, M. Devin, J. Dean, Multilingual Acoustic Models using Distributed Deep Neural Networks, ICASSP 2013
• Arnab Ghoshal, Pawel Swietojanski, Steve Renals, Multilingual Training of Deep Neural Networks, ICASSP 2013
• Stefano Scanzio, Pietro Laface, Luciano Fissore, Roberto Gemello, Franco Mana, On the Use of a Multilingual Neural Network Front-End Interspeech 2008
• Tanja Schultz, Alex Waibel, Language-independent and language-adaptive acoustic modeling for speech recognition, Speech Communication, 2001

12. Deep NNs for ASR (Hybrid Approach) (John; Betreuer: Zoltan Tüske)
    Vortrag: 18.02.2014
    
• George E. Dahl, Dong Yu, Li Deng, Alex Acero, Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition, IEEE Transactions on Audio, Speech, and Language Processing, Vol. 20, No. 1, January 2012
Tara N. Sainath, Brian Kingsbury, Bhuvana Ramabhadran, Petr Fousek, Petr Novak, Abdel-rahman Mohamed, Making Deep Belief Networks Effective for Large Vocabulary Continuous Speech Recognition, ASRU 2011
H. Bourlard and N. Morgan - Connectionist Speech Recognition: A Hybrid Approach, 1993

13. Deep NNs for ASR (NN features, Tandem Approach) (Sathyanarayana; Betreuer: Zoltan Tüske)
    Vortrag: 18.02.2014
    
• T.N. Sainath, B. Kingsbury, and B. Ramabhadran, Auto-Encoder Bottleneck Features using Deep Belief Networks, ICASSP, 2012
• F. Grezl, M. Karafiat, S. Kontar, and J. Cernocky, Probabilistic and Bottleneck Features for LVCSR of Meetings,¡ ICASSP, 2007.
• Hynek Hermansky, D.P.W. Ellis, Sangita Sharma, Tandem connectionist feature extraction for conventional HMM systems Icassp 2000

14. RNNs for ASR & Handwriting (Kamel; Betreuer: Patrick Dötsch)
    Vortrag: 18.02.2014
    
• J. L. Elman. Finding structure in time. Cognitive Science, Vol. 14, No. 2, pp. 179–211, 1990.
• A. Robinson. An application of recurrent nets to phone probability estimation. IEEE Transactions on Neural Networks, Vol. 5, No. 2, pp. 298–305, 1994.
• A. Graves, J. Schmidhuber. Offline handwriting recognition with multidimensional recurrent neural networks. In Neural Information Processing Systems, pp. 545–552, 2008.

15. Vanishing Gradient Problem (NN; Betreuer: Patrick Dötsch)
    
• Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is diffcult, IEEE Transactions on Neural Networks, 5(2):157-166, March 1994.
• Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, and Jürgen Schmidhuber. Gradient flow in recurrent nets: the diffculty of learning long-term dependencies. In: A Feld guide to dynamical recurrent neural networks, pages 237-243. Wiley-IEEE Press, January 2001.
• Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computations, 9(8):1735-1780, November 1997.

16. Log-linear Acoustic Modeling (Murmann; Betreuer: Simon Wiesler )
    Vortrag: 20.02.2014
    
• J. Lafferty, et al. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. ICML 2001
• Gunawardana, Asela, et al. Hidden conditional random fields for phone classification. In: Interspeech 2005.

17. Log-linear Language Modeling (Krebber; Betreuer: Martin Sundermeyer, proxy until end of August: Amr Ibrahim El-Desoky Mousa )
    Vortrag: 20.02.2014
    
• Klakow, D.: "Log-Linear Interpolation of Language Models", ICSLP 1998
• Chen, S.F., Rosenfeld, R.: "A Survey of Smoothing Techniques for ME Models", IEEE TSAP Vol. 8 No. 1, 2000

18. Model M, Shrinkage based Models (NN; Betreuer: Martin Sundermeyer, proxy until end of August: Amr Ibrahim El-Desoky Mousa )
    
• S. Chen, "Shrinking Exponential Language Models" (2009)
• S. Chen, "Performance Prediction For Exponential Language Models" (2009)
• R. Sarikaya, "Impact of Word Classing on Shrinkage-Based Language Models" (2010)

19. Neural Network Language Modeling (Hongmei; Betreuer: Martin Sundermeyer, proxy until end of August: Amr Ibrahim El-Desoky Mousa )
    Vortrag: 20.02.2014
    
• Y. Bengio, R. Ducharme, P. Vincent, C. Jauvain "A Neural Probabilistic Language Model" (2003), in Journal of Machine Learning Research 3 (2005), 1137-1155.
• H. Schwenk, "Continuous Space Language Models" (2007) in Computer Speech and Language (2007), 492-518
• T. Mikolov, "Recurrent Neural Network Based Language Model" in Proceedings of Interspeech 2010, Makuhari, 1045-1048

Guidelines for the article and presentation

• Online LaTeX-Documentation:

• LaTeX: "A document preparation system"
• LaTeX help 1.1
• The Not So Short Introduction to LaTeX
  
• LaTeX 2e for authors

• Document Templates:

• Article Template (164kB),  contains the template and all necessary files in tar format (or here 53kB in zip format).
• Presentation Slide Template (4.9MB), a zip file containing the template and all necessary graphics as well as the institute’s style template.

• Guidelines for articles and presentation slides:

• The aim of the seminar for the participants is to learn the following:
• to tackle a topic and to expand knowledge
• to critically analyze the literature
  
• to hold a presentation
  
• Take notice of references to other topics in the seminar and discuss topics with one another!
  
• Take care to stay within your own topic. To this end participants should be aware of the other topics in the seminar. If applicable, cross-reference other articles and presentations.
  

• Important: As part of the introduction, a slide should outline the most important literature used for the presentation. In addition, the presentation should clearly indicate which literature the particular elements of the presentation refer to.
  
• Take notice of references to other topics in the seminar and discuss topics with one another!
  
• Participants are expected to seek out additional literature on their topic. Assistance with the literature search is available at the faculty’s library. Access to literature is naturally also available at the Lehrstuhl für Informatik 6 library.
• Notation/Mathematical Formulas: consistent, correct notation is essential. When necessary, differing notation from various literature sources is to be modified or standardized in order to be clear and consistent. The lectures held by the Lehrstuhl für Informatik 6 should provide a guide as to what appropriate notation should look like.
  
• Tables must have titles (appearing above the table).
• Figures must have captions (appearing below the figure).
  
• In the case that no adequate translation of an English technical term is available, the term should be used unchanged.
• Articles and presentation slides can also be prepared in English.
  
• Completeness: acknowledge all literature and sources.
  
• Referencing must conform to the standard described in the article template.
• Examples should be used to illustrate points.
• Examples should be as complex as necessary but as simple as possible.
• Slides should be used as presentation aids and not to replace the role of the presenter; specifically, slides should:
• illustrate important points and relationships;
• remind the audience (and the presenter) of important aspects and considerations;
  
• give the audience an overview of the presentation.
  
• Slides should not contain chunks of text or complicated sentences; rather they should consist of succinct words and terms.
• Use illustrations where appropriate - a picture says a thousand words!
• Abbreviations should be defined at the first usage in the manner demonstrated in the following example: "[...] at the Rheinisch-Westfälischen Technischen Hochschule (RWTH) there are [...]".
  
• Take care to stay within your own topic. To this end participants should be aware of the other topics in the seminar. If applicable, cross-reference other articles and presentations.
• Usage of fonts, typefaces and colors in presentation slides must be consistent and appropriate. Such means should serve to clarify points or relationships, not be applied needlessly or at random.
  
• Care should be taken when selecting fonts for presentation slides (also within diagrams) to ensure legibility on a projector even for those seated far from the screen.

Contact

Dr. Ralf Schlüter
RWTH Aachen
Lehrstuhl für Informatik 6
Ahornstr. 55
52056 Aachen

Raum 6125b
Telefon: 0241 / 80-21612

E-Mail: schlueter@cs.rwth-aachen.de

Last modified: 2014-02-10 10:14:06 Disclaimer.