Pattern Recognition and Neural Networks

News

• January 27^th, 2009: A solution to task 1 a/b is now available:  
• January 21^st, 2009: Starting today it is possible to register with us for taking the oral exam. Three different timeframes are available, each consisting of two dates:
  • February 9th-10th, 2009
  • March 9th-10th, 2009
  • March 23th-24th, 2009
  Please sign up for one of the date pairs by sending an email to Jonas Lööf, stating name, immatriculation number, study direction in addition to your choosen time. Idealy to sign up for a certain time, it should be possible for you to take the exam at any time during the two days. You will be notified of the exact time when everyone has registered.
• January 14^th, 2009: There was a misprint on todays exercise sheet regarding the hand in date. The hand in date is Wednesday January 21^st, 2009, and nothing else.
• January 7^th, 2009: Everyone needing a separate (oral) exam for the course -- Master- and Batchelour students, etc:
  Please send an email to Jonas Lööf, stating:
  • Name
  • Immatriculation number
  • Study direction
  • Type of examination needed
  • Preferences regarding the date of exam.
• December 10^th, 2008: The portable, resonably fast, solution to Exercise 4, Problem 2, is now available on our ftp server.
• December 3^rd, 2008: As I mentioned on todays excercise lesson, if you want to use Netlab to solve the USPS tasks, you will need an extension that uses log-probabilities as discriminant. I have updatded the software section on Netlab (below) with instructions on this.
• November 19^th, 2008: As stated on todays exercise lecture, the hand-in date of exercise sheet 5 is postponed to the December 3^rd, 2008 - to allow for more time to discuss the solutions to task 3, as well as the new sheet.

Contents

• basic statistics
• training and learning
• model-free approaches
• neural networks and discriminative training
• error integral: characteristics and estimates
• mixture distributions and cluster analysis
• EM-algorithm and hidden Markov models
• feature extraction and linear mappings

Lecture Notes (Access only permitted within the RWTH domain)

• Overview of i6: Research and Courses  
• Lecture Notes (SS 05, English)    
• Lecture Notes (SS 02, German)    
• Appendix: Support Vector Machines, Logistic Regression, and Log-linear Models (SS 07, English)  
• Appendix: Maximum Entropy (WS 00/01, English)    

Software

• Netlab - a Toolbox for Matlab/Octave
  The implementation task of exercise sheet four is optional. To solve the remaining problems (if you choose not to do the implementation) you can utilize "Netlab", a free Matlab toolbox that also works with the free software Octave. Netlabs main focus lies on neural networks, but it also contains (limited) support for Gaussian models. Please consult the homepage of Netlab on how to install it. If you don't have access to Matlab, install Octave; any reasonably new version should work, I have used version 2.1.73. For windows, binaries are available for download here, for linux, use the package system of your distribution.
  Please note: This is not a complete solution, since this only implements and trains Gaussian distributions; you will still have to make the further work of using them for classification, and do the estimation of the class priors. Furthermore, no covariance pooling is implemented, but this can be done after the fact (as in Exercise sheet 4, task 2b.) As a final note, this software implements Gaussian mixture models. To use as single Gaussian models, you need to set the parameter ncentres to 1.
  To get some of Netlabs functions to work if your using Octave, you will have to download this file and put it where Octave can find it. (For instance in the same directory as the Netlab functions.)
  For the USPS task, in addition to the standard Netlab package, you will need an extension that uses a logarithm based implementation for Gaussian models. This extension is available from the Netlab homepage, but contains a problem/bug, so a fixed version is available on our ftp server.
  
  
• Python
  The demo program in exercise 2 is written in Python, and you will have to have python installed to run it. On a Windows computer, download and install this file from www.python.org (give the default answer to all installation questions), then you should be able to run the program by double clicking it. On most Linux computers Python is already installed and it should be enough to execute the script directly from the command line; if not consult the documentation of your distribution on how to install Python. Modern Mac computers already have Python installed, you can start the script by typing 'python gauss_demo.py' in a terminal window. Other users will have to download and install the correct file from the Python homepage.