Personal
Welcome to the homepage of Oscar Koller.
I have been a research assistant and phd student at the Human Language Technology and Pattern Recognition Group
at the RWTH Aachen University from May 2011 till December 2017.
Since the beginning of my PhD, I enjoyed a dual supervision by Prof. Dr.-Ing. Hermann Ney and Prof. Richard Bowden.
The research topics
of the chair are:
- Pattern Recognition
- Human Language Technology
- Automatic Speech Recognition
- Statistical Machine Translation
- Continuous Sign Language Recognition and Translation
- Handwriting Recognition
- Image Processing
In my scientific work, I have covered following areas (google scholar citations):
- sequence modelling of sign languages for recognition and translation
- face and mouth modelling
- hand shape and orientation recognition
- weakly supervised learning
- model free tracking
- gesture recognition
- activity recognition
- speech recognition
You can visit me in room 6130 of our department, call me at +49 241 80 21617, contact me at LinkedIn or write an e-mail to <surname>@cs.rwth-aachen.de.
Open Research - Sign Language Recognition
You may be interested accessing some of the resources I created during my PhD:
Publications
-
O. Koller, S. Zargaran, Ney Hermann, and R. Bowden. Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs. International Journal of Computer Vision, volume 126, number 12, pages 1311-1325, Heidelberg, Germany, December 2018.
-
C. Camgoz, S. Hadfield, O. Koller, H. Ney, and R. Bowden. Neural Sign Language Translation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, June 2018.
-
N. C. Camgoz, S. Hadfield, O. Koller, and R. Bowden. SubUNets: End-to-end Hand Shape and Continuous Sign Language Recognition. In International Conference on Computer Vision (ICCV), pages 22-27, October 2017.
-
O. Koller, S. Zargaran, and H. Ney. Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3416-3424, Honolulu, HI, USA, July 2017.
-
N. C. Camgoz, S. Hadfield, O. Koller, and R. Bowden. Using Convolutional 3D Neural Networks for User-Independent Continuous Gesture Recognition. In IEEE International Conference of Pattern Recognition, ChaLearn Workshop (ICPR), Cancun, Mexico, December 2016.
-
O. Koller, S. Zargaran, H. Ney, and R. Bowden. Deep Sign: Hybrid CNN-HMM for Continuous Sign Language Recognition. In British Machine Vision Conference (BMVC), York, UK, September 2016.
-
O. Koller, H. Ney, and R. Bowden. Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data Is Continuous and Weakly Labelled. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3793-3802, Las Vegas, NV, USA, June 2016.
-
O. Koller, H. Ney, and R. Bowden. Automatic Alignment of HamNoSys Subunits for Continuous Sign Language Recognition. In LREC Workshop on the Representation and Processing of Sign Languages: Corpus Mining, pages 121-128, Portorož, Slovenia, May 2016.
-
O. Koller, J. Forster, and H. Ney. Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers. Computer Vision and Image Understanding, volume 141, pages 108-125, December 2015.
-
O. Koller, H. Ney, and R. Bowden. Deep Learning of Mouth Shapes for Sign Language. In Third Workshop on Assistive Computer Vision and Robotics, ICCV (ACVR-15), pages 477-483, Santiago, Chile, December 2015.
-
O. Koller, H. Ney, and R. Bowden. Read My Lips: Continuous Signer Independent Weakly Supervised Viseme Recognition. In Proceedings of the 13th European Conference on Computer Vision (ECCV 2014), pages 281-296, Zurich, Switzerland, September 2014.
-
E. Ong, O. Koller, N. Pugeault, and R. Bowden. Sign Spotting using Hierarchical Sequential Patterns with Temporal Intervals. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1931-1938, Columbus, OH, USA, June 2014.
-
J. Forster, C. Schmidt, O. Koller, M. Bellgardt, and H. Ney. Extensions of the Sign Language Recognition and Translation Corpus RWTH-PHOENIX-Weather. In Language Resources and Evaluation (LREC), pages 1911-1916, Reykjavik, Island, May 2014.
-
O. Koller, H. Ney, and R. Bowden. Weakly Supervised Automatic Transcription of Mouthings for Gloss-Based Sign Language Corpora. In LREC Workshop on the Representation and Processing of Sign Languages: Beyond the Manual Channel, pages 98-94, Reykjavik, Iceland, May 2014.
-
C. Schmidt, O. Koller, H. Ney, T. Hoyoux, and J. Piater. Using Viseme Recognition to Improve a Sign Language Translation System. In International Workshop on Spoken Language Translation (IWSLT), pages 197-203, Heidelberg, Germany, December 2013.
-
C. Schmidt, O. Koller, H. Ney, T. Hoyoux, and J. Piater. Enhancing Gloss-Based Corpora with Facial Features Using Active Appearance Models. In International Symposium on Sign Language Translation and Avatar Technology (SLTAT), volume 2, number 1, Chicago, IL, USA, October 2013.
-
J. Forster, O. Koller, C. Oberdörfer, Y. Gweth, and H. Ney. Improving Continuous Sign Language Recognition: Speech Recognition Techniques and System Design. In Workshop on Speech and Language Processing for Assistive Technologies (SLPAT), pages 41-46, Grenoble, France, August 2013.
Satellite Workshop of INTERSPEECH 2013.
-
J. Forster, C. Oberdörfer, O. Koller, and H. Ney. Modality Combination Techniques for Continuous Sign Language Recognition. In Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA), Lecture Notes in Computer Science 7887, pages 89-99, Madeira, Portugal, June 2013.
Best oral paper award.
-
O. Koller, H. Ney, and R. Bowden. May the Force be with you: Force-Aligned SignWriting for Automatic Subunit Annotation of Corpora. In IEEE International Conference on Automatic Face and Gesture Recognition (FG), pages 1-6, Shanghai, China, April 2013.
-
J. Forster, C. Schmidt, T. Hoyoux, O. Koller, U. Zelle, J. Piater, and H. Ney. RWTH-PHOENIX-Weather: A Large Vocabulary Sign Language Recognition and Translation Corpus. In Language Resources and Evaluation (LREC), pages 3785-3789, Istanbul, Turkey, May 2012.
-
O. Koller, A. Abad, and I. Trancoso. Automatic Speech Recognition and Identification of African Portuguese. In PROPOR'2012 - MSc/PhD Thesis Contest (Propor 2012), pages 1-8, Coimbra, Portugal, April 2012.
-
L. J. Rodriguez-Fuentes, M. Penagarikano, A. Varona, M. Diez, G. Bordel, D. Martinez, J. Villalba, A. Miguel, A. Ortega, E. Lleida, A. Abad, O. Koller, I. Trancoso, P. Lopez-Otero, L. Docio-Fernandez, C. Garcia-Mateo, R. Saeidi, M. Soufifar, T. Kinnunen, T. Svendsen, and P. Franti. Multi-site heterogeneous system fusions for the Albayzin 2010 Language Recognition Evaluation. In IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 377-382, Waikoloa, HI, USA, December 2011.
-
A. Abad, O. Koller, and I. Trancoso. The L2F Language Verification Systems for Albayzin-2010 Evaluation. In Proceedings of FALA 2010 (Fala 2010), pages 383-388, Vigo, Spain, November 2010.
-
O. Koller, A. Abad, I. Trancoso, and Céu Viana. Exploiting variety-dependent Phones in Portuguese Variety Identification applied to Broadcast News Transcription. In Interspeech, pages 749-752, Makuhari, Japan, September 2010.
-
O. Koller. Automatic Speech Recognition and Identification of African Portuguese. Master Thesis, Berlin, Germany, June 2010.
-
O. Koller, A. Abad, and I. Trancoso. Exploiting variety-dependent Phones in Portuguese Variety Identification. In The Speaker and Language Recognition Workshop (Odyssey 2010), pages 279-285, Brno, Czech Republic, June 2010.
Invited Talks
I got invited to talk about my research at following occasions:
- "Computer Vision meets Speech Recognition: Sign Language Recognition & Use Cases", NVIDIA, Santa Clara, CA, USA, Jul 2017
- "Automatic Sign Language Recognition with the Statistical Paradigm", Intel Labs, Munich, Germany, June 2017
- "Recent Trends in Automatic Sign Language Recognition", Language and Speech Colloquium, Radboud University Nijmegen, Netherlands, Dec. 2016
- "Sign Language Recognition: Model, Recognise and Search Sign Videos", University of Hamburg, Germany, Jul. 2016
- "Look at My Face, Follow My Hand: Automatic Sign Language Recognition and Translation", Natural Media of Communication, RWTH Aachen University, Germany, Jun. 2016
- "Sign Language Recognition", University of Zurich, Switzerland, Oct. 2015
- "Technologies for the Deaf", Federal Office for Family and Civilright duties (BAFzA), Cologne, Germany, Sep. 2015
- "Bridging the Gap between Signers and Speakers", Humtec, RWTH Aachen University, Germany, May 2012
Peer Reviewing
I was/am on the program committee for the following peer-reviewed conferences and journals:
- FG 2013 - International Conference on Face and Gesture Recognition
- Ibpria 2013 - 6th Iberian Conference on Pattern Recognition and Image Analysis
- IMAVIS 2013 - Image and Vision Computing Journal
- FG 2015 - International Conference on Face and Gesture Recognition
- CSL 2015 - Computer Speech and Language Journal
- GESTURE 2015 - GESTURE Journal
- TPAMI 2015 - IEEE Transactions on Pattern Analysis and Machine Intelligence
- TACCESS 2015 - ACM Transactions on Accessible Computing
- TASL 2016 - IEEE/ACM Transactions on Audio, Speech, and Language Processing
- IJPRAI 2016 - International Journal of Pattern Recognition and Artificial Intelligence
- IMAVIS 2016 - Image and Vision Computing Journal
- MMSJ 2016 - Multimedia Systems Journal
- FG 2017 - International Conference on Face and Gesture Recognition
- IJPRAI 2017 - International Journal of Pattern Recognition and Artificial Intelligence
- TAC 2017 - Transactions on Affective Computing Journal
- INTERSPEECH 2017 - Interspeech
- TCSVT 2017 - IEEE Transactions on Circuits and Systems for Video Technology
- Chalearn @ ICCV 2017 - 2017 Chalearn Looking at People Workshop ICCV
