Contact

Fo further information, you should contact Philippe Dreuw and check the homepage of Philippe Dreuw at http://www-i6.informatik.rwth-aachen.de/~dreuw/.

Automatic Sign Language Recognition (ASLR)

Developing sign language applications for deaf people can be very important, as many of them, being not able to speak a language, are also not able to read or write a spoken language. Ideally, a translation systems would make it possible to communicate with deaf people. Compared to speech commands, hand gestures are advantageous in noisy environments, in situations where speech commands would be disturbing, as well as for communicating quantitative information and spatial relationships.

A gesture is a form of non-verbal communication made with a part of the body and used instead of verbal communication (or in combination with it). Most people use gestures and body language in addition to words when they speak. A sign language is a language which uses gestures instead of sound to convey meaning combining hand-shapes, orientation and movement of the hands, arms or body, facial expressions and lip-patterns. Contrary to popular belief, sign language is not international. As with spoken languages, these vary from region to region. They are not completely based on the spoken language in the country of origin.

Sign language is a visual language and consists of 3 major components:

Similar to automatic speech recognition (ASR), we focus in automatic sign language recognition (ASLR) on automatically recognizing sign language videos as gloses, which can be later translated by a statistical machine transaltion system into written text (see a demo video).

ASL sign language demo

Benchmark Databases for Sign Language Recognition and Translation

In the course of the diploma thesis work Appearance-Based Gesture Recognition of Philippe Dreuw, a new database of fingerspelling letters of German Sign Language (Deutsche Gebärdensprache, DGS) was created. The RWTH Fingerspelling Database contains 35 gestures with video sequences for the signs A to Z and SCH, the German umlauts Ä, Ö, Ü, and for the numbers 1 to 5. Five of the gestures contain inherent motion (J, Z, Ä, Ö and Ü). The recording was done under non-uniform daylight lighting conditions, the back- ground and the camera viewpoints are not constant, and the persons had no restrictions on the clothing while gesturing.

Other ASLR sign language recognition databases used at our institute:

If you are interested in using these sign language recognition databases too, please contact Philippe Dreuw or check his web page.


Philippe Dreuw > Research > Sign Language Recognition

Philippe Dreuw
Last modified: Thu Oct 25 10:00:27 CEST 2007 Disclaimer. Created Wed Dec 22 18:04:32 CET 2004

Valid HTML 4.01! xemacs logo debian logo