Public Hand Shape Data Set:

RWTH-PHOENIX-Weather 2014 MS Handshapes

Oscar Koller
Human Language Technology & Pattern Recognition Group
RWTH Aachen University, Germany

In Short:

We are sharing 3359 challenging hand shape test images (45MB). and the training images (17GB). If you use our data, please cite the following publication:

• O. Koller, H. Ney, and R. Bowden. Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data Is Continuous and Weakly Labelled. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3793-3802, Las Vegas, NV, USA, June 2016.
  

Detailed Description:

For our works on hand shape recognition we follow the hand shape taxonomy by the danish sign language lexicon team (Jette H. Kristoffersen and Thomas Troelsgård, Center for Tegnsprog, Denmark http://www.tegnsprog.dk), which amounts to over 60 different hand shapes, often with very subtle differences such as a flexed versus straight thumb. The employed classes are shown in Table1.

We manually labelled 3359 images from the RWTH-PHOENIX-Weather 2014 Development set. Some of the 45 encountered pose-independent hand shape classes are depicted in Figure 1. They show the large intra-class variance and the strong similarity between several classes. The hand shapes occur with different frequency in the data. The distribution of counts per class can be verified in Figure 2 showing that the top 14 hand shapes explain 90% of the annotated samples.

All hand shape icons employed in this document originate from the online Dictionary of New Zealand Sign Language http://nzsl.vuw.ac.nz.

Figure 1: 12 exemplary manually annotated hand shape classes are shown. Three labelled frames per class demonstrate intra-class variance and inter-class similarities.

Figure 2: Ground truth hand shape label count of all 3359 annotations. 45 out of 60 classes have been found in the data and could be labelled. If several hand shapes appear close to one label counting bar, each hand shape alone amounts to the mentioned fraction of labels.

Table 1: Table shows the inventory of modelled handshapes. The column `Nr' corresponds to the model index, where the first output Nr. `0' represents the garbage class (Not in Table).

Nr	Shape	Name	Nr	Shape	Name
1		1	31		l_hook
2		2	32		middle
3		3	33		m
4		3_hook	34		n
5		4	35		o
6		5	36		index
7		6	37		index_flex
8		7	38		index_hook
9		8	39		pincet
10		a	40		ital
11		b	41		ital_thumb
12		b_nothumb	42		ital_nothumb
13		b_thumb	43		ital_open
14		cbaby	44		r
15		obaby	45		s
16		by	46		write
17		c	47		spoon
18		d	48		t
19		e	49		v
20		f	50		v_flex
21		f_open	51		v_hook
22		fly	52		v_thumb
23		fly_nothumb	53		w
24		g	54		y
25		h	55		ae
26		h_hook	56		ae_thumb
27		h_thumb	57		pincet_double
28		i	58		obaby_double
29		jesus	59		m2
30		k	60		jesus_thumb

The data can be downloaded as packaged file containing the annotations and the images.