In this task the goal is to determine the gender of the persons depicted in the individual images.

Evaluation Metrics

We propose to use the following evaluation metrics:

• Accuracy (ACC)
• True Positive Rate (TPR)
• True Negative Rate (TNR)
• Area under the Receiver Operator Characteristic Curve (AUC)

Data Format

The label/fold files are structured such that there is one line per image. Each line starts with the filename followed by the fold ID and the gender. Each of the values is separated by a tab. The gender can be either M for male or F for female. An example line looks like the following:

224896_00M25.JPG        0        M

where 224896_00M25.JPG is the filename of the image, 0 is the fold ID, and M shows that the person depicted on the image is of male gender.

Benchmarking Protocol

For evaluating gender classification approaches 5-fold cross-validation shall be used for both conditions. To prevent algorithms from learning the identity of the persons in the training set rather than the gender it has to be made sure that all images of individual subjects are only in one fold at a time. Additionally, the folds are selected in such a way that the distribution of age, gender and ethnicity in the folds is similar to the distribution in the whole database. The file lists for these folds can be found in the Downloads section below.

Controlled Condition

We propose to use the MORPH-II database for the controlled labratory condition for gender classification.

Uncontrolled Condition

For the uncontrolled condition we decided to use the Labeled Faces in the Wild (LFW) dataset.

Downloads

• Complete Evaluation Guidelines
• Folds for controlled condition of gender classification (old MORPH distribution)
• Folds for controlled condition of gender classification (new MORPH distribution; thanks to Nesli Erdogmus)
• Folds for controlled condition of gender classification on balanced dataset
• Folds for uncontrolled condition of gender classification
• Folds for uncontrolled condition of gender classification on balanced dataset

Author

• Tobias Gehrig, Karlsruhe Institute of Technology, Germany

Evaluation Metrics

• Accuracy (ACC)
• True Positive Rate (TPR)
• True Negative Rate (TNR)

Benchmarking Protocol

See above.

Downloads

• Fold distribution for benchmarking gender classification on Gallagher's dataset

Authors

• Pablo Dago-Casas, GRADIANT
• Daniel González-Jiménez, GRADIANT
• José Luis Alba-Castro, Universidade de Vigo
• Long Long Yu, GRADIANT