 |
 |
 |
 |
The KIT Robo-Kitchen Activity Data Set
Overview
-
Human action and activity recognition from videos has attracted an increasing number of researchers in recent years. However, most of the works aim at multimedia retrieval
and surveillance applications, but rarely at humanoid household robots, even though the robotic perception of human activitie would allow a more natural human-robot interaction (HRI).
To encourage future studies in this domain, we present a novel data set specifically designed for the application in HRI scenarios. This Robo-kitchen data set consists of 14
typical kitchen activities recorded in two different stereo-camera setups, and each performed by 17 subjects.
Sensor setup
-
The recordings were conducted with multiple stereo cameras at a resolution of 640x480 pixels and a frame rate of 15 fps. The cameras were positioned
at different locations in the room that are easily accessible by a robot platform. Due to self occlusions when a person is working at the countertop area, two different sensor setups were used.
One of the main goals was that the activities were performed as natural as possible and thus, the actors only got brief information about what to do, such as where to find the required objects, for how many people to set the table or to perform the activity at a location of their choice at the table.
 |
 |
| room camera setup | countertop camera setup |
Data set
-
All sequences are stored using mp4 video file format (x264 compressed). If you have problems decoding the videos on a Windows machine, an installation
of ffdshow should help. An uncompressed version of the video sequences as well as the stereo data are available
on demand.
In our experiments[1], the testing data consisted of sequences from subjects 1, 3, 5, 8, 12, 14, 15, 20, 24, 25 and the remaining sequences formed the development set. Consequently, we used the data of seven different subjects for testing purposes, since not all subjects are present in each setup.
Please refer to our Humanoids'11 paper[1], if you use this data set in your publications.
Room setup
| door camera | window camera | |||
|---|---|---|---|---|
| activity | file | sample video | file | sample video |
| peel vegetables | .zip 202 MB |
 |
.zip 205 MB |
 |
| cut vegetables | .zip 175 MB |
 |
.zip 165 MB |
 |
| wipe table | .zip 156 MB |
 |
.zip 155 MB |
 |
| set the table | .zip 177 MB |
 |
.zip 181 MB |
 |
| clear the table | .zip 167 MB |
 |
.zip 164 MB |
 |
| empty the dishwasher | .zip 109 MB |
 |
.zip 111 MB |
 |
| sweep the floor | .zip 164 MB |
 |
.zip 151 MB |
 |
| drink coffee and read a newspaper |
.zip 244 MB |
 |
.zip 263 MB |
 |
| eat some pizza | .zip 271 MB |
 |
.zip 286 MB |
 |
| eat some soup | .zip 211 MB |
 |
.zip 212 MB |
 |
Countertop setup
| fridge camera | corner camera | sink camera | ||||
|---|---|---|---|---|---|---|
| activity | file | sample video | file | sample video | file | sample video |
| peel vegetables | .zip 237 MB |
 |
.zip 222 MB |
 |
.zip 222 MB |
 |
| cut vegetables | .zip 193 MB |
 |
.zip 188 MB |
 |
.zip 188 MB |
 |
| fry vegetables | .zip 139 MB |
 |
.zip 133 MB |
 |
.zip 133 MB |
 |
| stir a cooking soup |
.zip 116 MB |
 |
.zip 111 MB |
 |
.zip 112 MB |
 |
| wipe the countertop | .zip 64 MB |
 |
.zip 65 MB |
 |
.zip 61 MB |
 |
| wash dishes | .zip 226 MB |
 |
.zip 212 MB |
 |
N/A | |
| dry the washed dishes |
.zip 149 MB |
 |
.zip 152 MB |
 |
N/A | |
Related publications
| [1] |
The KIT Robo-Kitchen Data set for the Evaluation of View-based Activity Recognition Systems, Lukas Rybok, Simon Friedberger, Uwe D. Hanebeck, and Rainer Stiefelhagen; in IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia, October 2011 [paper] [bibtex] [poster] |
Contact
Last update 17.11.2011