The Karlsruhe Motion, Intention, and Activity Data set (MINTA)

Overview

    The MINTA data set was collected in a kitchen setting, where ten different persons performed seven kitchen tasks. For each task, the person entered the scene, performed manipulations at the table, and left the scene afterwards. The seven recorded tasks are: Lay Table, Prepare Cereals, Prepare Pudding, Eat with Spoon, Eat with Fork, Clear Table, and Wipe Table.

    The intentions combine the activities with motions and domain knowledge. We define nine intentions Lay Table, Prepare Cereals, Prepare Pudding, Spoon Breakfast, Spoon Lunch, Cut Breakfast, Cut Lunch, Clear Table, Wipe Table, which differentiate the tasks by object and time knowledge. We denote activities as tasks, that can be discerned without the need of explicit object knowledge. Hence, the tasks Prepare Cereals and Prepare Pudding are considered one activity, i.e., Prepare Meal, resulting in a total of six activity classes. For a fine-grained recognition of the performed tasks a set of 60 motion primitives was defined, e.g., Place Object on Table, Pour, or Stir.

    The sequences were recorded with a Point Grey Dragon-Fly Camera with a resolution of 640x480 pixels and a frame rate of 30 fps was used. The recordings contain a mix of artificial and day light (9 AM to 8 PM) as well as textured and plain background.

Video samples of all kitchen tasks

MINTA data set

    All sequences are stored using AVI video file format (x264 compressed). If you have problems decoding the videos on a Windows machine, an installation of ffdshow should help. An uncompressed version of the video sequences is available on demand.

    In the experiments reported in our IROS'11 paper, we used the recordings of person P01 and person P02 as test data, and the remaining sequences for training purposes.
    Please refer to Gehrig2011, if you use this data set in your publications.

    Person IDvideosmotion annotationsintention annotations
    P01avixmltxt
    P02avixmltxt
    P03avixmltxt
    P04avixmltxt
    P05avixmltxt
    P06avixmltxt
    P07avixmltxt
    P08avixmltxt
    P09avixmltxt
    P10avixmltxt

    A vocabulary of all defined motion primitives can be downloaded from here.

Related publications

    Combined Intention, Activity, and Motion Recognition for a Humanoid Household Robot,
    Dirk Gehrig, Peter Krauthausen, Lukas Rybok, Hildegard Kuehne, Uwe D. Hanebeck, Tanja Schultz, and Rainer Stiefelhagen;
    in International Conference on Intelligent Robots and Systems (IROS), San Francisco, USA, September 2011 [paper] [bibtex]

Contact

Last update 18.11.2011