The RBO Dataset of Articulated Objects and Interactions
Contact:Manuel Baum
baum at tu minus berlin dot de
Authors:
Roberto Martín-Martín
Clemens Eppner
Oliver Brock
The RBO dataset of articulated objects and interactions is a collection of 358 RGB-D video sequences (67:18 minutes) of humans manipulating 14 articulated objects under varying conditions (light, perspective, background, interaction). All sequences are annotated with ground truth of the poses of the rigid parts and the kinematic state of the articulated object (joint states) obtained with a motion capture system. We also provide complete kinematic models of these objects (kinematic structure and three-dimensional textured shape models). In 78 sequences the contact wrenches during the manipulation are also provided.
If you are using this data, please cite:
@misc{1806.06465,
Author = {Roberto Martín-Martín and Clemens Eppner and Oliver Brock},
Title = {The RBO Dataset of Articulated Objects and Interactions},
Year = {2018},
Eprint = {arXiv:1806.06465},
}
This dataset is published under the Creative Commons Attribution 4.0 International Public License.
Data Structure and Usage
All data is availabe in two versions: as csv files and images or as rosbags. Since there are already a lot of tools available for handling rosbags we recommend the later one. Both versions contain time series of:
- RGB images
- Depth images
- Marker positions
- Rigid body poses
- Articulated object poses and configurations
- Interaction wrenches
You can download the data following the links below or you can also go to the zenodo website for the dataset. We also provide the script rbo_downloader.py to simplify this process. You can define the downloading root folder (option --output_dir), the objects models and/or interaction sequences to download (options --objects and --interactions with arguments), if you prefer to download rosbags instead of raw sensor data (option --ros), and if the downloaded files should be automatically decompressed (option --no_decomp, default is auto-decomp). You can also download groups of interactions per property, e.g. with_ft for all interactions with force/torque (wrenches) measurements. To see the complete options, use the argument -h.
For example, python rbo_downloader.py --output_dir ./mydatasetfolder --objects ikea globe --interactions dark will download the models for the ikea furniture and the world globe and all the interactions with dark lighting conditions into the folder mydatasetfolder.
Visualizing data in CSV files and images
We provide the script rbo_visualizer.py to visualize the data and show how to read it and use it. You can visualize the RGB, depth, force/torque (wrenches) and/or joint state values for an interaction. To use the script you just need to provide as argument the folder with the interaction files and the flags for the types of data to visualize (--rgb, --d, --ft, and/or --js).
For example, python rbo_visualizer.py ./mydatasetfolder/interactions/ikea/ikea01_0 --rgb --js will start the visualization of the RGB images and joint states of the interaction in the provided folder. You can pause/resume the visualization pressing Enter in the terminal.
Visualizing data in rosbags
We provide a ROS package to visualize rosbags and models. To visualize rosbags of interactions together with the models of the articulated object (and the stick with the force/torque sensor if used), execute:
rosrun articulated_object_db play_recording.py <interaction_name>
the interaction_name is the name of the final rosbag (e.g. pliers01_o.bag) that you want to visualize. You can type rosrun articulated_object_db play_recording.py -h to see additional help. It is also possible to play a recording without starting rviz, which allows the use of custom launch configurations, if required.
To make possible for the ROS package to locate the dataset, create a link to your downloaded data by executing the following within the folder of the ROS package: ‘ln -s ~/folder_containing_rbo_dataset/ ./data’
Contributing
We are glad to augment our dataset with new models of articulated objects and/or sensor data of interactions with them. We can generate models of articulated objects from trajectories of the links tracked by a motion capture system (or any other 6D pose tracker) if you provide:
- Shape models for each link
- Transformation between the origin of the shape models and the tracked frames If your interaction data includes images, please provide also the pose of the camera wrt. the object. If you provide interaction wrenches, provide also the dynamic properties (mass, center of Mass, inertia matrix) of tool between the force/torque sensor and the object and the pose of the sensor wrt. object.
Please, contact Manuel Baum (baum@tu-berlin.de) to include your data into the dataset.
Models of Articulated Objects
| Object | Photo | Model | Video |
|---|---|---|---|
| Book |  |
 |
 |
| Cabinet |  |
 |
 |
| Cardboard box |  |
 |
 |
| Clamp |  |
 |
 |
| Folding rule |  |
 |
 |
| Globe |  |
 |
 |
| Ikea |  |
 |
 |
| Ikea small |  |
 |
 |
| Laptop |  |
 |
 |
| Microwave |  |
 |
 |
| Pliers |  |
 |
 |
| Rubik’s cube |  |
 |
 |
| Treasure box |  |
 |
 |
| Tripod |  |
 |
 |