We collected a large-scale dataset on three different real world tasks: Laundry Layout, Tower Creation, and Object Search. All three datasets were collected using the RoboTurk platform, collected by crowdsourced workers remotely. Our dataset consists of 2144 different demonstrations from 54 unique users. We provide both the complete dataset for training and smaller subsamples of the dataset for exploration.
We are providing this comprehensive and diverse dataset that, instead of largely 2D manipulation tasks, include tasks with complex 3D motions that can be utilized for similar 3D manipulation tasks. Furthermore, our tasks are long-horizon, so it is important for prediction models to be able to reason about different parts of the task, given some context or history. In addition, our dataset can be used for action-conditioned video prediction for model based predictive control1 or for action-free video prediction2.