OnionBot: A System for Collaborative Computational Cooking

Home robots are rapidly developing and help humans to perform different tasks of everyday life. Cooking is one of them. However, an unstructured and dynamic kitchen environment makes manipulation with robotic arms a significant challenge. Therefore, a recent paper suggests employing human-robot collaboration for cooking.

Image credit: Bennet Cobley, David Boyle / Imperial College London

The system assists humans by providing executable instructions and automatically controls heat. It is equipped with a camera, angular position, and temperature sensors. The image classification module learns to differentiate key events such as “Onions cooked until soft”.

The user is automatically guided through recipe instructions and is warned in case of boiling over or if the pan has not been stirred for a duration. During the experiment, a user was able to successfully make pasta with tomato sauce. A future recipe database could be crowd-sourced with users sharing their recipes with images.

An unsolved challenge in cooking automation is designing for shared kitchen workspaces. In particular, robots struggle with dexterity in the unstructured and dynamic kitchen environment. We propose that human-machine collaboration can be achieved without robotic manipulation. We describe a novel system design using computer vision to inform intelligent cooking interventions. This human-centered approach does not require actuators and promotes dynamic, natural collaboration. We show that automation that assists user-led actions can offer meaningful cooking assistance and can generate the image databases needed for fully autonomous robotic systems of the future. We provide an open source implementation of our work and encourage the research community to build upon it.

Link: https://arxiv.org/abs/2011.05039