Scientists figured out how robots could see transparent things

Robots can’t see transparent objects very well. This is a big problem, because many of our everyday objects are actually transparent. Scientists were thinking how to solve this issue and researchers at Carnegie Mellon University finally came to a good solution.

Problem. Many robots use IR sensors to locate various objects. For the most part, this technology works very well. However, these sensors are blind to transparent objects. This is a huge problem for modern robotics. How can you expect a robot not to run into a glass wall? How will your robotic butler bring you a glass of water if it can’t even see the glass? How will robots in chemistry labs collect and wash various tubes and other glassware? The closer we can get robotic vision to human capabilities, the better.

Robots typically struggle with transparent objects, because IR cameras just don’t see them very well. Image credit: Carnegie Mellon University

Solution. Scientists at CMU found that a normal colour camera can see transparent and reflective objects as well as opaque ones. And cameras are very cheap – that’s why there are so many camera modules even on cheap devices. However, a colour camera can’t measure shapes as well as the depth sensor. But scientists managed to circumvent this problem by training the system using both the depth sensor and the colour camera together.

Result. Scientists trained a robot to grasp various objects. And in general this technology works very well. It is able to grasp transparent and opaque objects. However, it is still more comfortable with non-transparent and non-reflective objects. With these tricky things, the robotic arm still makes many mistakes, even though it is more successful than not.

Trained robot in action:

The best thing is that this system uses a normal commercially available RGB-D camera. This module is basically a colour camera with a depth sensor. It allows the robot to grab various things, even if they are in an untidy pile of mess. David Held, one of the authors of the system, said: “We do sometimes miss, but for the most part it did a pretty good job, much better than any previous system for grasping transparent or reflective objects. Our system not only can pick up individual transparent and reflective objects, but it can also grasp such objects in cluttered piles”.

Will this make the future of home robots more certain? Well, it is already pretty certain. Just that this technology will make those robots better sooner. Probably the best thing is that these camera modules are very inexpensive. Robots are going to be littered with them for that reason alone.