A robot that finds lost objects

A busy passenger is ready to go out the door; They want to find out which pile of keys is hidden by a quick search for clutter.

MIT researchers have developed a robotic system that can do that. The system, RFusion, is a robot arm with a camera and a radio frequency (RF) antenna attached to the handle. Even though the object is buried under a pile and is completely out of sight, it detects signals from the antenna with the visual input from the camera to find and retrieve something.

The RFusion prototype developed by the researchers relies on RFID tags, which are cheap, battery-free tags attached to an item and reflect antenna signals. Because RF signals can travel through most surfaces (a dirty laundry pile that can be covered with keys), an RFusion labeled item can be found in a stack.

Using machine learning, the robot’s arm automatically zeros in on the correct position of the object, moves the object over it, grabs the object, and makes sure it picks up the correct object. Camera, antenna, robotic arm and eye are fully integrated, so RFusion can operate in any environment without the need for special settings.

While it is important to find the missing keys, RFusion can have a wide range of applications, such as stacking orders in a warehouse, sorting and loading parts in a car factory, or helping an elderly person perform daily tasks at home. Although the current model is not yet very fast for these uses.

“In this chaotic world, access to goods is an open problem that we have been working on for a few years. The existence of robots that can search for objects under piles is a growing demand in the industry today. Now you may think of this as a roba on steroids, but in the near future this could have many applications in the manufacturing and warehousing areas, ”said Associate Professor of Electrical Engineering and Computer Science at MIT Media Laboratory, Signal Kinesics Team Science and Director.

Collaborative authors include research assistant Tara Borushakin, author. Isaac Perper, a graduate student in electrical engineering and computer science; Research associate Mergen Nachin; And Alberto Rodriguez, 1957 Associate Professor in the Department of Mechanical Engineering. The study will be presented to the Computer Engineers Conference Conference on Network Sino Systems next month.

Sending signals

To identify the spherical location of the RFusion tag, start looking for something from the RFID tag (like a mirror shining like a mirror). It combines that sphere with camera input, which saves space. For example, the item cannot be found in an empty table area.

But once the robot has a general idea of ​​where the object is, it wants to swing around the room with additional dimensions to bring it to a slow and ineffective location.

The researchers used a tutorial to train a robotic neural network. In the tutorial, the algorithm is trained by trial and error with a reward system.

“Our brains also learn. We get rewards from our teachers, parents, computer games, and so on. The same thing happens with tutoring. We allow the agent to do wrong or do the right thing and then punish or reward the network. This is how the network learns what is really difficult for a model, ”explains Borushaki.

In the case of RFusion, an optimization algorithm is awarded when it limits the number of movements it takes to locate the item and the distance traveled to pick it up.

Once the system has identified the exact location, the neural network uses integrated RF and visual information to estimate how the robot’s arm should handle the object, including the angle of the arms and the handle, and so on. . It also scans the item for the last time to make sure it is the right thing to do.

Cut in half

The researchers tested RFusion in different areas. They buried the key chain in a cluttered box and hid the remote control under the sofa.

But if they fed all the camera data and RF measurements into the tutorial algorithm, it would have overwhelmed the system. Therefore, by drawing on the method GPS uses to integrate data from satellites, they summarize the RF parameters and limit the visual data in front of the robot.

Their approach worked well – RFusion had a 96% success rate by uncovering completely hidden objects under the pile.

“Sometimes, if you rely only on RF measurements, there will be external conditions, and if you rely only on vision, sometimes it will be wrong with the camera. But if you combine them, they will correct each other. That is what makes the system so strong, ”said Borushaki.

In the future, researchers hope to increase the speed of the system so that it operates more efficiently, rather than stopping to take measurements. This allows RFusion to be deployed in a fast-paced production or warehouse setting.

In addition to its potential industrial use, such a system could be incorporated into any modern household to assist people in the future, said Brushaki.

“Every year, billions of RFID tags are used to identify items in today’s complex supply chains, including clothing and many other consumer goods. The RFusion approach paves the way for automated robots to dig deeper into a pile of mixed objects rather than examine each item individually, using the information stored in RFID tags, especially from objects from a computer vision system, ”said Matthew S. Reynolds, who did not participate in the study. Associate Professor of Computer Engineering, President of Innovation. “The RFusion approach is a big step forward for robots working in the right supply chain and is the key to quickly and accurately identifying the right item and delivering orders on time and making customers happy.

The study is sponsored by the National Science Foundation, the Sloan Research Union, NTT Information, Topan, Topan Forms, and Abdul Latif Jamel Water and Food Systems Laboratory.


Leave a Comment