Roboticists and researchers at MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) are working to develop a system that can train robots to perform tasks in specific environments effectively. The ongoing research aims to help robots deal with disturbances, distractions, and changes in their operational environments. For this, they have proposed a method to create digital models of real-world environments. A user can then capture a digital replica of the real world with just their phones and use these replicas to train their robots. This approach would speed up the training process considerably.
The project has a system named RialTo, which aims to create a network that can teach robots to perform tasks in the scanned environments. First, a user scans their environment using tools like NeRFStudio, ARCode, or Polycam. After the scene is reconstructed, users can upload it to RialTo’s interface for making detailed adjustments and adding necessary joints to the robots.
Once the refined scene is exported into the simulator, a policy based on real-world actions and observations is created. This policy guides the robot in performing the actions in the real world. The system has been tested in various tasks in controlled lab settings and in natural settings and recorded a 67% improvement over imitation learning with the same number of demonstrations.
The next aim for the researchers is to improve the system’s adaptability to new environments and its ability to deal with a range of disturbances. They aim to use pre-trained models to speed up the learning process and minimize human input. Even though there are limitations to the current system, such as the need for initial demonstrations by a human and significant computation time for training, the researchers are optimistic about the possibility of creating robots that can quickly learn new tasks without extensive interactions with the real world.
Simulations have proven to be effective for preparing robots for real-world applications by providing virtually unlimited data for learning. Critics of the technology, however, argue that it is currently limited to specific scenarios and constructing corresponding simulations is expensive and time-consuming. But this new method could drastically reduce the time and resources needed for simulations.
The researchers believe RialTo has the potential to significantly upscale robot learning, enabling real-world adaptability. This could eventually lead to a future where robots can learn to adapt and perform tasks in real-world environments within a significantly reduced time frame.
This research is supported by the Sony Research Award, the U.S. government, and Hyundai Motor Co. The researchers have recently presented their work at the Robotics Science and Systems (RSS) conference.
