MIT scientists use AI to make a robot carry out multiple tasks

Called the compositional diffusion continuous constraint solver (Diffusion-CCSP), the model learns a family of diffusion models, a kind of generative artificial intelligence models that are trained together, so they share some knowledge, like the geometry of the objects the robot will tackle.

Their method uses generative AI to help robots solve transport problems like collisions and the stability of the dumped material. Traditional solutions to these problems are time-consuming.

“My vision is to push robots to do more complicated tasks that have many geometric constraints and more continuous decisions that need to be made — these are the kinds of problems service robots face in our unstructured and diverse human environments,” said Zhutian Yang, an electrical engineering and computer science graduate student and lead author the study.

“With the powerful tool of compositional diffusion models, we can now solve these more complex problems and get great generalization results,” added Yang.

The researchers’ primary motivation was to solve subproblems that arise during general robot manipulation planning. The researchers explained this in the press release using the example of packing objects in a car. They said one constraint might require a certain object to be next to another object, while a second constraint might specify where one of those objects must be located. For Diffusion-CCSP, the researchers wanted to capture the interconnectedness of the constraints.

“We don’t always get to a solution at the first guess. But when you keep refining the solution and some violation happens, it should lead you to a better solution. You get guidance from getting something wrong,” added Yang.

Yang explained that training individual models is time-consuming, costly, and requires a lot of training data. Her team found an alternative approach. They used fast algorithms to generate segmented boxes and fit a diverse set of 3D objects into each segment, ensuring tight packing, stable poses, and collision-free solutions.

“With this process, data generation is almost instantaneous in simulation. We can generate tens of thousands of environments where we know the problems are solvable,” she says.