Joint work with awesome collaborators @yilundu.bsky.social, Antonia Bronars, Jyo Pari, @joshtenenbaum.bsky.social, @tianminshu.bsky.social and Pulkit Agrawal.

Drop by our poster Thu, 4:30-7:30pm PT, West Ballroom A-D #6602! neurips.cc/virtual/2024...

We evaluate our approach extensively and demonstrate that we successfully learn novel tasks and generate corresponding agent plans and motion in (1) unseen configurations and (2) in composition with training tasks.

Thanks to generative models’ compositional properties, in addition to inverting demos to a single latent representation, we invert demos into the latent space of composed generative models, enabling us to learn new tasks that are substantially different from the training tasks.

Tasks are often defined by a policy, reward, or trajectory, requiring retraining for new tasks. Inspired by few-shot visual concept learning via inverting generative models, we infer tasks represented as latent vectors without retraining.