@arhanjain.bsky.social
Paper: arxiv.org/abs/2412.01770
Website: casher-robot-learning.github.io/CASHER/
USD assets: github.com/casher-robot...
Collaborators:
@marcelto.bsky.social
Carrie Yuan
Vidyaraanya Macha
@ankile.bsky.social
Anthony S
Pulkit Agrawal
@abhishekunique7.bsky.social
Robot Learning with Super-Linear Scaling
Scaling robot learning requires data collection pipelines that scale favorably with human effort. In this work, we propose Crowdsourcing and Amortizing Human Effort for Real-to-Sim-to-Real(CASHER), a ...
arxiv.org
December 5, 2024 at 5:48 AM
Paper: arxiv.org/abs/2412.01770
Website: casher-robot-learning.github.io/CASHER/
USD assets: github.com/casher-robot...
Collaborators:
@marcelto.bsky.social
Carrie Yuan
Vidyaraanya Macha
@ankile.bsky.social
Anthony S
Pulkit Agrawal
@abhishekunique7.bsky.social
We also have some great 3D interactive examples of our environments and rollouts on our website! We've released our simulation assets, and I'm excited to see where real2sim2real can take us 🤠 (6/N)
December 5, 2024 at 5:48 AM
We also have some great 3D interactive examples of our environments and rollouts on our website! We've released our simulation assets, and I'm excited to see where real2sim2real can take us 🤠 (6/N)
An amazing capability this enables is policies can learn to operate in new environments with nothing but a video of your new environment! We call this Scanned Deployed Finetuning. (5/N)
December 5, 2024 at 5:48 AM
An amazing capability this enables is policies can learn to operate in new environments with nothing but a video of your new environment! We call this Scanned Deployed Finetuning. (5/N)
But wait, wheres the self improvement?
We realize that even 10 demos becomes a burden when you scale up to 100, 500, 1000... environments. After an initial training phase of the generalist policy, we leverage it to provide its own demos to bootstrap RL and master new environments! (4/N)
We realize that even 10 demos becomes a burden when you scale up to 100, 500, 1000... environments. After an initial training phase of the generalist policy, we leverage it to provide its own demos to bootstrap RL and master new environments! (4/N)
December 5, 2024 at 5:48 AM
But wait, wheres the self improvement?
We realize that even 10 demos becomes a burden when you scale up to 100, 500, 1000... environments. After an initial training phase of the generalist policy, we leverage it to provide its own demos to bootstrap RL and master new environments! (4/N)
We realize that even 10 demos becomes a burden when you scale up to 100, 500, 1000... environments. After an initial training phase of the generalist policy, we leverage it to provide its own demos to bootstrap RL and master new environments! (4/N)
We then trained policies on these crowdsourced environments with RL bootstrapped from ~10 demos.After collecting rollouts from state-based policies, we distill them into a generalist visuo-motor policy that zero-shot/few-shot transfers to the real world! (3/N)
December 5, 2024 at 5:48 AM
We then trained policies on these crowdsourced environments with RL bootstrapped from ~10 demos.After collecting rollouts from state-based policies, we distill them into a generalist visuo-motor policy that zero-shot/few-shot transfers to the real world! (3/N)
We crowdsourced data across 3 different continents from family & friends - anyone can do it with their phone! We use 3D reconstruction methods like Gaussian splats to make diverse, visually & geometrically realistic sim environments for training policies (2/N)
December 5, 2024 at 5:48 AM
We crowdsourced data across 3 different continents from family & friends - anyone can do it with their phone! We use 3D reconstruction methods like Gaussian splats to make diverse, visually & geometrically realistic sim environments for training policies (2/N)