Learning to Throw with a Handful of Samples Using Decision Transformers

Maxim Monastirsky, Osher Azulay, Avishai Sintov*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Throwing objects by a robot extends its reach and has many industrial applications. While analytical models can provide efficient performance, they require accurate estimation of system parameters. Reinforcement Learning (RL) algorithms can provide an accurate throwing policy without prior knowledge. However, they require an extensive amount of real world samples which may be time consuming and, most importantly, pose danger. Training in simulation, on the other hand, would most likely result in poor performance on the real robot. In this letter, we explore the use of Decision Transformers (DT) and their ability to transfer from a simulation-based policy into the real-world. Contrary to RL, we re-frame the problem as sequence modelling and train a DT by supervised learning. The DT is trained off-line on data collected from a far-from-reality simulation through random actions without any prior knowledge on how to throw. Then, the DT is fine-tuned on an handful (∼5) of real throws. Results on various objects show accurate throws reaching an error of approximately 4 cm. Also, the DT can extrapolate and accurately throw to goals that are out-of-distribution to the training data. We additionally show that few expert throw samples, and no pre-training in simulation, are sufficient for training an accurate policy.

Original languageEnglish
Pages (from-to)576-583
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume8
Issue number2
DOIs
StatePublished - 1 Feb 2023

Keywords

  • Reinforcement learning
  • transfer learning

Fingerprint

Dive into the research topics of 'Learning to Throw with a Handful of Samples Using Decision Transformers'. Together they form a unique fingerprint.

Cite this