Learning a data-efficient model for a single agent in homogeneous multi-agent systems

Anton Gurevich, Eran Bamani, Avishai Sintov*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Training Reinforcement Learning (RL) policies for a robot requires an extensive amount of data recorded while interacting with the environment. Acquiring such a policy on a real robot is a tedious and time-consuming task. This is more challenging in a multi-agent system where individual data may be required from each agent. While training in simulations is the common approach due to efficiency and low-cost, they rarely describe the real world. Consequently, policies trained in simulations and transferred to the real robot usually perform poorly. In this paper, we present a novel real-to-sim-to-real framework to bridge the reality gap for an agent in collective motion of a homogeneous multi-agent system. First, we propose a novel deep neural-network architecture termed Convolutional-Recurrent Network (CR-Net) to capture the complex state transition of an agent and simulate its motion. Once trained with data from one agent, we show that the CR-Net can accurately predict motion of all agents in the group. Second, we propose to invest a limited amount of real data from the agent in a generative model. Then, training the CR-Net with synthetic data sampled from the generative model is shown to be at least equivalent to real data. Hence, the proposed approach provides a sufficiently accurate model with significantly less real data. The generative model can also be disseminated along with open-source hardware for easier usage. We show experiments on ground and underwater vehicles in which multi-agent RL policies are trained in the simulation for collective motion and successfully transferred to the real-world.

Original languageEnglish
Pages (from-to)20069-20085
Number of pages17
JournalNeural Computing and Applications
Volume35
Issue number27
DOIs
StatePublished - Sep 2023

Keywords

  • Collective motion
  • Forward dynamics model
  • Multi-Agent
  • Robot modeling

Fingerprint

Dive into the research topics of 'Learning a data-efficient model for a single agent in homogeneous multi-agent systems'. Together they form a unique fingerprint.

Cite this