[Submitted on 19 Apr 2020]

Download PDF

Abstract: Combining model-based and model-free learning systems has been shown to
improve the sample efficiency of learning to perform complex robotic tasks.
However, dual-system approaches fail to consider the reliability of the learned
model when it is applied to make multiple-step predictions, resulting in a
compounding of prediction errors and performance degradation. In this paper, we
present a novel dual-system motor learning approach where a meta-controller
arbitrates online between model-based and model-free decisions based on an
estimate of the local reliability of the learned model. The reliability
estimate is used in computing an intrinsic feedback signal, encouraging actions
that lead to data that improves the model. Our approach also integrates
arbitration with imagination where a learned latent-space model generates
imagined experiences, based on its local reliability, to be used as additional
training data. We evaluate our approach against baseline and state-of-the-art
methods on learning vision-based robotic grasping in simulation and real world.
The results show that our approach outperforms the compared methods and learns
near-optimal grasping policies in dense- and sparse-reward environments.