Zero-Shot Robot Design with Residual Physics

The automatic design of robots usually occurs in simulation and remains challenging due to unmodeled dynamics that arise during deployment. These dynamics gaps are typically viewed as a nuisance to be accommodated by domain randomization or removed by filtering the search space. This prevents robots from acquiring morphological innovations that exploit the dynamics of their physical environment. Recently neural networks have been trained to capture the residual dynamics missing from a base simulator, but this approach has been limited to control policy optimization in a predesigned robot; optimization could only exploit a fixed, morphology-specific subset of possible dynamics. Here we show for the first time the evolution of a robot's body plan in a neural augmented simulator. This is a uniquely challenging problem as the body plan is optimized to exploit the full set of fitness-relevant dynamics---including those due to neural inaccuracies, which in effect rewards the exploitation of hallucinated dynamics. We show that an ensemble of residual models can identify and avoid these hallucinations, and that morphologies optimized in our augmented training environment recover the majority of the performance degradation induced by the missing dynamics. Although we study the dynamics gap between simulations (of slush), which provides a convenient and transparent testbed, our approach only relies on data that could in principle be efficiently collected in the real world. We also validate the simulations by building an evolved design. Future work will extend our approach to simulation-reality gaps.