There are two main ways to reduce the sim-to-real gap. The first is accurate system identification, where we try to model the real robot as precisely as possible in simulation. However, real systems often contain complex and unobserved effects that are difficult to model perfectly.
Therefore, we also use domain randomization. Instead of relying on one perfectly identified simulation model, domain randomization exposes the policy to a range of physical variations and uncertainties during training. This makes the policy more robust to modeling errors and allows us to be slightly less strict about perfectly matching every real-world parameter.
Although domain randomization improves robustness, it is important to choose valid/feasible ranges for each term. The goal is to balance robustness and training efficiency: the range should be wide enough to cover real-world uncertainty, but not so wide that it creates unrealistic dynamics and confuses the policy during learning.
Practically, we started with basic domain randomization terms:
reset_base
Randomizes the robot initial base pose at reset, including x/y position, height offset, and yaw angle. This prevents the policy from overfitting to one fixed starting pose.push_robot
Applies random external velocity perturbations during the episode every 1–3 seconds. This trains the policy to recover from disturbances and improves push robustness.foot_friction
Randomizes the foot-ground friction coefficient in the range 0.3–1.2. This makes the policy robust to different ground surfaces.encoder_bias
Adds random joint encoder bias in the range -0.015 to 0.015 rad. This simulates small sensor calibration errors or joint position measurement offsets.base_com
Randomizes the torso/base center of mass offset in x/y/z.(5cm) This helps the policy handle mass-distribution mismatch between simulation and the real robot.rough_terrain
Trains the policy on slightly rough terrain with small height variations. This improves robustness to uneven ground and prevents the policy from overfitting to perfectly flat terrain.
During real deployment, we found that base_com is one of the most critical terms for improving policy robustness, because small errors in the robot’s mass distribution can strongly affect balance and gait stability. Therefore, we increased the base_com randomization range in x/y/z from 5cm to 10 cm. In addition, we also randomized the link masses to further improve robustness against hardware-model mismatch.
To further improve policy robustness and avoid feet sliding, we trained the policy on slightly rough terrain using the following terrain configuration:
COBBLESTONE_ROAD_CFG = terrain_gen.TerrainGeneratorCfg(
size=(8.0, 8.0),
border_width=20.0,
num_rows=9,
num_cols=21,
difficulty_range=(0.0, 1.0),
sub_terrains={
"flat": terrain_gen.BoxFlatTerrainCfg(proportion=0.2),
"random_rough": terrain_gen.HfRandomUniformTerrainCfg(
proportion=0.2,
noise_range=(0.02, 0.05),
noise_step=0.02,
horizontal_scale=0.1,
vertical_scale=0.005,
border_width=0.25,
),
},
)
Figure3. Asimov walking in slight rough terrain