Robust reinforcement learning
Methods and evaluation for distribution shift, rare failures, curriculum design, sim-to-real transfer, and policies deployed on physical robots.
Ivan Ovinnikov
Reinforcement learning for reliable agents
I develop reinforcement-learning methods for robust behavior under imperfect objectives and distribution shift—from deployed robotics to foundation-model post-training.
Reinforcement Learning Engineer, ANYbotics · ETH Zürich PhD
Selected research
Robust RL for Deployed Legged Robots
End-to-end training and evaluation for locomotion policies under terrain, sensing, and dynamics shift, centered on rare failures and sim-to-real robustness.
Physical deployment, failure-oriented evaluation, and large-scale GPU simulation.
Learning Rewards That Survive Distribution Shift
Invariant reward learning uses variation across demonstrators to identify features that remain stable under dynamics and nuisance shifts.
Improved reward transfer across five MuJoCo tasks under dynamics interventions.
Distribution-Matching Imitation Learning
Sliced-Wasserstein occupancy matching becomes a transition-level reward for off-policy imitation learning.
Remained effective under up to 100× demonstration subsampling on Ant and Humanoid.
Research themes
I study how supervision, objectives, and training distributions determine behavior after deployment. The work connects reward and imitation learning, failure-focused robot training, and diagnostics for RL post-training.
Learning from imperfect objectives
Reward and imitation-learning methods that address distribution matching, misspecification, and behavioral identifiability.
RL post-training and agent learning
Diagnostics for credit assignment, reward and verifier structure, optimization dynamics, and multimodal or embodied agents.
Publications
Learning Causally Invariant Reward Functions from Diverse Demonstrations
Preprint / manuscript · Ivan Ovinnikov, Eugene Bykovets, Joachim M. Buhmann
Fundamentals of Arthroscopic Surgery Training and beyond: a reinforcement learning exploration and benchmark
*International Journal of Computer Assisted Radiology and Surgery* · Ivan Ovinnikov, Ami Beuret, Flavia Cavaliere, Joachim M. Buhmann
Imitation Learning using Generalized Sliced Wasserstein Distances
Research manuscript · Ivan Ovinnikov, Alexander Terenin, Joachim M. Buhmann
Regularizing Adversarial Imitation Learning Using Causal Invariance
ICML 2023 Workshop on Spurious Correlations, Invariance, and Stability · Ivan Ovinnikov, Joachim M. Buhmann
Poincaré Wasserstein Autoencoder
Bayesian Deep Learning Workshop (NeurIPS 2018) · Ivan Ovinnikov
Background
Methods from learning signal to deployment
At ANYbotics, I develop and evaluate locomotion policies for industrial quadruped robots. My ETH Zürich PhD focused on reinforcement learning from demonstrations, reward inference, and distribution shift in surgical digital twins.
I build experiments with PyTorch, JAX, GPU simulation, reproducible evaluation, and HPC workflows.