Adaptive training distributions
Curriculum, sampling, and replay methods that shape the learner’s experience using capability, uncertainty, learning progress, and failure risk.
Reinforcement Learning · Reward Learning · Adaptive Training
Ivan Ovinnikov
Sequential decision-making under distribution shift.
Research on reinforcement learning, learning from demonstrations, and adaptive training distributions for robust, data-efficient decision-making.
ANYbotics · Zürich, Switzerland
About
Researching how objectives and experience shape learning.
I am a machine learning researcher working on sequential decision-making under distribution shift. My research spans reinforcement learning, imitation and inverse reinforcement learning, reward learning, curriculum design, and risk-sensitive objectives.
My work combines methodological research with evaluation in complex simulated and physical systems, including surgical digital twins and deployed quadruped locomotion.
Research areas
Learning from demonstrations
Imitation, inverse reinforcement learning, and reward inference for problems where desired behaviour is easier to demonstrate than to specify directly.
Robust and risk-sensitive learning
Objectives and evaluation methods for distribution shift, reward misspecification, and low-probability but consequential failures.
Interactive model post-training
Applying reinforcement learning, reward learning, and adaptive experience generation to multimodal models and interactive agents.
Selected work
Methods, systems, and research artifacts.
Adaptive training, learning from demonstrations, reward design, and evaluation in simulated and physical systems.
RL-Based Quadruped Locomotion Systems
Internal deployment work on RL locomotion controllers, sim-to-real training workflows, and safety-critical evaluation for industrial quadrupeds.
Causally Invariant Reward Learning
Reward-model objective design from diverse demonstrations, with an invariance constraint intended to remain useful when the learned objective is optimized under distribution shift.
FASTRL: Reinforcement Learning in Surgical Digital Twins
Peer-reviewed work on reinforcement-learning benchmarks and assistance policies in surgical digital-twin environments.
Sliced Wasserstein Imitation Learning
Turning optimal-transport distances between expert and policy behavior into practical reinforcement-learning rewards.
Hyperbolic Wasserstein Autoencoders
ArXiv preprint on generative modeling with Wasserstein autoencoders in hyperbolic latent spaces.
Selected publications
Papers and preprints.
Work spanning reward learning, reinforcement learning, causal invariance, and optimal transport.
Learning Causally Invariant Reward Functions from Diverse Demonstrations
Preprint / manuscript · Ivan Ovinnikov, Eugene Bykovets, Joachim M. Buhmann
Fundamentals of Arthroscopic Surgery Training and beyond: a reinforcement learning exploration and benchmark
*International Journal of Computer Assisted Radiology and Surgery* · Ivan Ovinnikov, Ami Beuret, Flavia Cavaliere, Joachim M Buhmann
Imitation Learning using Generalized Sliced Wasserstein Distances
Research manuscript · Ivan Ovinnikov, Alexander Terenin, Joachim M. Buhmann
Regularizing Adversarial Imitation Learning Using Causal Invariance
Preprint / manuscript · Ivan Ovinnikov, Joachim M. Buhmann
Poincare Wasserstein Autoencoder
Preprint / manuscript · Ivan Ovinnikov
Writing
Research notes and updates.
Measuring advantage collapse in VLA reinforcement learning
A work-in-progress measurement plan for locating where the learning signal disappears between reward, advantage estimation, and policy updates.
Research-grade applied ML for embodied agents
A short technical note on connecting reward learning, simulation-based training, and safety-critical evaluation for physical AI systems.
PhD examination
Successfully passed my PhD examination