Learning and intervention design for sequential decision-making
I study how structural assumptions and the design of training distributions and objectives constrain policy learning and behavioral generalization in sequential decision-making systems.
Reinforcement Learning Engineer, ANYbotics · ETH Zürich PhD
Training and evaluation for locomotion policies under terrain, sensing, and dynamics shift, with failure-oriented evaluation feeding weak deployment regions back into training.
Physical deployment, failure-oriented evaluation, and large-scale GPU simulation.
Uses variation across demonstrators as structural information to discourage reward features that fail under dynamics and nuisance shifts.
Improved reward transfer across five MuJoCo tasks under dynamics interventions.
Uses transport geometry to turn distribution-level imitation objectives into transition-level learning signals.
Remained effective under up to 100× demonstration subsampling on Ant and Humanoid.
Across these projects, I study how limited training information constrains what a sequential decision-making system can learn, and how additional structure or actively selected experience can resolve the remaining ambiguity. When data are fixed, this motivates structural priors and invariances; when the data-generating process can be controlled, it motivates adapting environments, tasks, or experiments to the current learner.
What structure should learning assume?
Learning policies or objectives from demonstrations is underdetermined. My PhD work studied causal invariance and distributional geometry as inductive structure for learning rewards and policies that remain useful under distribution shift.
Where should the learner train next?
In deployed robot learning, the environment distribution becomes a controllable part of training. I use policy performance and failure signals to identify weak regions and direct training effort toward behavior that matters under terrain, sensing, and dynamics shift.
What intervention should the system run next?
A broader direction I am exploring is how to select the next environment, task, or experiment based on what is currently uncertain and what matters for downstream decisions. This connects curriculum and environment design in RL with active data acquisition and goal-oriented experimental design.
Agent post-training. I am exploring how ideas from adaptive environment design transfer to task distributions, generated environments, and verifier-guided learning.
Preprint / manuscript · Ivan Ovinnikov, Eugene Bykovets, Joachim M. Buhmann
*International Journal of Computer Assisted Radiology and Surgery* · Ivan Ovinnikov, Ami Beuret, Flavia Cavaliere, Joachim M. Buhmann
Research manuscript · Ivan Ovinnikov, Alexander Terenin, Joachim M. Buhmann
ICML 2023 Workshop on Spurious Correlations, Invariance, and Stability · Ivan Ovinnikov, Joachim M. Buhmann
Bayesian Deep Learning Workshop (NeurIPS 2018) · Ivan Ovinnikov
At ANYbotics, I develop and evaluate reinforcement-learning-based locomotion policies for industrial quadruped robots, with a focus on robustness, failure analysis, and sim-to-real iteration. My ETH Zürich PhD studied reinforcement learning from demonstrations: how reward and policy learning can exploit causal structure and distributional geometry when demonstrations provide incomplete information about the intended behavior.
Across both settings, I am increasingly interested in the active version of the same problem: how to choose the environments, tasks, or interventions from which a sequential system should learn next.
I build experiments with PyTorch, JAX, GPU simulation, reproducible evaluation, and HPC workflows.