Agents that learn modular models of what their actions do.
I study how agents can learn structured world models: factored models in which each variable has its own local mechanism on an explicit graph, and how to use them to decide under uncertainty. With Neural Bayesian Networks, the graph stays explicit while each conditional is learned by a neural density estimator, so models run efficiently on GPUs, keep a modular, inspectable structure, and can be revised one mechanism at a time when the world changes. At Oxford I am exploring how such models connect to Markov decision processes and formal methods, to reason about how far an agent's decisions can be trusted.
Supervised by Franco Zambonelli and Stefano Mariani at UNIMORE. At Oxford I’m hosted by Alessandro Abate and Francesco Fabiano.
Three questions shape my PhD
They go from how a world model is represented, to how decisions built on it can be trusted, to what changes when many agents share one environment.
How should such models enter planning, policy optimisation and value estimation? And, in ongoing work at Oxford, how can the uncertainty of a learned model be carried into the decisions made with it, so that we can say how far those decisions can be trusted?
How can causality-driven RL extend to multi-agent settings, where cooperation and competition add new layers of interdependence?
Papers and preprints
Neural Bayesian Networks for Modular and Scalable Decision-Making Under Uncertainty
Neural Bayesian Networks as Continual World Models for Reinforcement Learning
Scaling Multi-Agent Epistemic Planning through GNN-Derived Heuristics
Causal Learning and Reasoning in Multi-Agent Reinforcement Learning
Causal Models Improve Reinforcement Learning for Pervasive and Robotic Tasks
Causal Learning and Reasoning in Multi-Agent Systems: Goals, Issues, and Taxonomy
Towards Safe Action Policies in Multi-robot Systems with Causal Reinforcement Learning
A Roadmap Towards Improving Multi-Agent Reinforcement Learning With Causal Discovery And Inference
Improving Reinforcement Learning-Based Autonomous Agents with Causal Models
Bearing Fault Detection and Recognition From Supply Currents With Decision Trees