I am a PhD student at the Bethge Lab, part of the Tübingen AI Center at the University of Tübingen, Germany. I am an ELLIS PhD Student and a scholar of the International Max Planck Research School for Intelligent Systems (IMPRS-IS). I also work with Tom Silver’s group for robotics and planning in Princeton, US. Previously, I was a researcher at FBK NLP in Trento, Italy, and I obtained my MSc degree in Machine Learning, Data Science and Artificial Intelligence from Aalto University in Helsinki, Finland.
I want to build autonomous AI agents that learn from experience and adapt quickly to novel situations. I am interested in how agents turn prior knowledge and experience into effective behavior, and what they do when their existing knowledge or strategies are no longer sufficient.
My recent work explores code as a way for agents to represent what they know: using language models to write executable world models and policies, programs that can be tested, revised, and reused to plan and act. I am particularly interested in how agents recognize the limits of these programs, recover from unexpected situations, and retain what they learn. I work with both simulated games and real-world robotics as environments for agents to learn and adapt in. I am also curious about the connections between AI and cognitive science, and how insights from human cognition can inform the development of more intelligent agents.
Task and motion planning (TAMP) problems remain difficult even with full observability and object-centric states because discrete decisions are tightly coupled to geometric, kinematic, and dynamic constraints. Generalized TAMP addresses this difficulty by exploiting regularities across problem instances to reduce planning effort on new instances. However, existing methods require substantial TAMP-specific engineering. We investigate whether coding agents can automate this process by synthesizing programs that generalize across instances. Given a task description and simulator access, each agent chooses how to interact with the environment while developing a program within a fixed synthesis budget. The program is then frozen and evaluated on unseen instances. We evaluate Claude Code (Opus 5) and Codex (GPT-5.6 Sol and GPT-6 Astra) on 28 simulated environments from KinDER and PDDLStream, with object counts beyond those evaluated in the original benchmark. Across all program synthesis methods, we evaluate 980 generated programs on 100 held-out instances each, 98,000 evaluation episodes in total. Overall, we find that coding agents are surprisingly effective at generalized TAMP: all three agent configurations outperform hand-engineered planners, one-shot generation, and an LLM-based generalized planning baseline in mean success (56% to 95% versus 47% for the planners, on the 16 environments where a planner is available). As object counts grow, the agents’ programs maintain higher success than the planner, using an order of magnitude less computation per instance on average. Logs show agents using interaction to calibrate physical models, test edge cases, and refine strategies. We release all code, including the full prompts given to the agents. These findings suggest that coding agents are a strong baseline for generalized TAMP.
@misc{merler2026coding,title={Coding Agents for Generalized Task and Motion Planning Problems},author={Merler, Matteo and Li, Bowen and Roy, Josh and Liang, Yichao and Wang, Qianwei and Huang, Yixuan and Silver, Tom},year={2026},month=sep,cv_date={2026-09-24},eprint={2609.30233},archiveprefix={arXiv},primaryclass={cs.RO},url={https://arxiv.org/abs/2609.30233},}
In this work we consider Code World Models, world models generated by a Large Language Model (LLM) in the form of Python code for model-based Reinforcement Learning (RL). Calling code instead of LLMs for planning has potential to be more precise, reliable, interpretable, and extremely efficient. However, writing appropriate Code World Models requires the ability to understand complex instructions, to generate exact code with non-trivial logic and to self-debug a long program with feedback from unit tests and environment trajectories. To address these challenges, we propose Generate, Improve and Fix with Monte Carlo Tree Search (GIF-MCTS), a new code generation strategy for LLMs. To test our approach in an offline RL setting, we introduce the Code World Models Benchmark (CWMB), a suite of program synthesis and planning tasks comprised of 18 diverse RL environments paired with corresponding textual descriptions and curated trajectories. GIF-MCTS surpasses all baselines on the CWMB and two other benchmarks, and we show that the Code World Models synthesized with it can be successfully used for planning, resulting in model-based RL agents with greatly improved sample efficiency and inference speed.
@inproceedings{dainese2024generating,author={Dainese, Nicola and Merler, Matteo and Alakuijala, Minttu and Marttinen, Pekka},booktitle={Advances in Neural Information Processing Systems},editor={Globerson, A. and Mackey, L. and Belgrave, D. and Fan, A. and Paquet, U. and Tomczak, J. and Zhang, C.},pages={60429--60474},publisher={Curran Associates, Inc.},title={Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search},url={https://proceedings.neurips.cc/paper_files/paper/2024/hash/6f479ea488e0908ac8b1b37b27fd134c-Abstract-Conference.html},volume={37},year={2024},cv_venue={Advances in Neural Information Processing Systems 37 (NeurIPS 2024)},cv_date={2024-12-13},}
