Cevahir Koprulu

I am a Ph.D. student at the Department of Electrical and Computer Engineering, University of Texas at Austin. I am advised by Prof. Ufuk Topcu. I received my B.Sc. degree from the Department of Electrical and Electronics Engineering at Bilkent University, where I worked with Assoc. Prof. Yildiray Yildiz.

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn Twitter  /  Center for Autonomy

profile photo
Taken at UT Austin, 2024.

News

Research

I do research on reinforcement learning (RL), with a focus on generalization and curriculum learning (CL). My publications can be found on my Google Scholar page, and some are listed with details below. During my Ph.D. so far, I have done research internships at Bosch Center for Artificial Intelligence and Honda Research Institute. At Bosch, I developed CL4AD, the first integration of CL into batched autonomous driving simulators, accelerating training by 77%. At HRI, I developed an action advising framework, Gen2Spec, that distills knowledge from generalist agents to specialists in a continual learning setting. Recently, I have been focusing on the following research directions:

  • Risk-aware CL for RL-based Post Training: We develop a risk-aware curriculum generation algorithm for RL-based fine-tuning of language models.
  • Conformal Prediction for Uncertainty-aware CL: We adopt conformal prediction for curriculum learning to prioritize tasks with high model uncertainty during post-training.
  • Synthetic Curricula for Autonomous Driving: We guide generative models via curriculum learning for synthetic traffic scenario generation.

Curriculum Learning for Reinforcement Learning

The design of task sequences, i.e., curricula, improves the performance of RL agents and speeds up the convergence in complex tasks. An effective curriculum typically begins with easy tasks and gradually changes them toward the target tasks. Common approaches require manually tailoring the curricula to identify easy and hard tasks, which requires domain knowledge that might be unavailable. My research focuses on developing automated curriculum generations algorithms that utilize generative models, introduce notions of risk and uncertainty, address constrained RL, and exploit task specifications.

Publications

Safety-Prioritizing Curricula for Constrained Reinforcement Learning
Cevahir Koprulu, Thiago D. Simão, Nils Jansen, Ufuk Topcu
International Conference on Learning Representations (ICLR), 2025

Risk-aware Curriculum Generation for Heavy-tailed Task Distributions
Cevahir Koprulu, Thiago D. Simão, Nils Jansen, Ufuk Topcu
Conference on Uncertainty in Artifical Intelligence (UAI), 2023

Reward-Machine-Guided, Self-Paced Reinforcement Learning
Cevahir Koprulu, Ufuk Topcu
Conference on Uncertainty in Artifical Intelligence (UAI), 2023

Reward-Machine-Guided, Self-Paced Reinforcement Learning
Cevahir Koprulu, Ufuk Topcu
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2023 (accepted as Extended Abstract)

Workshops

Prioritizing Safety via Curriculum Learning
Cevahir Koprulu, Thiago D. Simão, Nils Jansen, Ufuk Topcu
RLBRew and RLSW at Reinforcement Learning Conference , 2024

Exploiting Side Information for Offline and Mixed RL

We investigate use of side information to improve the performance of RL agents offline and mixed (offline + online) settings. For offline RL, we develop an uncertainty-aware, offline model-based reinforcement learning approach (NUNO) with neural stochastic differential equations, levelaraging prior physics knowledge as inductive bias, improving state-of-the-art in low-quality data regimes. For mixed settings, we introduce a systematic reward-shaping framework that distills the information contained in (1) a task-agnostic prior data set and (2) a few task-specific expert demonstrations for dense dynamics-aware reward synthesis.

Publications

Neural Stochastic Differential Equations for Uncertainty-Aware Offline RL
Cevahir Koprulu, Franck Djeumou, Ufuk Topcu
International Conference on Learning Representations (ICLR), 2025

Dense Dynamics-Aware Reward Synthesis: Integrating Prior Experience with Demonstrations
Cevahir Koprulu, Po-han Li, Tianyu Qiu, Ruihan Zhao, Tyler Westenbroek, David Fridovic-Keil, Sandeep Chinchali, Ufuk Topcu
Learning for Dynamics and Control (L4DC) Conference , 2025

Learning Reward Machines and Policies

We study the problem of reinforcement learning for a task encoded by a reward machine. The task is defined over a set of properties in the environment, called atomic propositions, and represented by Boolean variables. One unrealistic assumption commonly used in the literature is that the truth values of these propositions are accurately known. In real situations, however, these truth values are uncertain since they come from sensors that suffer from imperfections. At the same time, reward machines can be difficult to model explicitly, especially when they encode complicated tasks. We develop a reinforcement-learning algorithm that infers a reward machine that encodes the underlying task while learning how to execute it, despite the uncertainties of the propositions' truth values.

Publications

Joint Learning of Reward Machines and Policies in Environments with Partially Known Semantics
Christos Verginis, Cevahir Koprulu, Sandeep Chinchali, Ufuk Topcu
Aritificial Intelligence, 2024

Level-k Game Theory to Model Human Drivers

Level-k game theory is a hierarchical multi-agent decision-making model where a level-k player is a best responder to a level-(k-1) player. In this work, we studied level-k game theory to model reasoning levels of human drivers. Different from existing methods, we proposed a dynamic approach, where the actions are the levels themselves, resulting in a dynamic behavior. The agent adapts to its environment by exploiting different behavior models as available moves to choose from, depending on the requirements of the traffic situation.

Publications

Act to Reason: A Dynamic Game Theoretical Driving Model for Highway Merging Applications
Cevahir Koprulu, Yildiray Yildiz
IEEE Conference on Control Technology and Applications (CCTA), 2021

Act to reason: A dynamic game theoretical model of driving
Cevahir Koprulu, Yildiray Yildiz
ArXiV, 2021


This website is based on Jon Barron's source code. His website can be found here.