Cevahir Koprulu

I am a Ph.D. student at the Department of Electrical and Computer Engineering, University of Texas at Austin. I am advised by Prof. Ufuk Topcu. I received my B.Sc degree from the Department of Electrical and Electronics Engineering at Bilkent University, where I worked with Assoc. Prof. Yildiray Yildiz.

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn Twitter  /  Center for Autonomy

profile photo
Taken at UT Austin, 2024.

News

Research

I do research on reinforcement learning (RL), with a focus on generalization and adaptation. My publications can be found on my Google Scholar page, and some are listed with details below. During my research internship at HRI, I developed an action advising framework, Gen2Spec, that distills knowledge from generalist meta RL agents to specialists in a continual learning setting. Recently, I have been focusing on the following research directions:

  • CL for Multi-Modality RL: We study automated curriculum generation for domains with multiple modalities.
  • CL for Outliers in Fine-tuning: We investigate how to address outlier tasks/scenarios in fine-tuning of foundation models.

Curriculum Learning for Reinforcement Learning

The design of task sequences, i.e., curricula, improves the performance of RL agents and speeds up the convergence in complex tasks. An effective curriculum typically begins with easy tasks and gradually changes them toward the target tasks. Common approaches require manually tailoring the curricula to identify easy and hard tasks, which requires domain knowledge that might be unavailable. My research focuses on developing automated curriculum generations algorithms that introduce a notion of risk, address constrained RL, and exploits task specifications.

Publications

Safety-Prioritizing Curricula for Constrained Reinforcement Learning
Cevahir Koprulu, Thiago D. Simão, Nils Jansen, Ufuk Topcu
International Conference on Learning Representations (ICLR), 2025

Risk-aware Curriculum Generation for Heavy-tailed Task Distributions
Cevahir Koprulu, Thiago D. Simão, Nils Jansen, Ufuk Topcu
Conference on Uncertainty in Artifical Intelligence (UAI), 2023

Reward-Machine-Guided, Self-Paced Reinforcement Learning
Cevahir Koprulu, Ufuk Topcu
Conference on Uncertainty in Artifical Intelligence (UAI), 2023

Reward-Machine-Guided, Self-Paced Reinforcement Learning
Cevahir Koprulu, Ufuk Topcu
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2023 (accepted as Extended Abstract)

Workshops

Prioritizing Safety via Curriculum Learning
Cevahir Koprulu, Thiago D. Simão, Nils Jansen, Ufuk Topcu
RLBRew and RLSW at Reinforcement Learning Conference , 2024

Learning Reward Machines and Policies

We study the problem of reinforcement learning for a task encoded by a reward machine. The task is defined over a set of properties in the environment, called atomic propositions, and represented by Boolean variables. One unrealistic assumption commonly used in the literature is that the truth values of these propositions are accurately known. In real situations, however, these truth values are uncertain since they come from sensors that suffer from imperfections. At the same time, reward machines can be difficult to model explicitly, especially when they encode complicated tasks. We develop a reinforcement-learning algorithm that infers a reward machine that encodes the underlying task while learning how to execute it, despite the uncertainties of the propositions' truth values.

Publications

Joint Learning of Reward Machines and Policies in Environments with Partially Known Semantics
Christos Verginis, Cevahir Koprulu, Sandeep Chinchali, Ufuk Topcu
Aritificial Intelligence, 2024

Level-k Game Theory to Model Human Drivers

Level-k game theory is a hierarchical multi-agent decision-making model where a level-k player is a best responder to a level-(k-1) player. In this work, we studied level-k game theory to model reasoning levels of human drivers. Different from existing methods, we proposed a dynamic approach, where the actions are the levels themselves, resulting in a dynamic behavior. The agent adapts to its environment by exploiting different behavior models as available moves to choose from, depending on the requirements of the traffic situation.

Publications

Act to Reason: A Dynamic Game Theoretical Driving Model for Highway Merging Applications
Cevahir Koprulu, Yildiray Yildiz
IEEE Conference on Control Technology and Applications (CCTA), 2021

Act to reason: A dynamic game theoretical model of driving
Cevahir Koprulu, Yildiray Yildiz
ArXiV, 2021


This website is based on Jon Barron's source code. His website can be found here.