Kangrui Ruan (Darren)

I am currently an Applied Scientist at AWS AI Lab, focusing on CodeWhisperer (now Amazon Q inline) and Amazon Q. I obtained my Ph.D. at Columbia University advised by Prof. Xuan Di and Prof. Elias Bareinboim. Before joining Columbia, I obtained my Master’s degree from Carnegie Mellon University (CMU) in 2020.

I am generally interested in LLMs training, RLHF, Code Generation, LLM agent, Causality, Reinforcement Learning and so on. Representative papers are highlighted.

Email  /  CV (last updated: July 2023)  /  LinkedIn  /  Google Scholar  / 

Causal Imitation for Markov Decision Processes: A Partial Identification Approach

In this paper, we investigate robust imitation learning within the framework of canonical Markov Decision Processes (MDPs) using partial identification, allowing the agent to achieve expert performance even when the system dynamics are not uniquely determined from the confounded expert demonstrations. Specifically, we first theoretically demonstrate that when unobserved confounders (UCs) exist in an MDP, the learner is generally unable to imitate expert performance.

From Twitter to Reasoner: Understand Mobility Travel Modes and Sentiment Using Large Language Models

In this study, we introduce a novel methodological framework utilizing Large Language Models (LLMs) to infer the mentioned travel modes from social media posts, and reason people's attitudes toward the associated travel mode, without the need for manual annotation. We compare different LLMs along with various prompting engineering methods in light of human assessment and LLM verification.

S2E: Towards an End-to-End Entity Resolution Solution from Acoustic Signal

Traditional cascading Entity Resolution (ER) pipeline suffers from propagated errors from upstream tasks. We address this issue by formulating a new end-to-end (E2E) ER problem, Signal-to-Entity (S2E), resolving query entity mentions to actionable entities in textual catalogs directly from audio queries instead of audio transcriptions in raw or parsed format.

Causal Imitation Learning via Inverse Reinforcement Learning

This paper has 2 key contributions. First, the paper analyzes structural conditions on the causal model under which learning the expert policy is possible in the presence of unobserved confounding. Second, the authors further exploit knowledge of the graphical structure to extend IRL algorithms such as GAIL or MWAL to the confounded settings.

Learning Human Driving Behaviors with Sequential Causal Imitation Learning

We develop a sequential causal template that generalizes the default MDP settings to one with Unobserved Confounders (MDPUC-HD).

Autonomous navigation at unsignalized intersections: A coupled reinforcement learning and model predictive control approach

This paper develops an integrated safety-enhanced reinforcement learning (RL) and model predictive control (MPC) framework for autonomous vehicles (AVs) to navigate unsignalized intersections.

CVLight: Decentralized Learning for Adaptive Traffic Signal Control with Connected Vehicles

This paper develops a decentralized reinforcement learning (RL) scheme for multi-intersection adaptive traffic signal control (TSC), called “CVLight”, that leverages data collected from connected vehicles (CVs).