ADCL Research Overview

Professor Zachary Sunberg

March 18th, 2022

Research Focus: Decision Making under Uncertainty

Alleatory

Epistemic (Static)

Epistemic (Dynamic)

Interaction

Markov Decision Process

Reinforcement Learning

POMDP

Game

Online Planning in Large POMDPs

Online Planning in Large POMDPs

All drivers normal

Outcome only

Omniscient

Mean MPC

QMDP

POMCPOW

Simulation results

Human Behavior Model: IDM and MOBIL

\ddot{x}_\text{IDM} = a \left[ 1 - \left( \frac{\dot{x}}{\dot{x}_0} \right)^{\delta} - \left(\frac{g^*(\dot{x}, \Delta \dot{x})}{g}\right)^2 \right]
g^*(\dot{x}, \Delta \dot{x}) = g_0 + T \dot{x} + \frac{\dot{x}\Delta \dot{x}}{2 \sqrt{a b}}

M. Treiber, et al., “Congested traffic states in empirical observations and microscopic simulations,” Physical Review E, vol. 62, no. 2 (2000).

A. Kesting, et al., “General lane-changing model MOBIL for car-following models,” Transportation Research Record, vol. 1999 (2007).

A. Kesting, et al., "Agents for Traffic Simulation." Multi-Agent Systems: Simulation and Applications. CRC Press (2009).

All drivers normal

Omniscient

Mean MPC

QMDP

POMCPOW

Conventional 1D POMDP

2D POMDP

Online Planning in Large POMDPs

Intention-Aware Navigation in Crowds with Extended-Space POMDP Planning. Gupta, H.; Hayes, B.; and Sunberg, Z. AAMAS, 2022.

Online Planning in Large POMDPs

Deep RL: Adaptive Stress Testing

Johnathan Tucker and Zachary Sunberg. “Adaptive Stress Testing Applied To Space Domain Awareness Systems”. Abstract under review for the Advanced Maui Optical and Space Surveillance Technologies conference.

Background: https://arxiv.org/abs/1811.02188

Space Domain Awareness Games

\(\mathcal{A} = \mathbb{R}^{N\times N}\)

1

2

...

...

...

...

...

...

...

\(N\)

Tyler Becker and Zachary Sunberg. “Imperfect Information Games and Counterfac-
tual Regret Minimization in Space Domain Awareness”. Abstract under review for the
Advanced Maui Optical and Space Surveillance Technologies conference.

Resolving Equilibrium Uncertainty

POMDPs.jl - An interface for defining and solving MDPs and POMDPs in Julia

Open Source Software

Group

Thank You!

ADCL Research Overview

By Zachary Sunberg

ADCL Research Overview

  • 63