CU Boulder Smead Aerospace

PhD Applicant Visit Day 2025

Mission: deploy autonomy with confidence

Waymo Image By Dllu - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=64517567

Storm Science

Conventional 1D POMDP

2D POMDP

Pedestrian Navigation

[Gupta, Hayes, & Sunberg, AAMAS 2022]

Decision Making under Uncertainty

Theory

Computation

Hardware Testing

Autonomous Decision and Control Laboratory

cu-adcl.org

  • Algorithmic Contributions
    • Scalable algorithms for partially observable Markov decision processes (POMDPs)
    • Motion planning with safety guarantees
    • Game theoretic algorithms
  • Theoretical Contributions
    • Particle POMDP approximation bounds
  • Applications
    • Space Domain Awareness
    • Autonomous Driving
    • Autonomous Aerial Scientific Missions
    • Search and Rescue
    • Space Exploration
    • Ecology
  • Open Source Software
    • POMDPs.jl Julia ecosystem

PI: Prof. Zachary Sunberg

PhD Students

Postdoc

Partially Observable Markov Decision Processes

Breaking the Curse of Dimensionality in POMDPs

\[|Q_{\mathbf{P}}^*(b,a) - Q_{\mathbf{M}_{\mathbf{P}}}^*(\bar{b},a)| \leq \epsilon \quad \text{w.p. } 1-\delta\]

For any \(\epsilon>0\) and \(\delta>0\), if \(C\) (number of particles) is high enough,

[Lim, Becker, Kochenderfer, Tomlin, & Sunberg, JAIR 2023]

No dependence on \(|\mathcal{S}|\) or \(|\mathcal{O}|\)!

POMDP Planning with Learned Components

[Deglurkar, Lim, Sunberg, & Tomlin, 2023]

Navigation among Pedestrians

[Gupta, Hayes, & Sunberg, AAMAS 2022]

Previous solution: 1-D POMDP (92s avg)

Our solution (65s avg)

State:

  • Vehicle physical state
  • Human physical state
  • Human intention

Navigation among Pedestrians

[Gupta, Hayes, & Sunberg, AAMAS 2022]

State:

  • Vehicle physical state
  • Human physical state
  • Human intention

Meteorology

  • State: (physical state of aircraft, which forecast is the truth)
  • Action: (flight direction, drifter deploy)
  • Reward: Terminal reward for correct weather prediction

Drone Search and Rescue

State:

  • Location of Drone
  • Location of Human

Baseline

Our POMDP Planner

[Ray, Laouar, Sunberg, & Ahmed, ICRA 2023]

Drone Search and Rescue

[Ray, Laouar, Sunberg, & Ahmed, ICRA 2023]

Space Domain Awareness

(Result for simplified dynamical system)

State:

  • Position, velocity of object-of-interest
  • Anomalies: navigation failure, suspicious maneuver, thruster failure, etc.

Innovation: Large language models allow analysts to quickly specify anomaly hypotheses

Catalog Maintenance Plan

Tree Search Algorithms for POSGs

[Becker & Sunberg, AAMAS 2025]

POSG Example: Missile Defense

POMDP Solution:

  1. Assume a distribution for the missile's actions
  2. Update belief according to this distribution
  3. Use a POMDP planner to find the best defensive action

Nash equilibrium: All players play a best response to the other players

Fundamentally impossible for POMDP solvers to compute.

May include stochastic behavior (bluffing)

A shrewd missile operator will use different actions, invalidating our belief

Autonomous Decision and Control Laboratory

cu-adcl.org

  • Algorithmic Contributions
    • Scalable algorithms for partially observable Markov decision processes (POMDPs)
    • Motion planning with safety guarantees
    • Game theoretic algorithms
  • Theoretical Contributions
    • Particle POMDP approximation bounds
  • Applications
    • Space Domain Awareness
    • Autonomous Driving
    • Autonomous Aerial Scientific Missions
    • Search and Rescue
    • Space Exploration
    • Ecology
  • Open Source Software
    • POMDPs.jl Julia ecosystem

PI: Prof. Zachary Sunberg

PhD Students

Postdoc

Interaction Uncertainty

[Peters, Tomlin, and Sunberg 2020]

Space Domain Awareness Games

Open question: are there \(\mathcal{S}\)- and \(\mathcal{O}\)-independent algorithms for POMGs?

Incomplete Information Extensive form Game

Our new algorithm for POMGs

c_I = 100.0 \\ c_T = 1.0 \\ c_{TR} = 10.0

COVID POMDPs

Planning Rebuilding Ecosystems

POMDPs.jl - An interface for defining and solving MDPs and POMDPs in Julia

Open Source Software

https://github.com/JuliaPOMDP/POMDPs.jl

Autonomous Decision and Control Laboratory

cu-adcl.org

  • Algorithmic Contributions
    • Scalable algorithms for partially observable Markov decision processes (POMDPs)
    • Motion planning with safety guarantees
    • Game theoretic algorithms
  • Theoretical Contributions
    • Particle POMDP approximation bounds
  • Applications
    • Space Domain Awareness
    • Autonomous Driving
    • Autonomous Aerial Scientific Missions
    • Search and Rescue
    • Space Exploration
    • Ecology
  • Open Source Software
    • POMDPs.jl Julia ecosystem

PI: Prof. Zachary Sunberg

PhD Students

Postdoc

Thank You!

zachary.sunberg.net

ADCL Students

BOMCP

[Mern, Sunberg, et al. AAAI 2021]

MPC for Intermittent Rotor Failures

UAV Component Failures

Reward Decomposition for Adaptive Stress Testing

Voronoi Progressive Widening

[Lim, Tomlin, & Sunberg CDC 2021]

Active Information Gathering for Safety

Sparse PFT

I(t) = \int_0^\infty I(t-\tau)\beta(\tau)d\tau

COVID POMDP

Individual Infectiousness

Infection Age

Incident Infections

\beta(\tau)
\tau
I
I(t) = \int_0^\infty I(t-\tau)\beta(\tau)d\tau
\beta(\tau)

Need

Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance

Larremore et al.

\beta(\tau) \propto \log_{10}(\text{viral load})

Viral load represented by piecewise-linear hinge function

(t_0, 3)
(t_{\text{peak}}, V_{\text{peak}})
(t_f,6)
t_0 \sim \mathcal{U}[2.5,3.5]
t_\text{peak} - t_0 \sim 0.2 + \text{Gamma}(1.8)
V_\text{peak} \sim \mathcal{U}[7,11]
t_f - t_\text{peak} \sim \mathcal{U}[5,10]