CU Boulder Smead Aerospace
PhD Applicant Visit Day 2025
Mission: deploy autonomy with confidence
Waymo Image By Dllu - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=64517567
Storm Science
Conventional 1D POMDP
2D POMDP
Pedestrian Navigation
[Gupta, Hayes, & Sunberg, AAMAS 2022]
Decision Making under Uncertainty
Theory
Computation
Hardware Testing
Autonomous Decision and Control Laboratory
-
Algorithmic Contributions
- Scalable algorithms for partially observable Markov decision processes (POMDPs)
- Motion planning with safety guarantees
- Game theoretic algorithms
-
Theoretical Contributions
- Particle POMDP approximation bounds
-
Applications
- Space Domain Awareness
- Autonomous Driving
- Autonomous Aerial Scientific Missions
- Search and Rescue
- Space Exploration
- Ecology
-
Open Source Software
- POMDPs.jl Julia ecosystem
PI: Prof. Zachary Sunberg
PhD Students
Postdoc
Partially Observable Markov Decision Processes
Breaking the Curse of Dimensionality in POMDPs
\[|Q_{\mathbf{P}}^*(b,a) - Q_{\mathbf{M}_{\mathbf{P}}}^*(\bar{b},a)| \leq \epsilon \quad \text{w.p. } 1-\delta\]
For any \(\epsilon>0\) and \(\delta>0\), if \(C\) (number of particles) is high enough,
[Lim, Becker, Kochenderfer, Tomlin, & Sunberg, JAIR 2023]
No dependence on \(|\mathcal{S}|\) or \(|\mathcal{O}|\)!
POMDP Planning with Learned Components
[Deglurkar, Lim, Sunberg, & Tomlin, 2023]
Navigation among Pedestrians
[Gupta, Hayes, & Sunberg, AAMAS 2022]
Previous solution: 1-D POMDP (92s avg)
Our solution (65s avg)
State:
- Vehicle physical state
- Human physical state
- Human intention
Navigation among Pedestrians
[Gupta, Hayes, & Sunberg, AAMAS 2022]
State:
- Vehicle physical state
- Human physical state
- Human intention
Meteorology
- State: (physical state of aircraft, which forecast is the truth)
- Action: (flight direction, drifter deploy)
- Reward: Terminal reward for correct weather prediction
Drone Search and Rescue
State:
- Location of Drone
- Location of Human
Baseline
Our POMDP Planner
[Ray, Laouar, Sunberg, & Ahmed, ICRA 2023]
Drone Search and Rescue
[Ray, Laouar, Sunberg, & Ahmed, ICRA 2023]
Space Domain Awareness
(Result for simplified dynamical system)
State:
- Position, velocity of object-of-interest
- Anomalies: navigation failure, suspicious maneuver, thruster failure, etc.
Innovation: Large language models allow analysts to quickly specify anomaly hypotheses
Catalog Maintenance Plan
Tree Search Algorithms for POSGs
[Becker & Sunberg, AAMAS 2025]
POSG Example: Missile Defense
POMDP Solution:
- Assume a distribution for the missile's actions
- Update belief according to this distribution
- Use a POMDP planner to find the best defensive action
Nash equilibrium: All players play a best response to the other players
Fundamentally impossible for POMDP solvers to compute.
May include stochastic behavior (bluffing)
A shrewd missile operator will use different actions, invalidating our belief
Autonomous Decision and Control Laboratory
-
Algorithmic Contributions
- Scalable algorithms for partially observable Markov decision processes (POMDPs)
- Motion planning with safety guarantees
- Game theoretic algorithms
-
Theoretical Contributions
- Particle POMDP approximation bounds
-
Applications
- Space Domain Awareness
- Autonomous Driving
- Autonomous Aerial Scientific Missions
- Search and Rescue
- Space Exploration
- Ecology
-
Open Source Software
- POMDPs.jl Julia ecosystem
PI: Prof. Zachary Sunberg
PhD Students
Postdoc
Interaction Uncertainty
[Peters, Tomlin, and Sunberg 2020]
Space Domain Awareness Games
Open question: are there \(\mathcal{S}\)- and \(\mathcal{O}\)-independent algorithms for POMGs?
Incomplete Information Extensive form Game
Our new algorithm for POMGs
c_I = 100.0 \\
c_T = 1.0 \\
c_{TR} = 10.0
COVID POMDPs
Planning Rebuilding Ecosystems
POMDPs.jl - An interface for defining and solving MDPs and POMDPs in Julia
Open Source Software
Autonomous Decision and Control Laboratory
-
Algorithmic Contributions
- Scalable algorithms for partially observable Markov decision processes (POMDPs)
- Motion planning with safety guarantees
- Game theoretic algorithms
-
Theoretical Contributions
- Particle POMDP approximation bounds
-
Applications
- Space Domain Awareness
- Autonomous Driving
- Autonomous Aerial Scientific Missions
- Search and Rescue
- Space Exploration
- Ecology
-
Open Source Software
- POMDPs.jl Julia ecosystem
PI: Prof. Zachary Sunberg
PhD Students
Postdoc
Thank You!
ADCL Students
BOMCP
[Mern, Sunberg, et al. AAAI 2021]
MPC for Intermittent Rotor Failures
UAV Component Failures
Reward Decomposition for Adaptive Stress Testing
Voronoi Progressive Widening
[Lim, Tomlin, & Sunberg CDC 2021]
Active Information Gathering for Safety
Sparse PFT
I(t) = \int_0^\infty I(t-\tau)\beta(\tau)d\tau
COVID POMDP
Individual Infectiousness
Infection Age
Incident Infections
\beta(\tau)
\tau
I
I(t) = \int_0^\infty I(t-\tau)\beta(\tau)d\tau
\beta(\tau)
Need
Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance
Larremore et al.
\beta(\tau) \propto \log_{10}(\text{viral load})
Viral load represented by piecewise-linear hinge function
(t_0, 3)
(t_{\text{peak}}, V_{\text{peak}})
(t_f,6)
t_0 \sim \mathcal{U}[2.5,3.5]
t_\text{peak} - t_0 \sim 0.2 + \text{Gamma}(1.8)
V_\text{peak} \sim \mathcal{U}[7,11]
t_f - t_\text{peak} \sim \mathcal{U}[5,10]
PhD Applicant Visit Day
By Zachary Sunberg
