What else is the ADCL doing?
Professor Zachary Sunberg
University of Colorado Boulder
CAAMS Meeting, Feb. 2024
Autonomous Decision and Control Laboratory
-
Algorithmic Contributions
- Scalable algorithms for partially observable Markov decision processes (POMDPs)
- Motion planning with safety guarantees
- Game theoretic algorithms
-
Theoretical Contributions
- Particle POMDP approximation bounds
-
Applications
- Space Domain Awareness
- Autonomous Driving
- Autonomous Aerial Scientific Missions
- Search and Rescue
- Space Exploration
- Ecology
-
Open Source Software
- POMDPs.jl Julia ecosystem
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10272214/pasted-from-clipboard.png)
PI: Prof. Zachary Sunberg
PhD Students
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8458281/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8458283/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8458284/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8458406/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9421050/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10229532/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10273976/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/11106504/pasted-from-clipboard.png)
Postdoc
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10964498/orbital.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/11106615/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10964509/aspen-and-hex.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8462117/rrbt_considerbelief.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10275503/hippo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10964516/pasted-from-clipboard.png)
POMDP (decision problem) is PSPACE Complete
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10961923/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10961924/pasted-from-clipboard.png)
Breaking the Curse of Dimensionality in POMDPs
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9903364/pomdp-pb-mdp-2.png)
\[|Q_{\mathbf{P}}^*(b,a) - Q_{\mathbf{M}_{\mathbf{P}}}^*(\bar{b},a)| \leq \epsilon \quad \text{w.p. } 1-\delta\]
For any \(\epsilon>0\) and \(\delta>0\), if \(C\) (number of particles) is high enough,
[Lim, Becker, Kochenderfer, Tomlin, & Sunberg, JAIR 2023]
No dependence on \(|\mathcal{S}|\) or \(|\mathcal{O}|\)!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8832042/resusing_old_hybrid_astar_path_1d_action_space_speed_pomdp_planner_run.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8832043/safe_planner2.gif)
Conventional 1DOF POMDP
Multi-DOF POMDP
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8832045/pasted-from-clipboard.png)
Pedestrian Navigation
[Gupta, Hayes, & Sunberg, AAMAS 2021]
POMDP Planning with Learned Components
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8832055/pasted-from-clipboard.png)
[Deglurkar, Lim, Sunberg, & Tomlin, 2023]
Explainability
Provide explanations to improve trust and enable cooperation
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10492283/smead_annhj_aes_rev_left_color_0.webp)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10492286/ADCL-Gold.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10492708/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10492737/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10492708/pasted-from-clipboard.png)
"Action 2 accounts for changes in availability window"
Explanation
i
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10519652/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10519653/pasted-from-clipboard.png)
1
2
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10519653/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10519652/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10519653/pasted-from-clipboard.png)
1
2
![](https://s3.amazonaws.com/media-p.slid.es/uploads/2004519/images/10519653/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/11106426/pasted-from-clipboard.png)
"Everything is a POMDP"
"Autonomous storm data gathering is a POMDP"
Storm Science
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10954008/merged.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10954310/output.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9936153/single_tree.png)
Simplified Space Domain Awareness Game
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1539327/images/8994832/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1539327/images/8953973/pasted-from-clipboard.png)
1
2
...
...
...
...
...
...
...
\(N\)
[Becker & Sunberg AMOS 2021]
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9859702/deterministic-strategy.png)
Counterfactual Regret Minimization Training
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9859691/mode-change-game.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9348773/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9856540/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9856546/pasted-from-clipboard.png)
Open question: are there \(\mathcal{S}\)- and \(\mathcal{O}\)-independent algorithms for POMGs?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9052289/pasted-from-clipboard.png)
Incomplete Information Extensive form Game
Our new algorithm for POMGs
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/11106525/pasted-from-clipboard.png)
Airborne Wind Energy Optimization
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/11106555/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/11106557/pasted-from-clipboard.png)
![](http://juliapomdp.github.io/POMDPs.jl/latest/figures/concepts.png)
POMDPs.jl - An interface for defining and solving MDPs and POMDPs in Julia
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/4924690/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/4924692/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/4924693/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/4924695/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/4924723/pasted-from-clipboard.png)
Autonomous Decision and Control Laboratory
-
Algorithmic Contributions
- Scalable algorithms for partially observable Markov decision processes (POMDPs)
- Motion planning with safety guarantees
- Game theoretic algorithms
-
Theoretical Contributions
- Particle POMDP approximation bounds
-
Applications
- Space Domain Awareness
- Autonomous Driving
- Autonomous Aerial Scientific Missions
- Search and Rescue
- Space Exploration
- Ecology
-
Open Source Software
- POMDPs.jl Julia ecosystem
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10272214/pasted-from-clipboard.png)
PI: Prof. Zachary Sunberg
PhD Students
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8458281/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8458283/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8458284/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8458406/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/9421050/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10229532/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10273976/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/11106504/pasted-from-clipboard.png)
Postdoc
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10964498/orbital.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/11106615/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10964509/aspen-and-hex.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/8462117/rrbt_considerbelief.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10275503/hippo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/870752/images/10964516/pasted-from-clipboard.png)
CAAMS - What else are you doing?
By Zachary Sunberg
CAAMS - What else are you doing?
- 107