Incomplete Information

Partially Observable Markov Decision Process (POMDP)

• $\mathcal{S}$ - State space
• $T:\mathcal{S}\times \mathcal{A} \times\mathcal{S} \to \mathbb{R}$ - Transition probability distribution
• $\mathcal{A}$ - Action space
• $R:\mathcal{S}\times \mathcal{A} \to \mathbb{R}$ - Reward
• $\mathcal{O}$ - Observation space
• $Z:\mathcal{S} \times \mathcal{A}\times \mathcal{S} \times \mathcal{O} \to \mathbb{R}$ - Observation probability distribution

Alleatory

Hierarchy of Problems

Belief updates?

Pruning in Dynamic Programming

Extensive to Matrix Form

A more interesting example: Kuhn Poker

Fictitious Play in Extensive Form Games

This slide not covered on exam

Heinrich et al. 2015 "Fictitious Self Play in Extensive-Form Games"

Deep Stack: Scaling to Heads Up No Limit Texas Hold 'Em

Can game learning methods like CFR be used in Large POMGs?

Title Text

Next Year: Add papers like the stratego paper, AlphaStar, etc.