April 25th Announcements
"I consider this class among the most productive and informative I've taken while at CU." - 2021 student
AKA DMU++
Happy policy day!
3 Bellman/Dynamic Programming Equations
1. Grid World
HW Due Thursday
loop
\(\tau \gets \text{simulate}(\pi_\theta)\)
\(\theta \gets \theta + \alpha \sum_{k=0}^d \nabla_\theta \log \pi_\theta(a_k \mid s_k) \gamma^k r_{k, \text{to go}} \)
\[\pi_\theta (a=L \mid s) = \frac{\theta_1}{\theta_1 + \theta_2}\]
\[\pi_\theta (a=R \mid s) = \frac{\theta_2}{\theta_1 + \theta_2}\]
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin." - Richard Sutton
The American diplomat George Kennan, one of the architects of that order, once wrote that a healthy American foreign policy should create “the impression of a country which knows what it wants.” Yet it is just as important to know what one’s adversaries want. American adventures in East Asia, particularly, are notable for their long history of governments talking past one another.
"The single best piece of advice I ever got on marriage is that there is no use in thinking of your partner as a single stable entity that exists separate from you. There’s only your partner in a dynamic with you." - Ezra Klein