Announcements
May 2nd Announcements
- Last day of class!
- Next fall: Optimization!
- Project reports due 5/7 (Start Writing!)
April 30th announcements
- Online Quiz 3
- Final Projects
- FCQs due tonight
April 25th Announcements
- FCQs open until 4/30
- Final Project due 5/7
- Online Quiz due next week
April 16th Announcements
- HW 5 Leaderboard
- Class Poll
- Exam: Thursday-Saturday
March 12 Announcement
- Quiz due Tomorrow (Question)
- HW5 Due Next Friday (autograder still needs to be released)
March 5th Announcements
- Exam 2 -> Weekly Quizzes
- Exam 1 graded
Feb 27
- HW 3 Completed
- Project Ideas, due Feb 29
- HW 4 Released, due March 7
- Colorado Primary, March 5th 🇺🇸
- Vote at the UMC
- If you're unaffiliated, you can vote in either party's primary
Feb 15
- Exam today released at 1 today, due 11:59 pm tomorrow
- 120 minutes
- Four problems
- Four pages with room to write on (not required)
- Office Hours 6-7, private questions on piazza
- ChatGPT countermeasures: Images - not text, some consistent mistakes
Feb 13, 2024
- Leaderboard
- Exam Thursday and Friday
- Homework 3
- Review Session
Feb 1st, 2024
- HW 1 Leaderboard, Solution
- No Class next Tuesday
January 30th 2024
- HW2 due NEXT Friday at 5pm
Jan 25, 2024
- HW 1 due Friday
- Office Hours 1-2
Jan 18 2024
- HW 1 Released (will demo at end)
- HW Challenge Problems
- OH 2-3 today (AERO 263)
May 2nd 2023
- Project: Start Writing!
- FCQs
- Programming assignment
ASEN 6519-001: Advanced Survey of Sequential Decision Making (Fall 2021)
- Filet-mignon level understanding!
- Exploring the latest literature
- No homework! Final project, presentations, conceptual review quizzes
- Presentations: how to communicate about research
"I consider this class among the most productive and informative I've taken while at CU." - 2021 student
AKA DMU++
April 27, 2023
- Exam 3
- Final Project
- Seminar: Friday 2:45 AERO 114
Today: Overview of related topics
- Imitation Learning
- Inverse Reinforcement Learning (IRL)
- Transfer Learning
- Meta Learning
April 4, 2023
- Exam 2: Thursday-Friday
- HW 5 leaderboard
- HW6 Assigned
March 14, 2023
- Exam 1 scores
- Tips for getting RL to work
- Homework 5
- Python for HW 5
Happy policy day!
March 9, 2023
- Schedule
- Writing large programs in Julia
March 7, 2023
- Survey (required by tonight)
- HW 4 due Thursday
March 2nd, 2023
- Survey
- Leaderboard
Title Text
- How MCTS is used in a simulation
- Policy Gradient Figure
Feb 16, 2023
- Exam 1, Monday-Tuesday
- HW2, Problem 3
February 14th, 2023
- HW 2 Leaderboard
- HW 3 Released
- HW 1 Grades Posted
- Exam 1 Next Tuesday
Feb 9th 2023
- Leaderboard
- HW2 Extended
- HW1 solutions posted
- Extra Office Hours
- dot operator
Feb 2nd, 2023
- Leaderboard
- You can submit as many times as you want
- Jackson Office Hours
Jan 26th 2023
- HW 1 due tonight
- HW 2 released tomorrow
Jan 24th 2020
- Office hours changed from 4:30-5:30 Tuesdays
- Loose ends from last lecture
- ChatGPT
- Remote study groups
Jan 19 2023
- Julia 1.8
- Back to problem formulation
- Jackson
- Schedule
- Slides
Feb 1, 2022
- HW2 due this evening
- Quiz 1 next week
- HW3 released this week
3 Bellman/Dynamic Programming Equations
February 3rd, 2022
- HW2 Leaderboard
- Quiz 1, next Wednesday and Thursday
- HW 3
Breakout Rooms
1. Grid World
Feb 8th, 2022
- HW3 Released, due Wednesday
- Quiz 1 Tomorrow at Noon to Thursday
Feb 10th
- Quiz 1 finished by tonight at midnight (10 submissions so far)
- HW 3 Due Feb 17th at 5pm
Feb 10
HW Due Thursday
Feb 17
- HW3 Due Saturday at 5 pm
- HW4 Released Soon
Discuss
loop
\(\tau \gets \text{simulate}(\pi_\theta)\)
\(\theta \gets \theta + \alpha \sum_{k=0}^d \nabla_\theta \log \pi_\theta(a_k \mid s_k) \gamma^k r_{k, \text{to go}} \)
\[\pi_\theta (a=L \mid s) = \frac{\theta_1}{\theta_1 + \theta_2}\]
\[\pi_\theta (a=R \mid s) = \frac{\theta_2}{\theta_1 + \theta_2}\]
- 1. Given \(\theta = (0.2, 0.8)\) calculate \(\sum_{k=0}^d \nabla_\theta \log \pi_\theta(a_k \mid s_k) \gamma^k r_{k, \text{to go}} \) for two cases, (a) where \(a_0 = R\) and (b) where \(a_0 = L\)
- What happens if \(\theta_1 \to 0\)
Feb 22
- HW 3 Leaderboard
- HW 4
- Quiz almost graded
Feb 24
- Look at Schedule
- Project Proposal Assignment
- HW 4 Due Tuesday
- HW 5: Get started early!
Perspective
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin." - Richard Sutton
The Bitter Lesson
March 3rd
- Quiz 1 solutions released
- Schedule: Project Proposals, HW 5
March 15
- HW5 Due Today
- HW6 Out this week
- Quiz 2 (RL and POMDPs): March 29th - study materials out this week
Alpha Vector Recap
- How many states
- X and Y axis
- Value Function
- Each alpha vector corresponds to a conditional plan
- Can multiple alpha vectors correspond to the same action?
- How to execute a policy
March 29
- HW6: Due 4/7
- Quiz 2: tomorrow noon - Thursday midnight
April 18
- HW6 Leaderboard Results
- Quiz 3
- Project
The American diplomat George Kennan, one of the architects of that order, once wrote that a healthy American foreign policy should create “the impression of a country which knows what it wants.” Yet it is just as important to know what one’s adversaries want. American adventures in East Asia, particularly, are notable for their long history of governments talking past one another.
Game Theory Quotes
"The single best piece of advice I ever got on marriage is that there is no use in thinking of your partner as a single stable entity that exists separate from you. There’s only your partner in a dynamic with you." - Ezra Klein
April 26, 2022
- FCQs due tonight!
- Final Project due May 3rd
- Watch AlphaGo Documentary
Quizzes (grades in general) are for revealing how well you understand something
000-Announcements
By Zachary Sunberg
