Announcements
April 7, 2026
- HW 5 Leaderboard
- Exam 1: Next Tuesday (Remote, please fill out proctor form)
- HW 6: 2 weeks from today
- Final Project: 4/30
March 31, 2026
- Feedback on proposals coming
- HW5 Due Thursday
- Exam Topics
March 24th 2026
- Project Proposals due Thursday
- HW5 due next Thursday
Exam 1
For Exam 2:
- Compared to a typical engineering class, DMU involves more abstract mathematical objects (Conditional Distributions, MDPs vs. concrete equations) -> More "what does this mean" questions.
- While I intend to reduce time pressure on Exam 2, it will still be there. -> practice real problems.
- Instead of all notes, I plan to allow a reduced amount (~3 pages) to help you know where everything is.
March 12, 2026
- Project Proposals Due March 24th
- HW5 Due March 31st
Feb 24 2026
- Thursday is Reading Day
- Project Ideas due Tonight
- HW 4 Released
- Exam 1 Next Tuesday
Feb 19, 2026
- Decisions.jl and other project ideas
- All-time records
Feb 17 Announcements
- Project Ideas Assignment Released
- HW3 due tonight
- Clarified Question 2
Feb 12 Announcements
- Please submit pdfs and plain code files. Please DO NOT submit unrendered markdown or notebooks
- HW 1 Grades and solutions released
- Post on Ed if you're looking for groups
- Added additional MCTS pseudocode
- HW3
- Minor update to HW3
Feb 10, 2026
Jan 27, 2026
- HW 2 due next Tuesday
- Exam: anything not electronic
- Tyler and Xavier have posted office hours
Jan 22, 2026
- HW 2 (similar, but not identical to 2025 HW 2) will be released soon.
- HW 1 Leaderboard
January 20th
- Homework 1: Due tonight
- Homework 2: Released tomorrow
- Xavier: Office Hours
Jan 20, 2026
- Make sure autograder runs
Jan 13, 2026
HW 1 Released!
April 29
- Final Projects Due 5/5 (24 hour late period - 10% off)
April 22
- Thursday: Guest Lecture by Eric Brown (CU Philosophy) on Ethics for AI
- Final Projects
- Exam 3 Next Monday and Tuesday
- FCQs open now until 4/29
- 2 Purposes: Used for evaluating faculty, improving teaching and courses
- Constructive feedback is very helpful! What helped you learn the most?
- 2 Purposes: Used for evaluating faculty, improving teaching and courses
April 17
- HW6 Leaderboard
- Final Project
April 15
- HW6 Due 4/16
- Project Feedback
April 8
- HW6 Due 4/14
April 3
- HW5 Leaderboard
- HW6 Due 4/14
- Project proposals due Friday
Mar 20
- HW5 Due Today
- Project proposals due Friday
Mar 20
- HW5 Due Tuesday after Spring Break (new starter code)
- Project proposals due Thursday after spring break
- Feedback on Project Ideas
- If there are multiple decision-makers with multiple reward functions, it is not an MDP or a POMDP, it is a Markov Game or a POMG/POSG
Mar 18
- W5 Due Tuesday after Spring Break (new starter code)
- Project proposals due Thursday after spring break
- If you have Julia syntax/concept questions, I can answer them during this class
Mar 11
- Exam 2 Thursday and Friday
- HW5 Released
Feb 25th
- Project Ideas due Friday
- HW4 Released
- HW3 Leaderboard
Feb 20th
- HW3 Due Monday
- evaluate(..., time=true) for score > 50
- Project Ideas due next Thursday
- 3 Ideas
- Focus on MDPs, POMDPs, Games
- POMDP = Partially Observable MDP where you only get a noisy observation of the state
- Game - like an MDP/POMDP, but there are multiple decision makers, each with their own reward function
Feb 11th
- Note numbers posted on schedule
- HW 3 Released
- HW 2 Leaderboard
- Exam 1 Thursday and Friday
Feb 6th
- HW2 Due Next Monday
- Exam 1 Next Thursday and Friday
- Will release study materials by tomorrow morning
Jan 28
- HW2 released - due next week
- Office Hours tonight
- HW1 Leaderboard
Jan 21
- HW 1 due Friday
- TA Office Hours released
- Remote Office Hours Poll
Jan 16th
- HW1 released (will talk about autograded problem this afternoon or Tuesday)
May 2nd Announcements
- Last day of class!
- Next fall: Optimization!
- Project reports due 5/7 (Start Writing!)
April 30th announcements
- Online Quiz 3
- Final Projects
- FCQs due tonight
April 25th Announcements
- FCQs open until 4/30
- Final Project due 5/7
- Online Quiz due next week
April 16th Announcements
- HW 5 Leaderboard
- Class Poll
- Exam: Thursday-Saturday
March 12 Announcement
- Quiz due Tomorrow (Question)
- HW5 Due Next Friday (autograder still needs to be released)
March 5th Announcements
- Exam 2 -> Weekly Quizzes
- Exam 1 graded
Feb 27
- HW 3 Completed
- Project Ideas, due Feb 29
- HW 4 Released, due March 7
- Colorado Primary, March 5th 🇺🇸
- Vote at the UMC
- If you're unaffiliated, you can vote in either party's primary
Feb 15
- Exam today released at 1 today, due 11:59 pm tomorrow
- 120 minutes
- Four problems
- Four pages with room to write on (not required)
- Office Hours 6-7, private questions on piazza
- ChatGPT countermeasures: Images - not text, some consistent mistakes
Feb 13, 2024
- Leaderboard
- Exam Thursday and Friday
- Homework 3
- Review Session
Feb 1st, 2024
- HW 1 Leaderboard, Solution
- No Class next Tuesday
January 30th 2024
- HW2 due NEXT Friday at 5pm
Jan 25, 2024
- HW 1 due Friday
- Office Hours 1-2
Jan 18 2024
- HW 1 Released (will demo at end)
- HW Challenge Problems
- OH 2-3 today (AERO 263)
May 2nd 2023
- Project: Start Writing!
- FCQs
- Programming assignment
ASEN 6519-001: Advanced Survey of Sequential Decision Making (Fall 2021)
- Filet-mignon level understanding!
- Exploring the latest literature
- No homework! Final project, presentations, conceptual review quizzes
- Presentations: how to communicate about research
"I consider this class among the most productive and informative I've taken while at CU." - 2021 student
AKA DMU++
April 27, 2023
- Exam 3
- Final Project
- Seminar: Friday 2:45 AERO 114
Today: Overview of related topics
- Imitation Learning
- Inverse Reinforcement Learning (IRL)
- Transfer Learning
- Meta Learning
April 4, 2023
- Exam 2: Thursday-Friday
- HW 5 leaderboard
- HW6 Assigned
March 14, 2023
- Exam 1 scores
- Tips for getting RL to work
- Homework 5
- Python for HW 5
Happy policy day!
March 9, 2023
- Schedule
- Writing large programs in Julia
March 7, 2023
- Survey (required by tonight)
- HW 4 due Thursday
March 2nd, 2023
- Survey
- Leaderboard
Title Text
- How MCTS is used in a simulation
- Policy Gradient Figure
Feb 16, 2023
- Exam 1, Monday-Tuesday
- HW2, Problem 3
February 14th, 2023
- HW 2 Leaderboard
- HW 3 Released
- HW 1 Grades Posted
- Exam 1 Next Tuesday
Feb 9th 2023
- Leaderboard
- HW2 Extended
- HW1 solutions posted
- Extra Office Hours
- dot operator
Feb 2nd, 2023
- Leaderboard
- You can submit as many times as you want
- Jackson Office Hours
Jan 26th 2023
- HW 1 due tonight
- HW 2 released tomorrow
Jan 24th 2020
- Office hours changed from 4:30-5:30 Tuesdays
- Loose ends from last lecture
- ChatGPT
- Remote study groups
Jan 19 2023
- Julia 1.8
- Back to problem formulation
- Jackson
- Schedule
- Slides
Feb 1, 2022
- HW2 due this evening
- Quiz 1 next week
- HW3 released this week
3 Bellman/Dynamic Programming Equations
February 3rd, 2022
- HW2 Leaderboard
- Quiz 1, next Wednesday and Thursday
- HW 3
Breakout Rooms
1. Grid World
Feb 8th, 2022
- HW3 Released, due Wednesday
- Quiz 1 Tomorrow at Noon to Thursday
Feb 10th
- Quiz 1 finished by tonight at midnight (10 submissions so far)
- HW 3 Due Feb 17th at 5pm
Feb 10
HW Due Thursday
Feb 17
- HW3 Due Saturday at 5 pm
- HW4 Released Soon
Discuss
loop
\(\tau \gets \text{simulate}(\pi_\theta)\)
\(\theta \gets \theta + \alpha \sum_{k=0}^d \nabla_\theta \log \pi_\theta(a_k \mid s_k) \gamma^k r_{k, \text{to go}} \)
\[\pi_\theta (a=L \mid s) = \frac{\theta_1}{\theta_1 + \theta_2}\]
\[\pi_\theta (a=R \mid s) = \frac{\theta_2}{\theta_1 + \theta_2}\]
- 1. Given \(\theta = (0.2, 0.8)\) calculate \(\sum_{k=0}^d \nabla_\theta \log \pi_\theta(a_k \mid s_k) \gamma^k r_{k, \text{to go}} \) for two cases, (a) where \(a_0 = R\) and (b) where \(a_0 = L\)
- What happens if \(\theta_1 \to 0\)
Feb 22
- HW 3 Leaderboard
- HW 4
- Quiz almost graded
Feb 24
- Look at Schedule
- Project Proposal Assignment
- HW 4 Due Tuesday
- HW 5: Get started early!
Perspective
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin." - Richard Sutton
The Bitter Lesson
March 3rd
- Quiz 1 solutions released
- Schedule: Project Proposals, HW 5
March 15
- HW5 Due Today
- HW6 Out this week
- Quiz 2 (RL and POMDPs): March 29th - study materials out this week
Alpha Vector Recap
- How many states
- X and Y axis
- Value Function
- Each alpha vector corresponds to a conditional plan
- Can multiple alpha vectors correspond to the same action?
- How to execute a policy
March 29
- HW6: Due 4/7
- Quiz 2: tomorrow noon - Thursday midnight
April 18
- HW6 Leaderboard Results
- Quiz 3
- Project
The American diplomat George Kennan, one of the architects of that order, once wrote that a healthy American foreign policy should create “the impression of a country which knows what it wants.” Yet it is just as important to know what one’s adversaries want. American adventures in East Asia, particularly, are notable for their long history of governments talking past one another.
Game Theory Quotes
"The single best piece of advice I ever got on marriage is that there is no use in thinking of your partner as a single stable entity that exists separate from you. There’s only your partner in a dynamic with you." - Ezra Klein
April 26, 2022
- FCQs due tonight!
- Final Project due May 3rd
- Watch AlphaGo Documentary
Quizzes (grades in general) are for revealing how well you understand something
000-Announcements
By Zachary Sunberg
