Probability and Random Variables
Concepts
- Utility and Probability
- Random Variables
- Relationships between Random Variables
Utility and Probability
What is a Random Variable?
R.V. \(X\)
Vocabulary/Notation
| Term | Definition | Coinflip Example | Uniform Example |
|---|
\(\text{Bernoulli}(0.5)\)
\(\mathcal{U}(0,1)\)
support(\(X\))
All the values that \(X\) can take
\(\{h, t\}\) or \(\{0,1\}\)
\([0,1]\)
\(x \in X\)
\(X \in [0,1]\)
Distribution
Maps each value in the support to a real number indicating its probability
\(P(X=1) = 0.5\)
\(P(X=0) = 0.5\)
\(P(X)\) is a table
| X | P(X) |
|---|---|
| 0 | 0.5 |
| 1 | 0.5 |
- Discrete: PMF
- Continuous: PDF
\(p(x) = \begin{cases} 1 \text{ if } x \in [0, 1] \\ 0 \text{ o.w.} \end{cases}\)
\(P(X=1) = \)
\(P(X \in [a, b]) = \int_a^b p(x) dx\)
\(0\)
Expectation
First moment of the random variable, "mean"
\(E[X]\)
\[E[X] = \sum_{x \in X} x P(x)\]
\(=0.5\)
\[E[X] = \int_{x \in X} x p(x) dx\]
\(=0.5\)
Distributions of related R.V.s
Joint Distribution
Conditional Distribution
Marginal Distribution
\(P(X,Y,Z)\)
\(P(X \mid Y,Z)\)
\(P(X)\) \(P(Y)\) \(P(Z)\)
(Distribution - valued function)
| X | P(X|Y=1, Z=1) |
| 0 | 0.84 |
| 1 | 0.16 |
Distributions of related R.V.s
Joint Distribution
Conditional Distribution
Marginal Distribution
\(P(X,Y,Z)\)
\(P(X \mid Y,Z)\)
\(P(X)\) \(P(Y)\) \(P(Z)\)
3 Rules
(Burrito-level)
(Blackbelly Level: Axioms of Probability)
1)
a) \(0 \leq P(X \mid Y) \leq 1\)
b) \(\sum_{x \in X} P(x \mid Y) = 1\)
2) "Law of total probability"
\[P(X) = \sum_{y \in Y} P(X, y)\]
3) Definition of Conditional Probability
\[P(X \mid Y) = \frac{P(X, Y)}{P(Y)}\]
Joint \(\rightarrow\) Marginal
Joint + Marginal \(\rightarrow\) Conditional
Marginal + Conditional \(\rightarrow\) Joint
\[P(X, Y) = P(X | Y) \, P(Y)\]
not on exam
Distributions of related R.V.s
Joint Distribution
Conditional Distribution
Marginal Distribution
\(P(X,Y,Z)\)
\(P(X \mid Y,Z)\)
\(P(X)\) \(P(Y)\) \(P(Z)\)
3 Rules
1)
a) \(0 \leq P(X \mid Y) \leq 1\)
b) \(\sum_{x \in X} P(x \mid Y) = 1\)
2) "Law of total probability"
\[P(X) = \sum_{y \in Y} P(X, y)\]
3) Definition of Conditional Probability
\[P(X \mid Y) = \frac{P(X, Y)}{P(Y)}\]
Joint \(\rightarrow\) Marginal
Joint + Marginal \(\rightarrow\) Conditional
Marginal + Conditional \(\rightarrow\) Joint
\[P(X, Y) = P(X | Y) \, P(Y)\]
Break
- \(P \in \{0,1\}\): Powder Day
- \(C \in \{0,1\}\): Pass Clear
- 1 in 5 days is a powder day
- The pass is clear 8 in 10 days
- If it is a powder day, there is a 50% chance the pass is blocked
- Write out the joint probability distribution for P and C.
- Suppose it is a non-powder day, what is the probability that the pass is blocked?
1)
a) \(0 \leq P(X \mid Y) \leq 1\)
b) \(\sum_{x \in X} P(x \mid Y) = 1\)
2) \(P(X) = \sum_{y \in Y} P(X, y)\)
3) \(P(X \mid Y) = \frac{P(X, Y)}{P(Y)}\)
\(P(X, Y) = P(X | Y) \, P(Y)\)
Bayes Rule
- Know: \(P(B \mid A)\), \(P(A)\), \(P(B)\)
- Want: \(P(A \mid B)\)
Independence
Definition: \(X\) and \(Y\) are independent iff \(P(X, Y) = P(X)\, P(Y)\)
Definition: \(X\) and \(Y\) are conditionally independent given \(Z\) iff \(P(X, Y \mid Z) = P(X \mid Z)\, P(Y \mid Z)\)
Rules for Continuous RVs
1)
a) \(0 \leq P(X \mid Y) \leq 1\)
b) \(\sum_{x \in X} P(x \mid Y) = 1\)
2) \(P(X) = \sum_{y \in Y} P(X, y)\)
3) \(P(X \mid Y) = \frac{P(X, Y)}{P(Y)}\)
\(P(X, Y) = P(X \mid Y) \, P(Y)\)
1)
Discrete
Continuous
\[\int_X p(x | Y) \, dx = 1\]
2) \[p(X) = \int_{Y} p(X, y) dy\]
3) \(p(X \mid Y) = \frac{p(X, Y)}{p(Y)}\)
\(p(X, Y) = p(X \mid Y) \, p(Y)\)
\(0 \leq p(X \mid Y)\)
Multivariate Gaussian Distribution
Joint Distribution
Conditional Distribution
Marginal Distribution
Concepts
- Utility and Probability
- Random Variables
- Relationships between Random Variables