Bayesian Network Learning

  • Last time:

    • Conditional independence in Bayesian Networks
    • Sampling from Bayesian Networks

  • Today:

    • Given a Bayesian Network and some values, how do we calculate the probability of other values?
    • Given data, how do we fit a Bayesian network?

Bayesian Network

Structure

Parameters

Inference

Inputs

  • Bayesian network structure
  • Bayesian network parameters
  • Values of evidence variables

Outputs

  • Posterior distribution of query variables

Given that you have detected a trajectory deviation, and the battery has not failed what is the probability of a solar panel failure?

\(P(S=1 \mid D=1, B=0)\)

Exact

Approximate

Exact Inference

Exact Inference

\[P(S{=}1 \mid D{=}1, B{=}0)\]

\[= \frac{P(S{=}1, D{=}1, B{=}0)}{P(D{=}1, B{=}0)}\]

\[P(S{=}1, D{=}1, B{=}0) = \sum_{e, c}P(B{=}0, S{=}1, E{=}e, D{=}1, C{=}c)\]

\[P(B{=}0, S{=}1, E, D{=}1, C)\]

\[= P(B{=}0)\,P(S{=}1)\,P(E\mid B{=}0, S{=}1)\,P(D{=}1\mid E)\,P(C{=}1\mid E)\]

\(2^5= 32\) possible assignments, but quickly gets too large

Exact Inference

\(2^5= 32\) possible assignments, but quickly gets too large

Product

Condition

Marginalize

Exact Inference: Variable Elimination

Start with

Eliminate \(D\) and \(C\) (evidence) to get \(\phi_6(E)\) and \(\phi_7(E)\)

Eliminate \(E\)

Eliminate \(S\)

vs

Choosing optimal order is NP-hard

Approximate Inference

Approximate Inference: Direct Sampling

Analogous to

unweighted particle filtering

Approximate Inference: Weighted Sampling

Analogous to

weighted particle filtering

Approximate Inference: Gibbs Sampling

Markov Chain Monte Carlo (MCMC)

Learning

Bayesian Network Learning

Inputs

  • Data, \(D\)
  • Priors (?)

Outputs

  • Bayesian network structure, \(G\)
  • Bayesian network parameters, \(\theta\)

Counting Parameters

For discrete R.V.s:

\[\text{dim}(\theta_X) = \left(|\text{support}(X)|-1\right)\prod_{Y \in Pa(X)} |\text{support}(Y)|\]

Structure Learning Example

Parameter Learning

Maximum Likelihood

Bayesian

Multinomial:

Multinomial:

Structure Learning

NP-Hard

Markov Equivalence Class

Markov Equivalent iff

  1. Same undirected edges
  2. Same set of immoral v-structures

Recap

Inference

Learning