# Bayesian Network Learning

• ### Last time:

• Conditional independence in Bayesian Networks
• Sampling from Bayesian Networks
• ### Today:

• Given a Bayesian Network and some values, how do we calculate the probability of other values?
• Given data, how do we fit a Bayesian network?

Structure

Parameters

## Inference

Inputs

• Bayesian network structure
• Bayesian network parameters
• Values of evidence variables

Outputs

• Posterior distribution of query variables

Given that you have detected a trajectory deviation, and the battery has not failed what is the probability of a solar panel failure?

$$P(S=1 \mid D=1, B=0)$$

Exact

Approximate

# Exact Inference

## Exact Inference

$P(S{=}1 \mid D{=}1, B{=}0)$

$= \frac{P(S{=}1, D{=}1, B{=}0)}{P(D{=}1, B{=}0)}$

$P(S{=}1, D{=}1, B{=}0) = \sum_{e, c}P(B{=}0, S{=}1, E{=}e, D{=}1, C{=}c)$

$P(B{=}0, S{=}1, E, D{=}1, C)$

$= P(B{=}0)\,P(S{=}1)\,P(E\mid B{=}0, S{=}1)\,P(D{=}1\mid E)\,P(C{=}1\mid E)$

$$2^5= 32$$ possible assignments, but quickly gets too large

## Exact Inference

$$2^5= 32$$ possible assignments, but quickly gets too large

Product

Condition

Marginalize

## Exact Inference: Variable Elimination

Eliminate $$D$$ and $$C$$ (evidence) to get $$\phi_6(E)$$ and $$\phi_7(E)$$

Eliminate $$E$$

Eliminate $$S$$

vs

Choosing optimal order is NP-hard

# Approximate Inference

## Approximate Inference: Direct Sampling

Analogous to

unweighted particle filtering

## Approximate Inference: Weighted Sampling

Analogous to

weighted particle filtering

## Approximate Inference: Gibbs Sampling

Markov Chain Monte Carlo (MCMC)

# Learning

## Bayesian Network Learning

Inputs

• Data, $$D$$
• Priors (?)

Outputs

• Bayesian network structure, $$G$$
• Bayesian network parameters, $$\theta$$

## Counting Parameters

For discrete R.V.s:

$\text{dim}(\theta_X) = \left(|\text{support}(X)|-1\right)\prod_{Y \in Pa(X)} |\text{support}(Y)|$

## Parameter Learning

Maximum Likelihood

Bayesian

Multinomial:

Multinomial:

NP-Hard

## Markov Equivalence Class

Markov Equivalent iff

1. Same undirected edges
2. Same set of immoral v-structures

## Recap

Inference

Learning

#### 230 Bayesian Network Learning

By Zachary Sunberg

• 57