Hedging Against Misspecification

Which actions reveal the most information about a hidden variable? Can we trust those actions when our measure of “informativeness” itself depends on imperfect beliefs? To explore these questions, I aim to provide an informal introduction to Bayesian experimental design (BED) and its recent evolution under deep learning. This draws partly on research I conducted for my MSc dissertation, but I hope to broaden the discussion with connections to world models. Code for the experiments is available in the GitHub repository.

Bayesian experimental design

At its core, BED offers principled criteria for choosing actions or experimental designs that maximally reduce our uncertainty (in expectation) about a fixed but unknown parameter \(\theta^*\). Since \(\theta^*\) is unknown, we introduce a random variable \(\theta\) in its place and encode our beliefs about its value in a prior \(p(\theta)\). The experiment is assumed to generate an observable outcome \(y\) that depends on \(\theta^*\) and our chosen design \(\xi\) according to the likelihood \(p(y \mid \theta^*, \xi)\). We can then use this outcome to update our beliefs about \(\theta^*\) via Bayes’ rule:

\[\begin{aligned} p(\theta \mid y, \xi) &= \frac{p(y \mid \theta, \xi)\, p(\theta)}{p(y \mid \xi)}, \\[4pt] \text{where} \quad p(y \mid \xi) &:= \int p(y \mid \theta, \xi)\, p(\theta)\, \mathrm{d}\theta. \end{aligned}\]

In other words, the posterior upweights values of \(\theta\) under which the observed outcome \(y\) is more likely. Assuming the likelihood is well-specified and \(\theta^*\) lies within the support of our prior, the posterior tends to concentrate around the true value \(\theta^*\) as observations accumulate (van der Vaart, 1998). The interesting question is then: which designs \(\xi\) lead to the greatest concentration of our beliefs around the truth—that is, which are most informative?

Expected information gain

TBC…