## MCMC with mutually singular distributions

**A** paper that recently appeared in the latest issue of ** JCGS** deals with this important issue of using mutually singular proposals, written by Raphael Gottardo and Adrian Raftery. I personally find the problem quite interesting because it covers major applications like Bayesian model choice and Bayesian nonparametric density estimation. The beginning of the paper recalls some basics about computing densities for measures that are mixtures wrt mutually exclusive measures. (This may sound obvious to many, but there is this important remark that, for measures involving Lebesgue and Dirac components at , the density of the Lebesgue part must be zero at all ‘s.) Once the density of the target is clearly defined, Gottardo and Raftery move to define a general Metropolis-Hastings algorithm. This sets the correct setup for running samplers across models, as in hypothesis testing or variable selection.

**N**ow, I find it rather restrictive that in the paper ** both** the target and the proposal are defined against the

**dominating measure, while an almost standard MCMC algorithm (like Gibbs) would move only some components and thus be defined against another dominating measure. Maybe I did miss a major point but the paper seems to imply that Gibbs sampling works within this framework only if one defines the target as the posterior plus conditionals (?). Anyway, I would have thought that the proper setting would require both product measures and to be absolutely continuous wrt one another, rather than having to impose to all four components to be defined against the same dominating measure. Not that there is anything wrong in the current paper, but this choice seems to be limiting the appeal of the extension… (Note that the complexities of product measures on those variable dimension spaces are what prompted Peter Green to introduce reversible jump in 1995 in terms of a dominating symmetric measure.)**

*same*

April 15, 2009 at 12:27 pm

The topic is quite hot on the Internet at the moment. What do you pay the most attention to when choosing what to write ?

March 31, 2009 at 6:11 am

[…] with mutually singular distributions (2) In connection with questions I had posted earlier about the interesting paper of Raphael Gottardo and Adrian Raftery, I got several emails […]

March 30, 2009 at 10:27 pm

Salut Xian,

I told you I was going to post something so here it is, and sorry for the delay. As I said to you via email, you are not restricted to a proposal that is dominated by $\nu$. You could combine several moves with different proposals, not all dominated by $\nu$. This said, at least one of your proposals must be dominated by $\nu$ for the chain to be irreducible. You need to make sure that you have a positive prob. to visit all of the singular components (e.g. models). This is very similar to the within and between model moves in RJ-MCMC. In a sense, the between model proposal needs to be dominated by $\nu$. This is illustrated in the paper in several examples. Though, I agree that it may not have been terribly clear. For example, if you look at the paper, we have what we call the component-wise Gibbs, and the Gibbs. The component-wise Gibbs is a move that is not dominated by $\nu$ whereas the Gibbs is.

Anyway, I’d be happy to post more to clarify some more things if needed. I enjoy your blog, so please keep it up. I have added a link in the ISBA bulletin.

Cheers,

Raphael