Advanced Statistical ComputingFall 2012Lecture 4.ppt

Advanced Statistical ComputingFall 2012Lecture 4 Steve Qin Collapsing and grouping Want to sample from Regular Gibbs sampler: Sample x1(t+1) from Sample x2(t+1) from … Sample xd(t+1) from Alternatively: Grouping: Collapsing, i.e., integrate out xd: * The three-schemes * standard grouping collapsing Some theory Hilbert space L2(π) of functions h(). Define thus Define forward operator F as The convergence of Markov chains is tied to the norms of the corresponding forward operators. * Three-scheme theorem Standard Fs: Grouping Fg: Collapsing Fc: Theorem The norms of the three forward operators are ordered as * Examples Murray’s data Bivariate Gaussian with mean 0 and unknown covariance matrix Σ standard collapsing * Remarks Avoid introducing unnecessary parameters into a Gibbs sampler, Do as much analytical work as possible, However, introducing some clever auxiliary variables can greatly improve computation efficiency. * Sequential Monte Carlo We wish to evaluate an integral assume h(x) ≥ 0. Riemann sum (on grid points) as approximation. Alternatively, use Monte Carlo. Select random samples uniformly on its support. * An example Both grid-point method and vanilla Monte Carlo methods wasted resources on “boring” desert area. * The basic idea Marshall (1956) suggested that one should focus on the region(s) of “importance” so as to save computational resources—importance sampling. Essential in high-dimensional models. * The algorithm To evaluate Draw from a trial distribution g(). Calculate the importance weight Approximate μ by Remark: is better than the unbiased estimator why? * An example (cont.) Use proposal function with (x,y) ? [?1,1] x [?1,1], a truncated mixture of bivariate Gaussian Vanilla Monte Carlo Importance Sampling * Sequential importance sampling For high dimensional problem, how to design trial distribution is challenging. Suppose the target density of can