Crame'r-Rao Bounds and Monte Carlo Calculation of the Fisher Information Matrix in Difficult Problems
James C. Spall, (Johns Hopkins Univ., APL), james.spall@jhuapl.edu
Abstract
The Fisher information matrix summarizes the amount of information in the data relative to the quantities of interest. There are many applications of the information matrix in modeling, systems analysis, and estimation, including confidence region calculation, input design, prediction bounds, and "noninformative" priors for Bayesian analysis. This paper reviews some basic principles associated with the information matrix, presents a resampling-based method for computing the information matrix together with some new theory related to efficient implementation, and presents some numerical results. The resampling-based method relies on an efficient technique for estimating the Hessian matrix, introduced as part of the adaptive ("second-order") form of the simultaneous perturbation stochastic approximation (SPSA) optimization algorithm. Key words: Monte Carlo simulation; Crame'r-Rao bound; simultaneous perturbation; antithetic random numbers.