GEOF H. GIVENS, PhD, is Associate Professor in the Department of Statistics at Colorado State University. He serves as Associate Editor for Computational Statistics and Data Analysis. His research interests include statistical problems in wildlife conservation biology including ecology, population modeling and management, and automated computer face recognition.

JENNIFER A. HOETING, PhD, is Professor in the Department of Statistics at Colorado State University. She is an award-winning teacher who co-leads large research efforts for the National Science Foundation. She has served as associate editor for the Journal of the American Statistical Association and Environmetrics. Her research interests include spatial statistics, Bayesian methods, and model selection.

Givens and Hoeting have taught graduate courses on computational statistics for nearly twenty years, and short courses to leading statisticians and scientists around the world.

Preface xv

Acknowledgments xix

1 Review 1

1.1 Mathematical notation 1

1.2 Taylor’s theorem and mathematical limit theory 2

1.3 Statistical notation and probability distributions 4

1.4 Likelihood inference 6

1.5 Bayesian inference 11

1.6 Statistical limit theory 13

1.7 Markov chains 14

1.8 Computing 17

PART I OPTIMIZATION

2 Optimization and Solving Nonlinear Equations 3

2.1 Univariate problems 5

2.2 Multivariate problems 17

Problems 36

3 Combinatorial Optimization 43

3.1 Hard problems and NP-completeness 44

3.2 Local search 50

3.3 Simulated annealing 53

3.4 Genetic algorithms 60

3.5 Tabu algorithms 71

Problems 78

4 EM Optimization Methods 83

4.1 Missing data, marginalization, and notation 84

4.2 The EM algorithm 84

4.3 EM Variants 98

Problems 108

PART II INTEGRATION AND SIMULATION

5 Numerical Integration 117

5.1 Newton-Côtes quadrature 118

5.2 Romberg integration 127

5.3 Gaussian quadrature 131

5.4 Frequently encountered problems 135

Problems 137

6 Simulation and Monte Carlo Integration 139

6.1 Introduction to the Monte Carlo method 140

6.3 Approximate Simulation 152

6.4 Variance reduction techniques 170

Problems 185

7 Markov Chain Monte Carlo 191

7.1 Metropolis-Hastings algorithm 192

7.2 Gibbs sampling 199

7.3 Implementation 210

Problems 222

8 Advanced Topics in MCMC 229

8.1 Adaptive MCMC 229

8.2 Reversible Jump MCMC 243

8.3 Auxiliary variable methods 250

8.4 Other Metropolis Hastings Algorithms 254

8.5 Perfect sampling 258

8.6 Markov chain maximum likelihood 262

8.7 Example: MCMC for Markov random fields 263

Problems 274

PART III APPROXIMATING DISTRIBUTIONS

9 Bootstrapping 281

9.1 The bootstrap principle 281

9.2 Basic methods 283

9.3 Bootstrap inference 286

9.4 Reducing Monte Carlo error 297

9.5 Bootstrapping dependent data 298

9.6 Bootstrap performance 310

9.7 Other uses of the bootstrap 312

9.8 Permutation tests 313

Problems 314

PART IV DENSITY ESTIMATION AND SMOOTHING

10 Nonparametric Density Estimation 321

10.1 Measures of performance 322

10.2 Kernel density estimation 324

10.3 Nonkernel methods 338

10.4 Multivariate methods 341

Problems 356

11 Bivariate Smoothing 361

11.1 Predictor-response data 362

11.2 Linear smoothers 364

11.3 Comparison of linear smoothers 376

11.4 Nonlinear smoothers 378

11.5 Confidence bands 385

11.6 General bivariate data 388

Problems 389

12 Multivariate Smoothing 393

12.1 Predictor-response data 393

12.2 General multivariate data 415

Problems 419

Data Acknowledgments 423

References 425

Index 453