Probability and Statistical Inference, Third Edition is a user-friendly book that stresses the comprehension of concepts instead of the simple acquisition of a skill or tool. It provides a mathematical framework that permits students to carry out various procedures using R. Its unique approach to problems allows readers to integrate the knowledge gained from the text, thus, enhancing a more complete and honest understanding of the topic. The book focuses on the development of intuition and understanding through diversity of experience. New to this edition, in addition to R code, are a chapter on Bayesian statistics, additional concepts introduced, and new and improved problems and mini-projects. The book is intended for upper-level undergraduates or first‐year graduate students in the in statistics or related disciplines such as mathematics or engineering, where exposure to statistics is needed.

MAGDALENA NIEWIADOMSKA-BUGAJ, PHD, is Professor and Chair of the Statistics Department at Western Michigan University where she has taught over a dozen different courses in the last five years. Dr. Niewiadomska-Bugaj's areas of interest include general statistical methodology, nonparametric statistics, classification, and categorical data analysis. She has published over 50 papers, books, and book chapters in theoretical and applied statistics

The late ROBERT BARTOSZYŃSKI, PHD, was Professor in the Department of Statistics at The Ohio State University. His scientific contributions included research in the theory of stochastic processes and modeling biological phenomena. Throughout his career, Dr Bartoszyński published books, book chapters, and over 100 journal articles.

Preface vii

Preface vi

1 Experiments, Sample Spaces, and Events 1

1.1 Introduction 1

1.2 Sample Space 2

1.3 Algebra of Events 9

1.4 Infinite Operations on Events 15

2 Probability 25

2.1 Introduction 25

2.2 Probability as a Frequency 25

2.3 Axioms of Probability 26

2.4 Consequences of the Axioms 31

2.5 Classical Probability 35

2.6 Necessity of the Axioms* 36

2.7 Subjective Probability* 41

3 Counting 45

3.1 Introduction 45

3.2 Product Sets, Orderings, and Permutations 45

3.3 Binomial Coefficients 51

3.4 Multinomial Coefficients 64

4 Conditional Probability, Independence, and Markov Chains 67

4.1 Introduction 67

4.2 Conditional Probability 68

4.3 Partitions; Total Probability Formula 74

4.4 Bayes’ Formula 79

4.5 Independence 84

4.6 Exchangeability; Conditional Independence 90

4.7 Markov Chains* 93

5 Random Variables: Univariate Case 107

5.1 Introduction 107

5.2 Distributions of Random Variables 108

5.3 Discrete and Continuous Random Variables 117

5.4 Functions of Random Variables 129

5.5 Survival and Hazard Functions 136

6 Random Variables: Multivariate Case 141

6.1 Bivariate Distributions 141

6.2 Marginal Distributions; Independence 148

6.3 Conditional Distributions 160

6.4 Bivariate Transformations 167

6.5 Multidimensional Distributions 176

7 Expectation 183

7.1 Introduction 183

7.2 Expected Value 184

7.3 Expectation as an Integral* 192

7.4 Properties of Expectation 199

7.5 Moments 207

7.6 Variance 215

7.7 Conditional Expectation 227

7.8 Inequalities 231

8 Selected Families of Distributions 237

8.1 Bernoulli Trials and Related Distributions 237

8.2 Hypergeometric Distribution 251

8.3 Poisson Distribution and Poisson Process 256

8.4 Exponential, Gamma and Related Distributions 269

8.5 Normal Distribution 276

8.6 Beta Distribution 286

9 Random Samples 293

9.1 Statistics and Sampling Distributions 293

9.2 Distributions Related to Normal 295

9.3 Order Statistics 300

9.4 Generating Random Samples 307

9.5 Convergence 312

9.6 Central Limit Theorem 322

10 Introduction to Statistical Inference 331

10.1 Overview 331

10.2 Basic Models 334

10.3 Sampling 336

10.4 Measurement Scales 342

11 Estimation 347

11.1 Introduction 347

11.2 Consistency 352

11.3 Loss, Risk, and Admissibility 355

11.4 Efficiency 361

11.5 Methods of Obtaining Estimators 368

11.6 Sufficiency 387

11.7 Interval Estimation 403

12 Testing Statistical Hypotheses 419

12.1 Introduction 419

12.2 Intuitive Background 423

12.3 Most Powerful Tests 432

12.4 Uniformly Most Powerful Tests 445

12.5 Unbiased Tests 452

12.6 Generalized Likelihood Ratio Tests 456

12.7 Conditional Tests 463

12.8 Tests and Confidence Intervals 466

12.9 Review of Tests for Normal Distributions 467

12.10 Monte Carlo, Bootstrap, and Permutation Tests 477

14 Linear Models 483

14.1 Introduction 483

14.2 Regression of the First and Second Kind 485

14.3 Distributional Assumptions 491

14.4 Linear Regression in the Normal Case 494

14.5 Testing Linearity 500

14.6 Prediction 503

14.7 Inverse Regression 505

14.8 BLUE 508

14.9 Regression Toward the Mean 510

14.10 Analysis of Variance 512

14.11 One-Way Layout 512

14.12 Two-Way Layout 516

14.13 ANOVA Models with Interaction 518

14.14 Further Extensions 522

15 Rank Methods 525

15.1 Introduction 525

15.2 Glivenko-Cantelli Theorem 526

15.3 Kolmogorov-Smirnov Tests 530

15.4 One-Sample Rank Tests 537

15.5 Two-Sample Rank Tests 544

15.6 Kruskal-Wallis Test 548

16 Analysis of Categorical Data 551

16.1 Introduction 551

16.2 Chi-Square Tests 553

16.3 Homogeneity and Independence 559

16.4 Consistency and Power 565

16.5 2×2 Contingency Tables 570

16.6 r × c Contingency Tables 578

17 Basics of Bayesian Statistics 583

17.1 Introduction 583

17.2 Prior and Posterior Distributions 584

17.3 Bayesian Inference 592

17.4 Final Comments 608

Appendix 1 609

Appendix 2 616

Bibliography 619

Answers to Odd-Numbered Problems 624

Index 632

Index 632