Preface
1 Probability Basics 1
1.1 What is Probability?
1.2 Experiments, Outcomes, and Events
1.3 Venn Diagrams
1.4 Random Variables
1.5 Basic Probability Rules
1.6 Probability Formalized
1.7 Little Theorems
1.8 Compound Experiments
1.9 Independence
1.10 Example: Can S CommunicateWith D?
1.10.1 List All Outcomes
1.10.2 Probability of a Union
1.10.3 Probability of the Complement
1.11 Example: Now Can S CommunicateWith D?
1.11.1 A Big Table
1.11.2 Break Into Pieces
1.11.3 Probability of the Complement
1.12
Computational Procedures
1.13 Summary
1.14 Problems
2 Conditional Probability 30
2.1 Definitions of Conditional Probability
2.2 Law of Total Probability and Bayes Theorem
2.3 Example: UrnModels
2.4 Example: A Binary Channel
2.5 Example: Drug Testing
2.6
Example: A Diamond Network
2.7 Summary
2.8 Problems
3 A Little Combinatorics 49
3.1 Basics of Counting
3.2 Notes on Computation
3.3 Combinations and the Binomial Coefficients
3.4 The Binomial Theorem
3.5 Multinomial Coefficient and Theorem
3.6 The Birthday
Paradox andMessage Authentication
3.7 Hypergeometric Probabilities and Card Games
3.8 Summary
3.9 Problems
4 Discrete Probabilities and Random Variables
4.1 Discrete Random Variable and ProbabilityMass Functions
4.2 Cumulative Distribution Functions
4.3 Expected
Values
4.4 Moment Generating Functions .
4.5 Several Important Discrete PMF>'s
4.5.1 UniformPMF
4.5.2 Geometric ProbabilityMass Function (PMF)
4.5.3 The Poisson Distribution
4.6 Gambling and Financial DecisionMaking
4.7 Summary
4.8 Problems
5 Multiple
Discrete Random Variables 110
5.1 Multiple Random Variables and PMFs
5.2 Independence
5.3 Moments and Expected Values
5.3.1 Expected Values for Two Random Variables
5.3.2 Moments for Two Random Variables
5.4 Example of Two Discrete Random Variables
5.4.1 Marginal
PMFs and Expected Values
5.4.2 Independence
5.4.3 Joint Cumulative Distribution Function (CDF)
5.4.4 TransformationsWith One Output
5.4.5 TransformationsWith Several Outputs
5.4.6 Discussion
5.5 Sums of Independent Random Variables
5.6 Sample Probabilities,Mean, and
Variance .
5.7 Histograms
5.8 Entropy and Data Compression
5.8.1 Entropy and Information Theory
5.8.2 Variable Length Coding
5.8.3 Encoding Binary Sequences .
5.8.4 MaximumEntropy
5.9 Summary
5.10 Problems
6 Binomial Probabilities
6.1 Basics of the
Binomial Distribution
6.2 Computing Binomial Probabilities
6.3 Moments of the Binomial Distribution .
6.4 Sums of Independent Binomial Random Variables .
6.5 Distributions Related to the Binomial
6.5.1 Connections Between Binomial andHypergeometric Probabilities
6.5.2
Multinomial Probabilities .
6.5.3 Negative Binomial Distribution .
6.5.4 The Poisson Distribution
6.6 Parameter Estimation for Binomial andMultinomial Distributions
6.7 Alohanet
6.8 Error Control Codes .
6.8.1 Repetition by Three Code
6.8.2 General Linear Block Codes
6.8.3 Error Correcting Coding (ECC) Conclusions
6.9 Summary
6.10 Problems
7 A Continuous Random Variable
7.1 AContinuous Random Variable and Its Density,Distribution Function,
and Expected Values . .
7.2 Example Calculations for a Single Random Variable
7.3
Selected Continuous Distributions
7.3.1 The UniformDistribution
7.3.2 The Exponential Distribution
7.4 Conditional Probabilities for a Continuous Random Variable
7.5 Discrete PMF>'s and Delta Functions
7.6 Quantization
7.7 Summary .
7.8 A FinalWord
7.9
Problems
8 Multiple Continuous Random Variables 206
8.1 Joint Densities and Distribution Functions
8.2 Expected Values andMoments
8.3 Independence
8.4 Conditional Probabilities forMultiple Random Variables
8.5 Extended Example of Two Continuous Random Variables
8.6 Sums
of Independent Random Variables
8.7 Random Sums
8.8 General Transformations and the Jacobian
8.9 Parameter Estimation for the Exponential Distribution
8.10 Comparison of Discrete and Continuous Distributions
8.11 Summary
8.12 Problems
9 The Gaussian and Related
Distributions
9.1 The Gaussian Distribution and Density
9.2 Quantile Function
9.3 Moments of the Gaussian Distribution
9.4 The Central Limit Theorem
9.5 Related Distributions
9.5.1 Laplace Distribution
9.5.2 Rayleigh Distribution
9.5.3 Chi-Squared and F
Distributions
9.6 Multiple Gaussian RandomVariables .
9.6.1 Independent Gaussian Random Variables
9.6.2 Transformation to Polar Coordinates
9.6.3 Two Correlated Gaussians
9.7 Example: Digital Communications using QAM
9.7.1 Background
9.7.2 Discrete TimeModel
9.7.3
Monte Carlo Exercise
9.7.4 Quadrature AmplitudeModulation (QAM) Recap
9.8 Summary .
9.9 Problems
10 Elements of Statistics 282
10.1 A Simple Election Poll .
10.2 Estimating theMean and Variance .
10.3 Confidence Intervals
10.4 Recursive Calculation of the
SampleMean .
10.5 ExponentialWeighting
10.6 Estimating the Distribution Function
10.7 PMF and Density Estimates
10.8 Order Statistics and Robust Estimates .
10.9 Significance Tests and P-Values
10.10Introduction to Estimation Theory
10.11MinimumMean Squared Error
Estimation .
10.12Bayesian Estimation
10.13Problems
11 Gaussian Random Vectors and Linear Regression
11.1 Gaussian Random Vectors
11.2 Linear Operations on Gaussian Random Vectors
11.3 Linear Regression
11.3.1 Linear Regression in Detail .
11.3.2 Statistics of the
Linear Regression Estimates .
11.3.3 Computational Issues
11.3.4 Linear Regression Examples .
11.3.5 Extensions of Linear Regression .
11.4 Summary .
11.5 Problems .
12 Hypothesis Testing
12.1 Hypothesis Testing: Basic Principles
12.2 Example of Radar Detection
.
12.3 Hypothesis Tests and Likelihood Ratios
12.4 MaximumA Posteriori Tests
12.5 Summary .
12.6 Problems
13 Random Signals and Noise
13.1 Introduction to Random Signals
13.2 A Simple RandomProcess
13.3 Fourier Transforms . .
13.4 Wide Sense Stationary
RandomProcesses
13.5 Wide Sense Stationary (WSS) Signals and Linear Filters
13.6 Noise
13.6.1 Probabilistic Properties of Noise
There are no Instructor/Student Resources available at this time.
Charles Boncelet has a BS in Applied and Engineering Physics from Cornell University and an MS and PhD in Electrical Engineering and Computer Science from Princeton University. Since 1984, he has been employed at the University of Delaware. He is a Professor in the Electrical & Computer
Engineering Department and has a joint appointment in the Computer & Information Science Department. He is currently Associate Chair of Undergraduate Studies in the ECE Department. Boncelet has written approximately 100 research papers in journals and technical conferences on a variety of topics in
signal processing, information theory, probability, and algorithms. He regularly teaches courses in probability and statistics, signal processing, and communications. Boncelet is a senior member of the IEEE and a member of SIAM, Eta Kappa Nu, and the Delaware Academy of Science.
