Course Details
Contents
Introduction to Probability; Definitions, scope and history; limitation of classical and relative-frequency-based definitions, Sets, fields, sample space and events; axiomatic definition of probability , Combinatorics: Probability on finite sample spaces, Joint and conditional probabilities, independence, total probability; Bayes’ rule and applications, Random variables, Definition of random variables, continuous and discrete random variables, cumulative distribution function (cdf) for discrete and continuous random variables; probability mass function (pmf); probability density functions (pdf) and properties, Jointly distributed random variables, conditional and joint density and distribution functions, independence; Bayes’ rule for continuous and mixed random variables, Function of random a variable, pdf of the function of a random variable; Function of two random variables; Sum of two independent random variables, Expectation: mean, variance and moments of a random variable, Joint moments, conditional expectation; covariance and correlation; independent, uncorrelated and orthogonal random variables, Random vector: mean vector, covariance matrix and properties, Some special distributions: Uniform, Gaussian and Rayleigh distributions; Binomial, and Poisson distributions; Multivariate Gaussian distribution, Vector-space representation of random variables, linear independence, inner product, Schwarz Inequality, Elements of estimation theory: linear minimum mean-square error and orthogonality principle in estimation, Moment-generating and characteristic functions and their applications, Bounds and approximations: Chebysev inequality and Chernoff Bound, Sequence of random variables and convergence, Almost sure (a.s.) convergence and strong law of large numbers; convergence in mean square sense with examples from parameter estimation; convergence in probability with examples; convergence in distribution, Central limit theorem and its significance, Random process: realizations, sample paths, discrete and continuous time processes, examples, Probabilistic structure of a random process; mean, autocorrelation and autocovariance functions, Stationarity: strict-sense stationary (SSS) and wide-sense stationary (WSS) processes, Autocorrelation function of a real WSS process and its properties, cross-correlation function, Ergodicity and its importance, Spectral representation of a real WSS process: power spectral density, properties of power spectral density ; cross-power spectral density and properties; auto-correlation function and power spectral density of a WSS random sequence, Linear time-invariant system with a WSS process as an input: sationarity of the output, auto-correlation and power-spectral density of the output; examples with white-noise as input; linear shift-invariant discrete-time system with a WSS sequence as input, Spectral factorization theorem, Examples of random processes: white noise process and white noise sequence; Gaussian process; Poisson process, Markov Process.
Textbooks
1. Bertsekas, Dimitri P., and John N. Tsitsiklis. "Introduction to Probability Vol. 1." (2002).
2. Probability in Electrical Engineering and Computer Science: An Application-Driven Course, by Jean Walrand (1st ed), Amazon, 2014.
3. Sheldon Ross, A First Course in Probability (10th Ed.), Pearson Prentice Hall, 2018.
4. Gallager, Robert G. Stochastic processes: theory for applications. Cambridge University Press, 2013.
References
1. Bertsekas, Dimitri P., and John N. Tsitsiklis. "Introduction to Probability Vol. 1." (2002).
2. Probability in Electrical Engineering and Computer Science: An Application-Driven Course, by Jean Walrand (1st ed), Amazon, 2014.
3. Sheldon Ross, A First Course in Probability (10th Ed.), Pearson Prentice Hall, 2018.
4. Gallager, Robert G. Stochastic processes: theory for applications. Cambridge University Press, 2013.