Note for Probability - P by Placement Factory

Topics ( 25 )

Text from page-1

Probability About these notes. Many people have written excellent notes for introductory courses in probability. Mine draw freely on material prepared by others in presenting this course to students at Cambridge. I wish to acknowledge especially Geoffrey Grimmett, Frank Kelly and Doug Kennedy. The order I follow is a bit different to that listed in the Schedules. Most of the material can be found in the recommended books by Grimmett & Welsh, and Ross. Many of the examples are classics and mandatory in any sensible introductory course on probability. The book by Grinstead & Snell is easy reading and I know students have enjoyed it. There are also some very good Wikipedia articles on many of the topics we will consider. In these notes I attempt a ‘Goldilocks path’ by being neither too detailed or too brief. • Each lecture has a title and focuses upon just one or two ideas. • My notes for each lecture are limited to 4 pages. I also include some entertaining, but nonexaminable topics, some of which are unusual for a course at this level (such as random permutations, entropy, reflection principle, Benford and Zipf distributions, Erd˝ os’s probabilistic method, value at risk, eigenvalues of random matrices, Kelly criterion, Chernoff bound). You should enjoy the book of Grimmett & Welsh, and the notes notes of Kennedy. Printed notes, good or bad? I have wondered whether it is helpful or not to publish full course notes. On balance, I think that it is. It is helpful in that we can dispense with some tedious copying-out, and you are guaranteed an accurate account. But there are also benefits to hearing and writing down things yourself during a lecture, and so I recommend that you still do some of that. I will say things in every lecture that are not in the notes. I will sometimes tell you when it would be good to make an extra note. In learning mathematics repeated exposure to ideas is essential. I hope that by doing all of reading, listening, writing and (most importantly) solving problems you will master and enjoy this course. I recommend Tom K¨ orner’s treatise on how to listen to a maths lecture. i

Text from page-2

Contents About these notes i Table of Contents ii Schedules vi Learning outcomes vii 1 Classical probability 1.1 Diverse notions of ‘probability’ 1.2 Classical probability . . . . . . 1.3 Sample space and events . . . . 1.4 Equalizations in random walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 2 3 2 Combinatorial analysis 2.1 Counting . . . . . . . . . . . . . . . . . . . . 2.2 Sampling with or without replacement . . . . 2.3 Sampling with or without regard to ordering 2.4 Four cases of enumerative combinatorics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 6 6 8 8 3 Stirling’s formula 3.1 Multinomial coefficient . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Stirling’s formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Improved Stirling’s formula . . . . . . . . . . . . . . . . . . . . . . . . . 10 10 11 13 4 Axiomatic approach 4.1 Axioms of probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Boole’s inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Inclusion-exclusion formula . . . . . . . . . . . . . . . . . . . . . . . . . 14 14 15 17 5 Independence 5.1 Bonferroni’s inequalities . . . . . . . . 5.2 Independence of two events . . . . . . 5.3 Independence of multiple events . . . . 5.4 Important distributions . . . . . . . . 5.5 Poisson approximation to the binomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 18 19 20 20 21 6 Conditional probability 6.1 Conditional probability 6.2 Properties of conditional 6.3 Law of total probability 6.4 Bayes’ formula . . . . . 6.5 Simpson’s paradox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 22 22 23 23 24 . . . . . . . . . . . . . . . . . . . probability . . . . . . . . . . . . . . . . . . . . . ii . . . . . . . . . . . . . . . . . . . . .

Text from page-3

7 Discrete random variables 7.1 Continuity of P . . . . . . . . 7.2 Discrete random variables . . 7.3 Expectation . . . . . . . . . . 7.4 Function of a random variable 7.5 Properties of expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 26 27 27 29 29 8 Further functions of random variables 8.1 Expectation of sum is sum of expectations 8.2 Variance . . . . . . . . . . . . . . . . . . . 8.3 Indicator random variables . . . . . . . . 8.4 Reproof of inclusion-exclusion formula . . 8.5 Zipf’s law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 30 30 32 33 33 9 Independent random variables 9.1 Independent random variables . . . . . . 9.2 Variance of a sum . . . . . . . . . . . . 9.3 Efron’s dice . . . . . . . . . . . . . . . . 9.4 Cycle lengths in a random permutation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 34 35 36 37 10 Inequalities 10.1 Jensen’s inequality . . . . . 10.2 AM–GM inequality . . . . . 10.3 Cauchy-Schwarz inequality . 10.4 Covariance and correlation . 10.5 Information entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 38 39 39 40 41 11 Weak law of large numbers 11.1 Markov inequality . . . . . . . . . . . . . . . . . . . . . . 11.2 Chebyshev inequality . . . . . . . . . . . . . . . . . . . . . 11.3 Weak law of large numbers . . . . . . . . . . . . . . . . . 11.4 Probabilistic proof of Weierstrass approximation theorem 11.5 Benford’s law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 42 42 43 44 45 12 Probability generating functions 12.1 Probability generating function . . . . . . . . . . . . . . . . . . . . . . . 12.2 Combinatorial applications . . . . . . . . . . . . . . . . . . . . . . . . . 46 46 48 13 Conditional expectation 13.1 Conditional distribution and expectation 13.2 Properties of conditional expectation . . 13.3 Sums with a random number of terms . 13.4 Aggregate loss distribution and VaR . . 13.5 Conditional entropy . . . . . . . . . . . 50 50 51 51 52 53 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Text from page-4

14 Branching processes 14.1 Branching processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2 Generating function of a branching process . . . . . . . . . . . . . . . . 14.3 Probability of extinction . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 54 54 56 15 Random walk and gambler’s ruin 15.1 Random walks . . . . . . . . . . . . . . . . 15.2 Gambler’s ruin . . . . . . . . . . . . . . . . 15.3 Duration of the game . . . . . . . . . . . . . 15.4 Use of generating functions in random walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 58 58 60 61 16 Continuous random variables 16.1 Continuous random variables . . . . . . . . . 16.2 Uniform distribution . . . . . . . . . . . . . . 16.3 Exponential distribution . . . . . . . . . . . . 16.4 Hazard rate . . . . . . . . . . . . . . . . . . . 16.5 Relationships among probability distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 62 64 64 65 65 17 Functions of a continuous random variable 17.1 Distribution of a function of a random variable 17.2 Expectation . . . . . . . . . . . . . . . . . . . . 17.3 Stochastic ordering of random variables . . . . 17.4 Variance . . . . . . . . . . . . . . . . . . . . . . 17.5 Inspection paradox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 66 67 68 68 69 18 Jointly distributed random variables 18.1 Jointly distributed random variables . . . . . 18.2 Independence of continuous random variables 18.3 Geometric probability . . . . . . . . . . . . . 18.4 Bertrand’s paradox . . . . . . . . . . . . . . . 18.5 Buffon’s needle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 70 71 71 72 73 19 Normal distribution 19.1 Normal distribution . . . . . . . . . . . . 19.2 Calculations with the normal distribution 19.3 Mode, median and sample mean . . . . . 19.4 Distribution of order statistics . . . . . . . 19.5 Stochastic bin packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 74 75 76 76 77 20 Transformations of random variables 20.1 Transformation of random variables . . . . . . . . . . . . . . . . . . . . 20.2 Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.3 Cauchy distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 78 80 81 21 Moment generating functions 82 iv . . . . . . . . . .