Probability Theory (Part 1)
•Random experiments
•Samples and Sample
Space
• Events • Events
•Probability Space
•Axioms of Probability
•Conditional Probability
•Bayes’ RuleRandom Experiments
•A random experiment is an experiment in which the outcome varies in
a unpredictable fashion when the experiment is repeated under the
same condition.
•A random experiment is specified by stating an experimental
procedure and a set of one or more measurements or observations.
•Examples:
– E1:Toss a coin three times and note the sides facing up (heads or – E1:Toss a coin three times and note the sides facing up (heads or
tail)
–E2:Pick a number at random between zero and one.
–E3: Poses done by a rookie dancerSamples and Sample Space
•A sample point (o) or an outcome of a random
experiment is defined as a result that cannot be
decomposed into other results.
•Sample Space (S): defined as the set of all
possible outcomes from a random experiment. possible outcomes from a random experiment.
•Countable or discrete sample space, one-to-one
correspondence between outcomes and
integers
•Uncountable or continuous sample spaceEvents
•A event is a subset of the sample space S,
a set of samples.
•Two special events:
–Certain event: S
–Impossible or null event: ?Axioms of Probability
•0?P(A)
•P(S)=1
•If A ? B =?, then P(A U B)=P(A)+P(B)
• Given a sequence of event, Ai, if ? i?j, • Given a sequence of event, Ai, if ? i?j,
Ai ? Bj =?,
•P(U ?i=1 Ai )=??i=1 P(Ai)
•referred to as countable additivity•P(Ac) = 1 –P(A)
•P(?) = 0
•P(A) ? 1
•Given a sequence of event, Ai,..., An, if ? i?j, Ai ? Bj =?,
•P(U ni=1 Ai )=? ni=1 P(Ai)
• P(AUB)=P(A)+P(B)-P(A ? B)
Some Corollaries
• P(AUB)=P(A)+P(B)-P(A ? B)
B A
A ? BConditional Probability
) (
) (
) | (
B P
B A P
B A P
I = A ? B
Imagine that P(A) is proportional to the size of the area
S
B ABayes’ Rule
•Let {Bi} be a partition of the sample space S.
•Suppose that event A occurs, what is the probability of the event Bj?
•By the definition of conditional
•probability we have
a priori
? =
= = n
k k k
j j j
j
B P B A P
B P B A P
A P
B A P
A B P
1
) ( ) | (
) ( ) | (
) (
) (
) | (
I
a posteriori Independence of Events
•If knowledge of the occurrence of an event B does not alter the
probability of some other event A, then it would be natural to say that
event A is independent of B.
• The most common application of the independence concept is in making
) (
) (
) | ( ) (
B P
B A P
B A P A P
I = = ) ( ) ( ) ( B P A P B A P = I
• The most common application of the independence concept is in making
the assumption that the events of separate experiments are independent,
which are referred as independent experiments.Random Variables
•A random variable X is a function that assigns a real number, X(?),
to each outcome ? in the sample space of a random experiment.
S X(?) S
?
SX
Real line
X(?)
xRandom Variables
•Let SX be the set of the sample space.
•X(?) can be considered as a new random experiments with
outcomes X(?) as a function of ?, the outcome of the original
experiment.
S
?
SX
Real line
X(?)
xExamples
•E1:
–Toss a coin three times
–S={HHH,HHT,HTH,HTT, THH,THT,TTH,TTT}
–X(?)=number of heads in three coins tosses. Note that
sometimes a few ? share the same value of X(?).
–SX={0,1,2,3}
–X is then a random variable taking on the values in
the set SX
•If the outcome ? of some experiment is already a
numerical value, we can immediately reinterpret the
outcome as a random variable defined by X(?)= ?.Expectation
•The expectation of a random variable is given by the
weighted average of the values in the support of the
random variable
? =
N
k X k x p x X E ) ( } {
?
+?
? ?
= dx x xp X E X ) ( } {
? = k
k X k
1Smoothing Property of Conditional Expectation
•EY|X {Y|X=x}=g(x)
•E{Y}=EX{EY|X {Y|X=x}}Thank you !