Basic Probability Theory
Random Experiments
A random experiment is an experiment in which the outcome varies in a unpredictable fashion when the experiment is repeated under the same condition.
A random experiment is specified by stating an experimental procedure and a set of one or more measurements or observations.
Examples:
E1:Toss a coin three times and note the sides facing up (heads or tail)
E2:Pick a number at random between zero and one.
E3: Poses done by a rookie dancer
Samples and Sample Space
A sample point (o) or an outcome of a random experiment is defined as a result that cannot be decomposed into other results.
Sample Space (S): defined as the set of all possible outcomes from a random experiment.
Countable or discrete sample space, one-to-one correspondence between outcomes and integers
Uncountable or continuous sample space
Events
A event is a subset of the sample space S, a set of samples.
Two special events:
Certain event: S
Impossible or null event: ?
Axioms of Probability
1. 0?P(A)
2. P(S)=1
3. If A ? B =?, then P(A U B)=P(A)+P(B)
4. Given a sequence of event, Ai
, if ? i?j, Ai
? Bj
=?,
P(U
?
i=1
Ai
)=? ?
i=1
P(Ai
)
Sensor Fusion Fall 2008
P(U i=1
Ai
)=? i=1
P(Ai
)
referred to as countable additivity
Bayes’ Rule
Let {Bi
} be a partition of the sample space S.
Suppose that event A occurs, what is the probability of the event Bj
?
By the definition of conditional
probability we have
Independence of Events
If knowledge of the occurrence of an event B does not alter the
probability of some other event A, then it would be natural to say that
event A is independent of B
The most common application of the independence concept is in making
) (
) (
) | ( ) (
B P
B A P
B A P A P
I = = ) ( ) ( ) ( B P A P B A P = I
The most common application of the independence concept is in making
the assumption that the events of separate experiments are independent,
which are referred as independent experiments.
Random Variables
n
A random variable X is a function that assigns a real number, X(z), to each outcome z in the sample space of a random experiment.
A random variable X is a function that assigns a real number, X(z), to each outcome z in the sample space of a random experiment.
Examples
n E1:
¨ Toss a coin three times
¨ S={HHH,HHT,HTH,HTT, THH,THT,TTH,TTT}
¨ X(z)=number of heads in three coins tosses. Note that sometimes a few z share the same value of X(z).
¨ SX={0,1,2,3}
¨ X is then a random variable taking on the values in the set SX
n If the outcome z of some experiment is already a numerical value, we can immediately reinterpret the outcome as a random variable defined by X(z)= z.