انت هنا الان : شبكة جامعة بابل > موقع الكلية > نظام التعليم الالكتروني > مشاهدة المحاضرة

Probability Theory (Part 2)

Probability Axioms and Definitions

2)Conditional Probability

3) Bayes’ Theorem

Outlines

3) Bayes’ TheoremThe Axiomatic “Definition” of Probability

Suppose that for experimental model M, the sample space S of possible 

outcomes is defined as: {A1, … ,An}  ? S.

Let Pr(Ai

) = the probability of an event Ai

in the sample space S.

A probability distribution on a sample space S is a specification of numbers 

Pr(Ai

) which satisfy A1, A2, A3.

A1. For any outcome A, Pr(A) ? 0. A1. For any outcome Ai

, Pr(Ai

) ? 0.

A2. Pr(S) = 1.

A3. For any infinite sequence of disjoint events A1, …, An:

Pr( ?i=1 to ? Ai

) =  ?i=1 to ? Pr (Ai

)

Note: it turns out that each of these three axioms can be justified using the 

coherence criterion.Some Theorems Based on the Definition of 

Probability and a Few Proofs

Theorem 1. Pr(? ?? ?) = 0

Proof:

By definition, Aj

and Ak are disjoint if Aj

? Ak = ?.

Further, it is obvious that: ??? = ?.

Thus, if Aj

= ? and Ak = ?, then Aj

and Ak 

are disjoint.

Let A1 … An define the set of events such that Aj

= ?.

By the above definitions, it follows that the events Aj

are disjoint.

Since the Aj

are disjoint, we can exploit A3 such that:

Pr(?) = Pr( ?i 

Ai

) =  ?i

Pr (Ai

) = ?i

Pr(?) = n Pr(?)

In order that Pr(?) = n Pr(?), Pr(?) must equal 0.Some Theorems cont.

Theorem 2. For any sequence of n disjoint events A1,…,An,

Pr( ?i to n 

Ai

) =  ?i to n 

Pr (Ai

)

Proof:

Let A1,…,An define the n disjoint events and let Ak = ? for events k ?

{n+1,…, ?}.  {n+1,…, ?}.

By the definition of disjoint events, we have an infinite series of disjoint 

events.

By A3 and Theorem 1 which states that Pr(?)=0:

Pr( ?i to n 

Ai

) = Pr( ?i to ? Ai

) =  ?i to ? Pr (Ai

).

= ?i to n 

Pr (Ai

) + ?n+1 to ? Pr (Ai

)

= ?i to n 

Pr (Ai

) + 0

= ?i to n 

Pr (Ai

) Some Theorems cont.

Theorem 3. For any event A, Pr(AC) = 1 –Pr(A)

Theorem 4. For any event A, 0 ? Pr(A) ? 1

Proof by contradiction in two parts: Proof by contradiction in two parts:

Part 1. Suppose Pr(A) < 0. Then that would 

violate axiom A1, a contradiction.

Part 2. Suppose Pr(A) > 1. Then by Theorem 3,

Pr(AC) < 0, which also contradicts A1.

Thus, 0 ? Pr(A) ? 1.Some Theorems cont.

Theorem 5. For any two events A and B,

Pr(A  ?  ? ?  ? B) = Pr(A) + Pr(B) –Pr(A ? ?? ? B)

Proof:

A ? B = (A ? BC) ? (A ? B) ? (AC ? B)

Since all three elements in the equation are disjoint, Theorem 2 implies:

Pr(A ? B) = Pr(A ? BC) + Pr(A ? B) + Pr(AC ? B)

= Pr(A ? BC) + Pr(A ? B) + Pr(AC ? B) + Pr(A ? B) -Pr(A ? B)

Further, we know that Pr(A) = Pr(A ? BC) + Pr(A ? B)

and that Pr(B) = Pr(A ? B) + Pr(AC ? B)

Thus, Pr(A ? B) = Pr(A) + Pr(B) -Pr(A ? B) Independent Events

Intuitively, we define independence as:

Two events A and B are independent if the occurrence or non-

occurrence of one of the events has no influence on the occurrence 

or non-occurrence of the other event.

Mathematically, we write define independence as: Mathematically, we write define independence as:

Two events A and B are independent if Pr(A ? B) = Pr(A)Pr(B).Example of Independence

Are party id and vote choice independent in presidential elections?

Suppose Pr(Rep. ID) = .4, Pr(Rep. Vote) = .5, and Pr(Rep. ID ? Rep. Vote) = .35

To test for independence, we ask whether:

Pr Pr(Rep. ID) * Pr(Rep. Vote) = .35 ?

Substituting into the equations, we find that:

Pr Pr(Rep. ID) * Pr(Rep. Vote) = .4*.5 = .2  ? .35,

so the events are not independent.Independence of Several Events

The events A1, …, An are independent if:

Pr(A1 ? A2 ? … ? An) = Pr(A1)Pr(A2)…Pr(An)

And, this identity must hold for any subset of 

events. events.Conditional Probability

Conditional probabilities allow us to understand how the probability of an 

event A changes after it has been learned that some other event B has 

occurred.

The key concept for thinking about conditional probabilities is that the 

occurrence of B reshapes the sample space for subsequent events.

-That is, we begin with a sample space S

-A and B ? S

- The conditional probability of A given that B looks just at the subset of the  - The conditional probability of A given that B looks just at the subset of the 

sample space for B.

S

A

B

Pr(A | B)

The conditional probability of A given B is 

denoted Pr(A | B).

-Importantly, according to Bayesian 

orthodoxy, all probability distributions are 

implicitly or explicitly conditioned on the model.Conditional Probability Cont.

By definition: If A and B are two events such that 

Pr(B) > 0, then:

S

A

B

Pr(A | B)

Pr(B)

B)  Pr(A 

    B) | Pr(A 

? =

A

Pr(B)

Example: What is the Pr(Republican Vote | Republican Identifier)?

Pr(Rep. Vote ? Rep. Id) = .35 and Pr(Rep ID) = .4

Thus, Pr(Republican Vote | Republican Identifier) = .35 / .4 = .875Useful Properties of Conditional Probabilities

Property 1. The Conditional Probability for Independent Events

If A and B are independent events, then:

Pr(A)

Pr(B)

Pr(B) Pr(A)