conditional distribution2

As usual, we start with a random experiment that has a sample space R and probability measure P on R. Suppose that X is a random variable for the experiment, taking values in a set S. The purpose of this section is to study the conditional probability measure on R given X = x, for x in S. Thus, we would like to define and study
P(A | X = x) for A R and for x in S.
We will see that if X has a discrete distribution, no new concepts are involved, and the simple definition of conditional probability suffices. When X has a continuous distribution, however, a fundamentally new approach is needed.
Definitions and Basic Properties
Suppose first that X has a discrete distribution with density function g. Thus, S is countable and we can assume that g(x) > 0 for x in S.
1. Show that
P(A | X = x) = P(X = x, A) / g(x) for A R, x in S.
2. Prove the following version of the law of total probability
P(X B, A) = x in B P(A | X = x)g(x) for A R, B S.
Conversely, the law of total probability completely characterizes the conditional distribution given X = x.
3. Suppose that Q(x, A), for x S, A R, satisfies
P(A, X B) = x in B Q(x, A) g(x) for B S.
Show that Q(x, A) = P(A | X = x) for x S, A R.
Suppose now that X has a continuous distribution on S Rn, with density function g. We assume that g(x) > 0 for x in S. Unlike the discrete case, we cannot use simple conditional probability to define the conditional probability of an event given X = x, because the conditioning event has probability 0 for any x. Nonetheless, the concept should make sense. If we actually run the experiment, X will take on some value x (even though a priori, this event occurs with probability 0), and surely the information that X = x should in general alter the probabilities that we assign to other events. A natural approach is to use the results obtained in the discrete case as definitions in the continuous case. Thus, based on the characterization above, we define the conditional probability
P(A | X = x) for x in S, A R.
by the requirement that the law of total probability hold:
P(A, X B) = B P(A | X = x) g(x)dx for any B S.
For now, we will accept the fact that P(A | X = x) can be defined by this condition. However, we will return to this point in the section on Conditional Expectation in the chapter on Expected Value.
Bayes Theorem, named after Thomas Bayes, gives a formula for the conditional density of X given A, in terms of the density of X and the conditional probability of A given X = x.
4. Let A be an event with P(A) > 0. Show that the conditional density of X given A is
a. g(x | A) = g(x)P(A | X = x) / s in S g(s)P(A | X = s) if X is discrete.
b. g(x | A) = g(x)P(A | X = x) / S g(s)P(A | X = s)ds if X is continuous.
In the context of Bayes theorem, g is called the prior density of X and g( • | A) is the posterior density of X given A.
Conditional Densities
The definitions and results above apply, of course, if A is an event defined in terms of another random variable for our experiment.
Thus, suppose that Y is a random variable taking values in a set T. Then (X, Y) is a random variable taking values in the product set S × T, which we assume has (joint) probability density function f. (In particular, we are assuming one of the standard distribution types: jointly discrete, jointly continuous with density, or mixed