Sigma Algebra Generated by Random Variable Same as by Continuous Injective Function

In this lecture we provide a rigorous definition of conditional probability, based on sigma-algebras.

The main advantage of this definition with respect to a more elementary definition is that it allows us to condition on zero-probability events.

Table of contents

1. Motivation

2. Roadmap

3. Partitions of events

4. Probability conditional on a partition

5. The fundamental property of conditional probability

6. The fundamental property as a defining property

7. Conditioning with respect to sigma-algebras

8. How to interpret conditioning with respect to sigma-algebras

9. Finer and coarser sigma-algebras

10. Regular conditional probabilities

11. Applications

In the introduction to conditional probability, we have stated a number of properties that conditional probabilities should satisfy to be rational in some sense.

We have proved that, whenever , these properties are satisfied if and only if

However, we have not been able to derive a formula for probabilities conditional on zero-probability events. In other words, we have not been able to find a way to compute when .

Thus, we have concluded that the above elementary formula cannot be taken as a general definition of conditional probability because it does not cover zero-probability events.

We now discuss a completely general definition of conditional probability, which covers also the case in which . The resulting concept is called conditional probability with respect to a sigma-algebra.

The plan of the lecture is as follows:

1. We define the concept of a partition of events.

2. We show that, given a partition of events, conditional probability can be regarded as a random variable (probability conditional on a partition).

3. We show that, when no zero-probability events are involved, probabilities conditional on a partition satisfy a certain property (the fundamental property of conditional probability).

4. We require that the fundamental property of conditional probability be satisfied also when zero-probability events are involved. We show that this requirement is sufficient to unambiguously pin down probabilities conditional on a partition; therefore, it can be used to provide a completely general definition of conditional probability.

5. Finally, we discuss how to replace partitions with sigma-algebras.

Let be a sample space and let denote the probability assigned to the events .

Define a partition of events of as follows.

In other words, a partition is a subdivision of into non-overlapping and non-empty events that cover all of .

Example Consider the sample space which describes the possible outcomes of the toss of a die. Define the two events Then, is a partition of events of . In fact, and .

Take a finite partition of events of , such that for every .

Suppose that at a certain time in the future we will be told that the realized outcome belongs to a set .

After receiving this information, we will update our assessment of the probability of an event , by computing its conditional probability

Before receiving the information, this conditional probability is unknown and can be regarded as a random variable, denoted by and defined as follows:

The random variable is the probability of conditional on the partition .

Example Let us continue with the example above, where the sample space is We consider the partition where We assign equal probability to all the outcomes: We would like to analyze the conditional probability of the event We have The conditional probability is a discrete random variable defined as follows: Since , the probability mass function of is

A property of is that its expected value equals the unconditional probability :

Proof

This is proved as follows:

Example In the previous example, the probability mass function of was It is easy to verify that

The property above can be generalized as follows.

Proposition (Fundamental property) Let be a finite partition of events of such that for every . Let be any event obtained as a union of events . Let be the indicator function of . Let be defined as above. Then,

Proof

Suppose that we are not able to explicitly define because contains a zero-probability event and, therefore, we cannot use the formula to define for .

When we are not able to explicitly define , what we can do is to define implicitly, by requiring that it satisfies the fundamental property of conditional probability for all events obtained as unions of events .

How can we be sure that there exists a random variable satisfying this property?

Existence is guaranteed by the following important theorem, that we state without providing a proof.

Proposition Let be an arbitrary partition of events of . Let be an event. Then, there exists at least one random variable that satisfies the property for all the events obtained as unions of events . Furthermore, if and are two random variables such that for all , then and are almost surely equal.

Thus, a random variable satisfying the fundamental property of conditional probability exists and is unique (up to almost sure equality).

As a consequence, we can indirectly define the probability of an event conditional on the partition as .

This indirect way of defining conditional probability is summarized as follows..

Definition (Probability conditional on a partition) Let be a partition of events of . Let be an event. The probability of conditional on the partition is a random variable that satisfies for all events obtainable as unions of events .

As we have seen above, such a random variable is guaranteed to exist and is unique up to almost sure equality.

Different random variables satisfying the criterion in the definition are called versions of the conditional probability.

In rigorous probability theory, conditional probability is defined with respect to sigma-algebras, rather than with respect to partitions.

Let be a probability space.

Let be a sub-sigma-algebra of (i.e., is a sigma-algebra and ).

Let be an event.

We say that a random variable is a conditional probability of with respect to the sigma-algebra if and only if

It can be shown that this definition is equivalent to our definition of probability conditional on a partition.

In particular, if is a partition of events of and is the smallest sigma-algebra containing all the possible unions of events , then is the same as .

Thus, when we see the abstract definition of conditional probability with respect to a sigma-algebra , we can think about it as follows:

1. there is a partition of the sample space;

2. the sigma-algebra is the smallest sigma-algebra that contains all the elements of ;

3. at some future time we will be told that the realized outcome belongs to a set ;

4. at that time, we will be able to compute a conditional probability ;

5. until that time, this conditional probability is unknown and it can be regarded as a random variable, denoted by .

Let us get back to our toss-of-a-die example, in which the sample space is

Consider the partition

The smallest sigma-algebra containing is

Now, consider the partition

The smallest sigma-algebra containing is

Since contains all the sets included in plus some more, we say that is finer than . Conversely, we say that is coarser than .

We also say that the conditional probability is based on a larger information set as compared to . In intuitive terms, this means that the information we receive when we are able to actually calculate the conditional probability is more precise.

Until now, we have kept fixed the event in the conditional probability . Moreover, we have regarded as a random variable.

If we allow to vary, becomes a random probability measure.

Or does it? Mathematicians have found some examples (see here), in which, despite a careful choice of the versions of , it is not possible to simultaneously satisfy the two requirements that

1. be -measurable (i.e., a proper random variable) for each choice of ;

2. be a proper probability measure for each choice of (except possibly for some forming a set of measure zero).

When these two requirements can be satisfied, we say that the probability space admits a regular probability conditional on the sigma-algebra .

The abstract definition of conditional probability with respect to a sigma-algebra is extremely useful.

One of its most important applications is the derivation of conditional probability density functions for continuous random vectors. To learn about this application, read the next lecture, on Conditional probability distributions.

Please cite as:

Taboga, Marco (2021). "Conditional probability with respect to a sigma-algebra", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/conditional-probability-as-a-random-variable.

rosagoodue.blogspot.com

Source: https://www.statlect.com/fundamentals-of-probability/conditional-probability-as-a-random-variable

Share this post