Total Probability Theorem

Probability is a concept used in Mathematics especially in statistics to predict the likelihood of occurrence of an event. Probability gives an idea of whether an event will happen or not and if the event is predicted to happen, how much can we rely on the occurrence of an event. The probability of an event is a dimensionless quantity. It may be any numerical value that ranges from zero to one. If the probability of an event is 1, then the event is sure to happen and hence is called a sure event or a certain event. If the probability of an event is ‘0’, the event will not happen at all and hence is called an impossible event. The total probability theorem is a theorem that relates the conditional probability with the marginal probability.

Total Probability Theorem:

Consider two events A and B as indicated in the Venn diagram shown in figure 1. Let us consider that these two events are associated with the same sample space represented as ‘S’. The entire sample space can be divided into the following portions indicating the subsets of sample space ‘S”: (A ⋂ B’), (A’ ⋂ B’), (A ⋂ B), and (A’ ⋂ B). These subsets are mutually disjoint sets because they are disjoint pairwise. For any pair combinations of the four sets, they remain disjoint. However, the probability of occurrence of any of these events depends on the occurrence of the other events in the sample space. In such cases where the probability of an event is dependent on the probability of the other events in the same sample space, the total probability theorem is used to find the probability of the event. 

(Image will be Uploaded soon)

How to State and Prove Total Probability Theorem:

The total probability theorem states that “if A1, A2, A3 …… An are the partitions of the sample space S such that the probability of none of these events is equal to zero, then the probability of an event ‘E’ occurring in such a sample space is given as:

P(E) =  \[\sum_{i = 1}^{n}\] P(Ai).P(E | Ai)

Total Probability Theorem Proof:

(Image will be Uploaded soon)

Consider a sample space shown in the above figure such that C1, C2, C3 ………… Cn is the partitions of the sample space ‘S’ such that Cp ⋂ Cq = ∅ (Null set). i.e. the partitions are disjoint when p ≠ q where p and q = 1, 2, 3, 4 …. n. It is also true that P (Cm) ≠ 0. i.e. no event in the sample space has a non zero probability. The sample space can be represented as: 

S = C1 U C2 U C3 U ………… U Cn → (1)

For any event “A’ in the sample space S, the event ‘A’ can be denoted as:

A = A ⋂ S

Substituting (1) in the above equation, the above equation gives

A = A ⋂ (C1 U C2 U C3 U ………… U Cn)

A = (A ⋂ C1) U (A ⋂ C2) U (A ⋂ C3) U ……… U (A ⋂ Cn)

It is clear that A ⋂ Cp and A ⋂ Cq are the subsets of Cp and Cq respectively. Hence it is true that 

A ⋂ Cp and A ⋂ Cq are also disjoint for p ≠ q. So, the probability of event ‘A’ can be calculated as:

P(A) = P [(A ⋂ C1) U (A ⋂ C2) U (A ⋂ C3) U ……… U (A ⋂ Cn)]

P(A) = P (A ⋂ C1) + P (A ⋂ C2) + P (A ⋂ C3) + ……… + P (A ⋂ Cn) → (3)

However, the multiplication rule of probability states that,

P (A ⋂ Cp) = P (Cp) . P (A | Cp)  → (4) 

Substituting (3) in (4), we get

P (A) =  P (C1) . P (A | C1) +  P (C2) . P (A | C2) +  P (C3) . P (A | C3) + …. +  P (Cn) . P (A | Cn)

The above equation can be shortly written as:

P(A) = \[\sum_{i = 1}^{n}\] P(Ci).P(A | Ci)

Total Probability Theorem Examples: 

1. Sharon has three bags with 100 marbles in each bag. The first bag has 25 blue and 75 red marbles, the second bag has 40 blue and 60 red marbles, and the third bad consists of 55 blue and 45 red marbles. A marble is randomly chosen from one of the bags. What is the probability that it is a red marble? (Hint: Consider it as a total probability theorem example)

Solution:

Let ‘A’ represent the event of choosing a red marble and Ci represents the event of choosing the bag. From the given data, we can infer the following.

Probability of choosing a red marble from bag 1 is 

P (A | C1) = 75 / 100 = 0.75

Probability of choosing a red marble from bag 2 is 

P (A | C2) = 60 / 100 = 0.6

Probability of choosing a red marble from bag 3 is 

P (A | C3) = 45 / 100 = 0.45

The probability of choosing one among the three bags remains the same.

P (Ci) = ⅓ 

The formula representing total probability theorem proof is 

P(A) = \[\sum_{i = 1}^{n}\] P(Ci).P(A | Ci)

So, the probability of choosing a red ball is calculated as:

P(A) = \[\sum_{i = 1}^{3}\] P(Ci).P(A | Ci)

P(A) = P(C1).P(A | C1) + P(C2).P(A | C2) + P(C3).P(A | C3)

P(A) = ⅓(0.75) + ⅓(0.6) + ⅓(0.45) = 0.25 + 0.2 + 0.15 = 0.6

Therefore, the probability of choosing a red ball at a random from one of the three bags is 0.6

Fun Facts about Total Probability Theorem-proof:

In the theory of probability, conditional probability is calculated when the occurrence of one event is possible only when the other event has already occurred.

Marginal probability is calculated when the events in a sample space are independent of each other. The events do not depend on each other in case of marginal probability. 

Total probability theorem-proof establishes a relationship between the conditional probability and the marginal probability and defines the probability of an event as the sum of the probabilities of other events in the sample space.

The decision tree is a simple and easy way to look at problems through a complete Probability law strategy. The decision tree shows all the possible events in sequence. Using the decision tree, you can quickly identify relationships between events and calculate conditional probabilities.

To understand how you can use the decision tree in calculating full potential, consider the following example:

• The person in consideration is a stock analyst following ABC Corp. He found that the company was planning to launch a new project that could affect the company's stock price. Identify the following probabilities:

• There is a 60% chance of starting a new project.

• When a company launches a project, there is a 75% chance that its stock will increase.

• If a company does not start a project, there is a 30% chance that its stock will increase.

Then the chances of the second card becoming king or not will be represented by the law of full probabilities such as:

P (E) = P (A) P (E | A) + P (B) P (E | B)

There,

• P (E) chances of the second card being king,

• P (A) chances that the first card is king,

• P (E | A) is the chance that the second card is king given that the first card is king,

• P (B) is the chance that the first card is not king,

• P (E | B) is the chance that the second card is king but the first card issued is not king.

Explanation of Total Probability Theorem with the Following Examples:

• The doctor reaches out to the patient for treatment, takes different modes of transport, and his or her chances of arriving on time are equal to the reduction of his or her chances of arriving on time, from different modes of travel.

• The student must represent the school in an external competition. The chances of selecting a student in a competition are the same as shortening the chances of selecting that student in the different classes in the school.

• The chances of finding a defective mango are shortening the chances of finding a defective mango in different mango boxes.

The foundation of Bayes' Theorem is the theorem of total probability which assists the students in deriving the reverse probability due to which an event takes place from partitions of a sample S. The calculation of the happening of an event due to the partition of the defined space is done by using the theorem of total Probability,   and the reverse probability from the particular partition of the sample space of S, due to the given Probability of the event can be ascertained with the assistance of the Bayes' Theorem. Therefore, this is completed by deriving the theorem of total Probability.