Bayes Theorem

Bayes theorem explains the probability of an event based on the conditions responsible for that event. Bayes theorem finds its application in statistical computations, Bayesian inference, probability interpretations etc. The statistical computations that use Bayes theorem are grouped under a separate category of statistics called Bayesian Statistics.

Bayes Theorem Formula:

If A and B are two independent events, then the probability of event A when B is true is calculated as the ratio of probability of event B such that event A is true and the individual probability of event A to the individual probability of event B. Bayes theorem Formula is written as:

\[P\left( {A/B} \right) = \frac{{P\left( {B/A} \right).P\left( A \right)}}{{P\left( B \right)}}\]

In the above mentioned Bayes Theorem Formula, 

P (A | B) is the probability of event A being true when event B is true.

P (B | A) is the probability of event B being true when event A is true.

P (A) is the probability of event A being true.

P (B) is the probability of event B being true.

An Important Concept Required to State and Prove Bayes Theorem:

Conditional probability is the probability of one event when one or more other individual events are true. It can be better explained with the help of an example. 

Suresh visits a library in which one of the book racks contains 3 rows. All the three rows are stacked with a mixture of reference books, journals and annual reports. Let us consider that Suresh picks a book from the second rack. The probability of whether the book picked by Suresh is a reference material depends on the other two events. (i.e. whether the book is a journal or an annual magazine). In general, conditional probability means the measure of probability of one event when the other event is true. 

\[P\left( {A/B} \right) = \frac{{P\left( {A \cap B} \right)}}{{P\left( B \right)}}\]

where , A and B are two individual events and P (B) not equal to zero.

P (A | B) is the probability of event A being true when event B is true.

P (A ⋂ B) is the probability of occurrence of both A and B.

P (B) is the probability of individual event B.

How to State and Prove Bayes Theorem:

Bayes theorem formula is stated as 

\[P\left( {A/B} \right) = \frac{{P\left( {B/A} \right).P\left( A \right)}}{{P\left( B \right)}}\]

Bayes theorem proof can be derived using the concept of conditional probability. The probability of occurrence of both the events A and B is given in terms of their individual probabilities and conditional probability as:

\[P{\text{ }}\left( {A{\text{ }} \cap {\text{ }}B} \right){\text{ }} = {\text{ }}P{\text{ }}\left( A \right).{\text{ }}P{\text{ }}\left( {B{\text{ }}|{\text{ }}A} \right)\]

Similarly the occurrence of both the events simultaneously can also be given in terms of the probability of second event as:

\[P{\text{ }}\left( {A{\text{ }} \cap {\text{ }}B} \right){\text{ }} = {\text{ }}P{\text{ }}\left( B \right).{\text{ }}P{\text{ }}\left( {A{\text{ }}|{\text{ }}B} \right)\]

In both the equations, the left hand side is equal. So RHS can be equated.

\[P{\text{ }}\left( B \right).{\text{ }}P{\text{ }}\left( {A{\text{ }}|{\text{ }}B} \right){\text{ }} = {\text{ }}P{\text{ }}\left( A \right).{\text{ }}P{\text{ }}\left( {B{\text{ }}|{\text{ }}A} \right)\]

Further simplification gives the Bayes theorem formula as 

\[P\left( {A/B} \right) = \frac{{P\left( {B/A} \right).P\left( A \right)}}{{P\left( B \right)}}\]

Bayes Theorem Problems:

1. Two boxes are placed in a cupboard out of which the first box contains 1 black and 3 red balls and the second box contains 4 black and 2 red balls.A ball picked at random from one of the boxes is found to be black. What is the probability that the ball is drawn from the first box?

Solution:

Let ‘A’ be the event of choosing a black ball

Let E1 and E2 represent the events of choosing a ball from the first and second box respectively. Probability of choosing the ball from first or second box is given by:

\[P{\text{ }}\left( {E1} \right){\text{ }} = {\text{ }}P{\text{ }}\left( {E2} \right){\text{ }} = {\text{ }}\frac{1}{2}\]

Probability of drawing a black ball from the first box is P(A | E1) = ¼ 

Probability of drawing a black ball from the second box is \[P\left( {A|E2} \right) = \frac{4}{6} = \frac{2}{3}\]

\[P(E1|A) = \frac{P(A|E1) . P(E1)}{P(A|E1) . P(E1) + P(A|E2) . P(E2)}\]

\[P(E1|A) = \frac{\frac{1}{4}\times \frac{1}{2}}{\frac{1}{4}\times \frac{1}{2} + \frac{2}{3}\times \frac{1}{2}}\]

\[P(E1|A) = \frac{\frac{1}{8}}{\frac{1}{8} + \frac{1}{3}} = \frac{\frac{1}{8}}{\frac{3}{24} + \frac{8}{24}} = \frac{\frac{1}{8}}{\frac{11}{8}} = \frac{3}{11}\]

The probability of choosing a black ball from the first box is \[\frac{3}{{11}}\].

2. In a gathering of 100 members, 25 of them wore black suits out of which 5 were men. Remaining 75 members are dressed up in a different attire out of which 35 are men. Find the probability that the person wearing pink is a man? (Use the above situation as a Bayes theorem example).

Solution:

Given: 

Total members in the party = 100

Number of men wearing black suits = 5

Number of men not wearing black suit = 35

Total Men = 35 + 5 = 40

If the event of the member being a man is ‘A’, then probability of member being man is 

P (A) = Number of men / Total members

P (A) = 40 / 100

P (A) = ⅖ 

If the event of member wearing black suit is ‘B’, then the probability of member wearing black suit is 

P (B) = Number of members wearing black suit / Total members

P (B) = 25 / 100

P (B) = ¼ 

Probability of men wearing black suit is calculated as

P (B | A) = Number of men wearing black suit / Total number of men

P (B | A) = 5 / 40

P (B | A) = ⅛

In the given Bayes theorem example, probability of members wearing black suit being men is calculated using Bayes theorem proof as 

\[P\left( {A/B} \right) = \frac{{P\left( {B/A} \right).P\left( A \right)}}{{P\left( B \right)}}\]

\[P\left( {A|B} \right) = \frac{{\frac{1}{8} \times \frac{2}{5}}}{{\frac{1}{4}}} = \frac{1}{5}\]

The probability that a member wearing black suits being a man is\[\frac{1}{5}\].

Fun facts About Bayes Theorem Formula:

• Reverend Thomas Bayes, in his work “An Essay towards solving a Problem in the Doctrine of Chances”, published in the year 1763, was the first person to use conditional probability in one of his algorithms. So, the theorem is named after him as Bayes theorem.

• Bayes theorem problems give the relationship between independent and dependent probabilities of any two events.