We go over the definitions of independence of events and conditional probability. In addition to that we gonna deal with sampling with / without replacement.
1. Independence of events
The necessary and sufficient condition of that two events are independent each other is,
P(A ∩ B) = P(A) * P(B) ⇔ P(B ∩ A) = P(B) * P(A)
ex1)
If we toss a coin more than once, or roll a die more than once, then you may assume that different tosses or rolls are independent. More precisely, if we roll a fair six-sided die twice, then the probability of getting 4 on the first throw and 5 on the second is 1/36, since we assume that all 36 combinations of the two throws are equally likely. But (1/36) = (1/6)·(1/6), and the separate probabilities of getting 4 on the first throw and of getting 5 on the second are both equal to 1/6. So the two events are independent. This would work just as well for any other combination.
ex2)
(a) I roll a fair 6-sided die. Let A be the event that the number is 3 or
smaller, and B the event that it is even. Then A = {1,2,3}, B = {2,4,6}, A∩B = {2};
so P(A) = 1/2, P(B) = 1/2, and P(A∩B) = 1/6. So the events are not independent.
(b) I roll a fair 6-sided die twice. Let A be the event that the number on the first roll is 3 or smaller, and B the event that the number on the second roll is even. These events should be independent, since they depend on different rolls. Let us see. We have:
A = {(1,1),…,(1,6),(2,1),…,(2,6),(3,1),…,(3,6)},
B = {(1,2),…,(6,2),(1,4),…,(6,4),(1,6),…,(6,6)},
A∩B = {(1,2),(1,4),(1,6),(2,2),(2,4),(2,6),(3,2),(3,4),(3,6),
P(A) = 18/36 = 1/2, P(B) = 18/36 = 1/2, P(A∩B) = 9/36 = 1/4.
So the events are independent.
2. Sampling with / without replacement
I have four pens in my satchel; they are red, green, blue, and purple. I
sample two pens. Let A be the event that the first pen is red or green, and B the event that the second pen is red or green.
Are A and B independent?
① Sampling with replacement
S = {RR, RG, RB, RP, GR, GG, GB, GP, BR, BG, BB, BP, PR, PG, PB, PP}
A = {RR, RG, RB, RP, GR, GG, GB, GP}
B = {RR, RG,GR, GG, BR, BG, PR, PG}
A ∩ B = {RR, RG, GR, GG}
p(A ∩ B) = 4/16 = 1/4, P(A) = 8/16 = 1/2, P(B) = 8/16 = 1/2,
∴ P(A∩B) = P(A) * P(B), Event A and Event B are independent.
② Sampling without replacement
S = {RG, RB, RP, GR, GB, GP, BR, BG, BP, PR, PG, PB}
A = {RG, RB, RP, GR, GB, GP}
B = {RG,GR, BR, BG, PR, PG}
A ∩ B = {RG, GR}
p(A ∩ B) = 2/12 = 1/6, P(A) = 6/12 = 1/2, P(B) = 6/12 = 1/2,
∴ P(A∩B) ≠ P(A) * P(B), Event A and Event B are not independent.
3. Conditional Probability
The conditional probability of B given A is defined as
P(B|A) = \dfrac {P(B \cap A)}{P(A)}
To check the formula in our previous example
① with replacement : P(B|A) = P(B∩A) / P(A) = (1/4) / (1/2) = 1/2
② without replacement : P(B|A) = P(B∩A) / P(A) = (1/6) / (1/2) = 1/3
Conditional probability is used in two distinct ways:
(a) given P(A) and P(A∩B), calculate the conditional probability P(B | A)
(b) given P(A) and P(B | A), calculate P(A∩B) from the rule
P(A∩B) = P(A)×P(B | A).
There is an funny example.
Alice and Bob are going out to dinner. They toss a fair coin ‘best of three’ to decide who pays: if there are more heads than tails in the three tosses then Alice pays, otherwise Bob pays.
Clearly each has a 50% chance of paying.
The sample space is,
S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
A = “‘Alice pays” = {HHH, HHT, HTH, THH}
B = “Bob pays” ={HTT, THT, TTH, TTT}
They toss the coin once and the result is heads; call this event E. How should we
now reassess their chances? We have
E = {HHH, HHT, HTH, HTT}
P(A|E) = P(A∩E) / P(E) = P({HHH, HHT, HTH}) / 1/2 = (3/8) / (1/2) = 3/4
P(B|E) = (B∩E) / P(E) = P({HTT}) / 1/2 = (1/8) / (1/2) = 1/4