4.11 Conditional probability and independence

New information changes probabilities. The chance that a randomly chosen person is over $1.90$ m tall is small, but it is much larger if the person is known to be a professional basketball player. A conditional probability measures the chance of one event when another event is known to have occurred, and it is the basis of medical diagnosis, weather forecasting and risk assessment.

Conditional probability also gives a precise meaning to the idea that two events are unrelated. Two events are independent exactly when knowing that one has occurred leaves the probability of the other unchanged.

Conditional probability

The idea of a reduced sample space

In 4.6, "given that" probabilities were read from diagrams by restricting attention to one group. When it is known that $B$ has occurred, every outcome outside $B$ is ruled out, so $B$ becomes the new sample space. The event $A$ can now occur only through the outcomes it shares with $B$, which are the outcomes in $A \cap B$.

With equally likely outcomes, the probability of $A$ given $B$ is therefore the number of outcomes in $A \cap B$ divided by the number in $B$:

Dividing the top and bottom by $n(U)$ turns the counts into probabilities, which gives the general definition.

The vertical line is read "given". The formula is in the formula booklet. It is only defined when $P(B) > 0$, since it makes no sense to condition on an event that cannot happen. A conditional probability is still a probability, so it lies between $0$ and $1$, and $P(A \mid B) + P(A' \mid B) = 1$.

Using the formula with given probabilities

Conditional probability from a sample space

When the outcomes are equally likely, conditional probabilities can be found by counting within the given event. This is often quicker than the formula, and it is a good check on it.

Conditional probability from a two-way table

In a two-way table, the given event picks out a row or a column. Its total becomes the denominator.

A café records the drink ordered by each of $300$ customers and whether it was ordered in the morning or the afternoon.

The multiplication rule and tree diagrams

Rearranging the definition

Multiplying both sides of the definition by $P(B)$ gives a rule for the probability that both events occur:

This is the rule used on every tree diagram. The first set of branches gives $P(A)$ and $P(A')$, and the second set gives conditional probabilities such as $P(B \mid A)$, because each second-stage branch describes what happens after the first stage. Multiplying along a path is exactly the multiplication rule.

The multiplication rule is equally useful without a tree. Given one conditional probability and the probability of the condition, it gives the intersection, and from there the addition rule and the definition give everything else.

Finding the probability of the second event

The probability that the second event occurs, whatever happened first, is found by adding every path that ends in it. On a two-stage tree this is

The two paths are mutually exclusive, because the first stage cannot give both $A$ and $A'$.

Reversing a conditional probability

A tree diagram is drawn in the order the events happen, so its branches give probabilities such as $P(L \mid R)$: the bus is late given that it rains. A question may instead ask about the reverse: given that the bus was late, what is the probability that it rained? This reverse conditional is found with the definition, taking both the numerator and the denominator from the tree.

The numerator is the single path through $R$ and $L$. The denominator is the sum of all the paths that end in $L$.

The same method answers questions about which of two causes produced an observed result.

The general form of this reasoning, for any number of causes, is Bayes' theorem, which HL students meet in 4.13. With two events and a tree, the definition of conditional probability is all that is needed.

Independent events

The definition

In 4.6, independent events were described as events that do not affect each other, and the rule $P(A \cap B) = P(A)P(B)$ was used for them. Conditional probability makes this precise. If knowing that $B$ has occurred does not change the probability of $A$, then $P(A \mid B) = P(A)$.

The two forms are equivalent. If $P(A \mid B) = P(A)$, then substituting into the definition of conditional probability gives $\frac{P(A \cap B)}{P(B)} = P(A)$, so $P(A \cap B) = P(A)P(B)$. Reversing the steps shows that the product rule implies $P(A \mid B) = P(A)$. The condition $P(A \mid B') = P(A)$ says that learning that $B$ did not occur also leaves $A$ unchanged, which follows from the product rule in the same way.

For independent events the rule works in both directions: $P(B \mid A) = P(B)$ as well. Dependent events are the ones for which at least one of these equalities fails, and then all of them fail.

Testing for independence

To determine whether two events are independent, calculate both sides of one of the equivalent conditions and compare them.

One test is enough: if one condition holds, they all hold. To show that events are not independent, a single condition that fails is also enough.

When the information is in a table, the conditional form of the test compares the probability of an event within one group with its probability in the whole sample.

With real data, two relative frequencies are rarely exactly equal, even when the events are unrelated, because samples vary. A large difference, as in the café data, suggests dependence; a very small one suggests that independence is a reasonable model. Deciding how large a difference must be before it counts as evidence of dependence needs a statistical test, such as the chi-squared test of independence.

Independence of complements

If $A$ and $B$ are independent, then $A$ and $B'$ are independent, and so are $A'$ and $B'$. This is used often: for example, if hitting a target with two shots are independent events, then so are missing with each.

Problems with an unknown probability

When independence is given, the product rule supplies an extra equation, which can be combined with the addition rule to find an unknown probability.

Several independent events

The product rule extends to any number of independent events: if $A$, $B$ and $C$ are independent, then $P(A \cap B \cap C) = P(A)P(B)P(C)$. This is how the reliability of a system built from separate components is calculated, and it is the basis of the binomial distribution in 4.8.

Independence is a modelling assumption here. If all three components were affected by the same power surge, their failures would be linked, and the true probability that the machine works could be quite different from $0.684$.

Exam-style questions on conditional probability

Conditional probability questions usually build a tree or a table in the first part and then ask for a reverse conditional and a test for independence. Labelling events with letters at the start makes every later line shorter and clearer.

The second question uses a context and allows a graphic display calculator (GDC), but the reasoning is the same.

Self-check

• To do:

  I can state the definition $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$ and explain it as a reduced sample space.

• To do:

  I can find conditional probabilities from given probabilities, sample spaces, Venn diagrams and two-way tables.

• To do:

  I can explain why $P(A \mid B)$ and $P(B \mid A)$ are usually different.

• To do:

  I can use $P(A \cap B) = P(A)P(B \mid A)$ and recognise that second-stage tree branches are conditional probabilities.

• To do:

  I can find the total probability of an event from a tree and reverse a conditional probability.

• To do:

  I can test whether two events are independent, using $P(A \mid B) = P(A)$ or $P(A \cap B) = P(A)P(B)$, and justify my conclusion with values.

• To do:

  I can prove that if $A$ and $B$ are independent then so are $A$ and $B'$, and use independence to find an unknown probability.