Chapter 4 Probability Concepts and Conditional Probability
4.1 Learning goals
After this chapter, you should be able to:
- describe random experiments, outcomes, sample spaces, and events;
- construct a probability function on a discrete sample space;
- apply the probability axioms and their consequences;
- calculate and interpret conditional probabilities;
- use multiplication rules, the law of total probability, and Bayes’ theorem;
- distinguish independence from mutual exclusivity;
- integrate these ideas in a real-data case study.
4.2 Basic Probability Concepts
Probability provides a numerical description of uncertainty. A model begins by identifying the possible outcomes and then assigning probability in a consistent way.
4.2.1 Random experiments, outcomes, sample spaces, and events
A random experiment is a repeatable process whose result is uncertain before it is performed. An outcome is one possible result. The sample space \(S\) is the set of all possible outcomes, and an event is any subset of \(S\).
The sample space must contain enough detail to answer the question being studied. An event occurs when the observed outcome belongs to that event.
An outcome is what happens in one performance of the experiment. An event is a statement about the outcome and may contain one, several, all, or none of the possible outcomes.
Example: one die roll
For one six-sided die roll, \(S=\{1,2,3,4,5,6\}\). If \(E=\{2,4,6\}\) means “even” and \(G=\{5,6\}\) means “greater than 4,” then \(E\cap G=\{6\}\).
Example: two insurance claims
Classify each of two claims as accepted (\(A\)) or denied (\(D\)), recording their order:
\[S=\{AA,AD,DA,DD\}.\]
The events “both accepted,” “exactly one accepted,” and “at least one accepted” are
\[B=\{AA\},\qquad L=\{AD,DA\},\qquad B\cup L=\{AA,AD,DA\}.\]
The outcomes \(AD\) and \(DA\) are different because the claims are distinguished by submission order.
Example: a continuous sample space
If the experiment records a claim amount \(X\), a natural sample space is \(S=[0,\infty)\). The event “the claim exceeds QAR 10,000” is \((10{,}000,\infty)\). This section first focuses on discrete spaces; continuous models are developed later.
Exercises: experiments and events
Exercise 1. Two coins are tossed in order. Write the sample space and the events: (a) exactly one head; (b) at least one head; (c) no heads.
Show solution to Exercise 1
\(S=\{HH,HT,TH,TT\}\). The events are (a) \(\{HT,TH\}\); (b) \(\{HH,HT,TH\}\); and (c) \(\{TT\}\).Exercise 2. A policyholder is classified by age—young (\(Y\)), middle-aged (\(M\)), or senior (\(R\))—and by claim status—filed (\(C\)) or not filed (\(N\)). Construct \(S\) and the event “senior or filed a claim.”
Show solution to Exercise 2
\(S=\{YC,YN,MC,MN,RC,RN\}\). The event is \(\{YC,MC,RC,RN\}\).Exercise 3. For one die roll let \(A=\{1,2,3\}\) and \(B=\{2,4,6\}\). List \(A\cup B\), \(A\cap B\), \(A-B\), and \(A^c\).
Show solution to Exercise 3
\(A\cup B=\{1,2,3,4,6\}\), \(A\cap B=\{2\}\), \(A-B=\{1,3\}\), and \(A^c=\{4,5,6\}\).4.2.2 Probability function for a discrete probability space and probability axioms
When \(S\) is finite or countably infinite, probability can be assigned first to individual outcomes and then accumulated over events.
A discrete probability function assigns to every \(\omega\in S\) a mass \(p(\omega)\) such that
\[p(\omega)\ge0,\qquad \sum_{\omega\in S}p(\omega)=1.\]
For an event \(A\subseteq S\),
\[\mathbb{P}(A)=\sum_{\omega\in A}p(\omega).\]
The function \(p\) acts on elementary outcomes; \(\mathbb{P}\) acts on events. In particular, \(\mathbb{P}(\{\omega\})=p(\omega)\).
Example: a loaded die
Suppose \(p(k)=ck\) for faces \(k=1,\ldots,6\). Normalization gives \(21c=1\), so \(c=1/21\). Hence
\[\mathbb{P}(\text{even})=\frac{2+4+6}{21}=\frac47.\]
Counting favorable faces alone would be wrong because the faces are not equally likely.
Probability axioms
For events in \(S\), a probability function satisfies:
- Non-negativity: \(\mathbb{P}(A)\ge0\);
- Normalization: \(\mathbb{P}(S)=1\);
- Countable additivity: for pairwise disjoint \(A_1,A_2,\ldots\), \[\mathbb{P}\!\left(\bigcup_{i=1}^{\infty}A_i\right)=\sum_{i=1}^{\infty}\mathbb{P}(A_i).\]
These axioms express consistency: probability cannot be negative, some outcome in \(S\) must occur, and mutually exclusive events can be added without double-counting.
Particular case: uniform probability function
If a finite sample space \(S\) has \(N\) equally likely outcomes, the uniform probability function is
\[p(\omega)=\frac1N,\]
and therefore
\[\mathbb{P}(A)=\frac{n(A)}{n(S)}.\]
Uniformity is an assumption, not an automatic fact. It is appropriate for a fair die or a thoroughly shuffled deck, but not for a loaded die or a choice influenced by preferences.
Example: sampling claims
Select two claims uniformly without replacement from 8 paid and 4 denied claims. Every unordered pair is equally likely, so
\[\mathbb{P}(\text{both paid})=\frac{\binom82}{\binom{12}{2}}=\frac{14}{33}.\]
Exercises: discrete probability functions
Exercise 1. On \(S=\{1,2,3,4\}\) let \(p(k)=ck^2\). Find \(c\) and \(\mathbb{P}(\{2,4\})\).
Show solution to Exercise 1
\(30c=1\), so \(c=1/30\) and \(\mathbb{P}(\{2,4\})=(4+16)/30=2/3\).Exercise 2. Do \(0.15,0.35,0.40,0.20\) define a probability function on a four-outcome space?
Show solution to Exercise 2
No. Although all masses are non-negative, they sum to \(1.10\), not 1.Exercise 3. A card is selected uniformly from 52 cards. Find the probability of a heart or a king.
Show solution to Exercise 3
There are \(13+4-1=16\) favorable cards, so the probability is \(16/52=4/13\).4.2.3 Rules concerning probability
The axioms imply several practical rules. Each rule corresponds to a set operation, which is why careful event notation matters.
Impossible, certain, and complement events
\[\mathbb{P}(\varnothing)=0,\qquad \mathbb{P}(S)=1,\qquad \mathbb{P}(A^c)=1-\mathbb{P}(A).\]
The complement rule is particularly useful for “none” and “at least one.” If three independent policyholders each have claim probability 0.20,
\[\mathbb{P}(\text{at least one claim})=1-(0.80)^3=0.488.\]
Monotonicity and difference
If \(A\subseteq B\), then \(\mathbb{P}(A)\le\mathbb{P}(B)\). For any \(A,B\),
\[\mathbb{P}(A-B)=\mathbb{P}(A)-\mathbb{P}(A\cap B).\]
The difference rule removes from \(A\) the part shared with \(B\). If \(B\subseteq A\), it reduces to \(\mathbb{P}(A-B)=\mathbb{P}(A)-\mathbb{P}(B)\).
Addition rule
For any two events,
\[\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B).\]
If \(A\cap B=\varnothing\), then \(\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)\).
The intersection is subtracted because it was counted in both individual probabilities.
Inclusion–exclusion and the union bound
For three events,
\[ \begin{aligned} \mathbb{P}(A\cup B\cup C)={}&\mathbb{P}(A)+\mathbb{P}(B)+\mathbb{P}(C)\\ &-\mathbb{P}(A\cap B)-\mathbb{P}(A\cap C)-\mathbb{P}(B\cap C)\\ &+\mathbb{P}(A\cap B\cap C). \end{aligned} \]
When intersections are unknown, the union bound gives
\[\mathbb{P}\!\left(\bigcup_i A_i\right)\le\sum_i\mathbb{P}(A_i).\]
It is an upper bound because overlaps may have been counted repeatedly.
Example: insurance coverages
If 46% have life coverage, 38% disability coverage, and 21% both, then
\[\mathbb{P}(L\cup D)=0.46+0.38-0.21=0.63,\]
so the probability of neither coverage is \(1-0.63=0.37\).
Exercises: probability rules
Exercise 1. If \(\mathbb{P}(A)=0.62\), find \(\mathbb{P}(A^c)\).
Show solution to Exercise 1
\(\mathbb{P}(A^c)=1-0.62=0.38\).Exercise 2. If \(\mathbb{P}(A)=0.55\), \(\mathbb{P}(B)=0.40\), and \(\mathbb{P}(A\cap B)=0.18\), find \(\mathbb{P}(A\cup B)\), \(\mathbb{P}(A-B)\), and the probability of neither.
Show solution to Exercise 2
\(\mathbb{P}(A\cup B)=0.77\), \(\mathbb{P}(A-B)=0.37\), and the probability of neither is \(0.23\).Exercise 3. Risk events have probabilities 0.04, 0.07, and 0.03. With no intersection information, give an upper bound for the probability that at least one occurs.
Show solution to Exercise 3
The union bound gives \(\mathbb{P}(A_1\cup A_2\cup A_3)\le0.04+0.07+0.03=0.14\).Exercise 4. Suppose \(\mathbb{P}(A)=0.30\), \(\mathbb{P}(B)=0.25\), \(\mathbb{P}(C)=0.20\), the pairwise intersections are 0.10, 0.08, and 0.05, and \(\mathbb{P}(A\cap B\cap C)=0.02\). Find the probability of at least one event.
Show solution to Exercise 4
Inclusion–exclusion gives \(0.30+0.25+0.20-0.10-0.08-0.05+0.02=0.54\).4.2.4 Practice exercises
The following problems combine set operations, complements, inclusion–exclusion, and discrete probability functions. Each solution is hidden so that you can attempt the problem first.
Exercise 1 — Walking and biking. In a survey, 80% like walking, 60% like biking, and everyone likes at least one activity. Find the probability that a randomly selected person likes biking but not walking.
Show solution to Exercise 1
Let \(W\) and \(B\) denote the two preferences. Since \(\mathbb{P}(W\cup B)=1\), \[\mathbb{P}(W\cap B)=0.80+0.60-1=0.40.\] Therefore \(\mathbb{P}(B-W)=0.60-0.40=0.20\).Exercise 2 — Treatment visits. Among patients recovering from shoulder injuries, 22% visit both a physical therapist and a chiropractor, while 12% visit neither. The probability of visiting a chiropractor is 0.14 greater than the probability of visiting a physical therapist. Find the probability of visiting a physical therapist.
Show solution to Exercise 2
Let \(x=\mathbb{P}(P)\), so \(\mathbb{P}(C)=x+0.14\) and \(\mathbb{P}(P\cup C)=0.88\). Thus \[0.88=x+(x+0.14)-0.22,\] which gives \(2x=0.96\) and \(x=0.48\).Exercise 3 — Supplementary coverages. Employees choose exactly two of coverages \(A,B,C\), or choose none. The proportions choosing \(A,B,C\) are \(1/4\), \(1/3\), and \(5/12\). Find the probability of choosing no supplementary coverage.
Show solution to Exercise 3
Every employee with coverage is counted exactly twice in \[\mathbb{P}(A)+\mathbb{P}(B)+\mathbb{P}(C)=\frac14+\frac13+\frac5{12}=1.\] Hence \(2\mathbb{P}(A\cup B\cup C)=1\), so the probability of some coverage is \(1/2\) and the probability of none is \(1/2\).Exercise 4 — Policyholder classification. An insurer has 10,000 policyholders. There are 3,000 young, 4,600 male, and 7,000 married policyholders; 1,320 are young males, 3,010 are married males, 1,400 are young married, and 600 are young married males. How many are young, female, and single?
Show solution to Exercise 4
Within the young group: young married males number 600; young single males number \(1320-600=720\); and young married females number \(1400-600=800\). Therefore young single females number \[3000-600-720-800=880.\]Exercise 5 — Medical referrals. The probability that a primary-care visit results in neither lab work nor specialist referral is 0.35. Referral occurs with probability 0.30 and lab work with probability 0.40. Find the probability of both.
Show solution to Exercise 5
\(\mathbb{P}(L\cup R)=1-0.35=0.65\). Hence \[\mathbb{P}(L\cap R)=0.40+0.30-0.65=0.05.\]Exercise 6 — Two complementary unions. Given \(\mathbb{P}(A\cup B)=0.70\) and \(\mathbb{P}(A\cup B^c)=0.90\), find \(\mathbb{P}(A)\).
Show solution to Exercise 6
The disjoint regions \(B-A\) and \(B^c-A\) partition \(A^c\). Therefore \[\mathbb{P}(A\cup B)+\mathbb{P}(A\cup B^c)=1+\mathbb{P}(A).\] Thus \(0.70+0.90=1+\mathbb{P}(A)\) and \(\mathbb{P}(A)=0.60\).Exercise 7 — Sports audiences. Suppose 28% watched gymnastics, 29% baseball, and 19% soccer. The pairwise intersection percentages are 14%, 12%, and 10%, and 8% watched all three. Find the percentage who watched none.
Show solution to Exercise 7
Inclusion–exclusion gives \[\mathbb{P}(G\cup B\cup S)=0.28+0.29+0.19-0.14-0.12-0.10+0.08=0.48.\] Therefore 52% watched none.Exercise 8 — Number of annual claims. At most five claims may be filed. Let \(p_n\) be the probability of \(n\) claims, for \(n=0,\ldots,5\). Suppose \(p_n\ge p_{n+1}\), the difference \(p_n-p_{n+1}\) is constant, and \(p_0+p_1=0.40\). Find \(\mathbb{P}(N>3)\).
Show solution to Exercise 8
Write \(p_n=a-nd\). From \(p_0+p_1=0.40\), \(2a-d=0.40\). Normalization gives \[\sum_{n=0}^5(a-nd)=6a-15d=1.\] Solving yields \(d=1/60\) and \(a=5/24\). Hence \[\mathbb{P}(N>3)=p_4+p_5=\frac{17}{120}+\frac{15}{120}=\frac4{15}\approx0.27.\]Exercise 9 — Homeowners’ claims. Liability- and property-claim probabilities are 0.04 and 0.10. The probability of a liability claim but no property claim is 0.01. Find the probability of neither type.
Show solution to Exercise 9
The intersection probability is \(0.04-0.01=0.03\). Thus \[\mathbb{P}(L\cup P)=0.04+0.10-0.03=0.11,\] and the probability of neither is \(1-0.11=0.89\).Exercise 10 — Mattress sales. A store sells only king, queen, and twin mattresses. Queen sales equal one-fourth of combined king and twin sales, and king sales are three times twin sales. Find the probability that the next mattress sold is king or queen.
Show solution to Exercise 10
Let twin sales have weight \(t\). King sales have weight \(3t\), and queen sales have weight \((3t+t)/4=t\). The total weight is \(5t\), of which king or queen contributes \(4t\). Therefore the probability is \(4/5=0.80\).4.3 Conditional probability
An unconditional probability uses the entire population or sample space. In practice, however, we often receive information before making a decision: a policyholder belongs to a particular age group, a company has missed an earnings target, or a diagnostic test is positive. Conditional probability measures uncertainty after restricting attention to cases consistent with that information.
4.3.1 Motivation: information changes the reference group
Suppose 200 policyholders are classified by age and whether they filed a claim.
| Age group | Filed a claim | No claim | Total |
|---|---|---|---|
| Under 30 | 36 | 44 | 80 |
| 30 or older | 24 | 96 | 120 |
| Total | 60 | 140 | 200 |
For a randomly selected policyholder,
\[\mathbb{P}(C)=\frac{60}{200}=0.30.\]
If we learn that the policyholder is under 30, the other 120 policyholders are no longer relevant. The new reference group contains only 80 people, of whom 36 filed a claim:
\[\mathbb{P}(C\mid U)=\frac{36}{80}=0.45.\]
The probability changed because the available information changed the denominator—not because the historical records themselves changed.
4.3.2 Definition and interpretation
For events \(A\) and \(B\) with \(\mathbb{P}(B)>0\), the conditional probability of \(A\) given \(B\) is
\[ \mathbb{P}(A\mid B)=\frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)}. \]
The vertical bar is read “given.” The event to the right of the bar is the information assumed to be true.
Conditioning performs two operations. First, discard outcomes outside \(B\). Second, rescale the probabilities inside \(B\) so their total becomes one. The numerator is the probability mass that belongs to both \(A\) and the restricted space \(B\).
The order matters. In general,
\[\mathbb{P}(A\mid B)\ne\mathbb{P}(B\mid A).\]
The two probabilities use different reference groups.
Example: a die with restricted information
Let \(A=\{2,4,6\}\) be the event “even” and \(B=\{4,5,6\}\) the event “greater than 3.” Given \(B\), only three outcomes remain possible, and two are even. Thus
\[\mathbb{P}(A\mid B)=\frac{\mathbb{P}(\{4,6\})}{\mathbb{P}(\{4,5,6\})}=\frac{2/6}{3/6}=\frac23.\]
In the reverse direction,
\[\mathbb{P}(B\mid A)=\frac{\mathbb{P}(\{4,6\})}{\mathbb{P}(\{2,4,6\})}=\frac23.\]
These happen to be equal because the two conditioning events have the same probability; this is not true in general.
Example: claim severity
Suppose 12% of policies produce a claim during the year, and 3% produce a claim exceeding QAR 50,000. Let \(C\) denote a claim and \(L\) a large claim. Since \(L\subseteq C\),
\[ \mathbb{P}(L\mid C)=\frac{\mathbb{P}(L)}{\mathbb{P}(C)}=\frac{0.03}{0.12}=0.25. \]
Interpretation: among policies that produced a claim, 25% produced a large claim. It does not mean that 25% of all policies produced a large claim.
4.3.3 The multiplication rule
Rearranging the definition gives a useful way to calculate intersections.
For events \(A\) and \(B\),
\[ \mathbb{P}(A\cap B) =\mathbb{P}(B)\mathbb{P}(A\mid B) =\mathbb{P}(A)\mathbb{P}(B\mid A). \]
Use this rule when a probability question describes successive stages: first \(B\) occurs, and then \(A\) occurs under the condition that \(B\) has occurred.
For three events, the chain rule is
\[ \mathbb{P}(A\cap B\cap C) =\mathbb{P}(A)\mathbb{P}(B\mid A)\mathbb{P}(C\mid A\cap B). \]
Example: sampling without replacement
An audit file contains 8 paid and 4 denied claims. Two claims are selected without replacement. The probability that both are paid is
\[ \mathbb{P}(P_1\cap P_2) =\mathbb{P}(P_1)\mathbb{P}(P_2\mid P_1) =\frac8{12}\frac7{11} =\frac{14}{33}. \]
The second factor is \(7/11\), not \(8/12\), because the first paid claim has been removed.
Example: market stress and default
Suppose the probability of a stressed market year is 0.20. In a stressed year, a bond defaults with probability 0.08. Then
\[ \mathbb{P}(\text{stress and default}) =0.20(0.08)=0.016. \]
Thus 1.6% of years are expected to contain both conditions under the model. The 8% figure applies only inside stressed years.
4.3.4 Reading conditional probability from a table
For a contingency table, the conditioning event determines the relevant row or column total:
\[ \mathbb{P}(A\mid B) =\frac{\text{count in the }A\cap B\text{ cell}} {\text{total count in the }B\text{ row or column}}. \]
Circle the event to the right of the conditioning bar before calculating. Its count or probability must be the denominator.
4.3.5 Conditional probability versus causation
A difference between \(\mathbb{P}(A\mid B)\) and \(\mathbb{P}(A)\) indicates that \(A\) and \(B\) are associated in the model or data. It does not by itself establish that \(B\) causes \(A\).
To understand the distinction, compare the two statements:
- Association: claim frequency is higher among drivers under 25 than among all drivers.
- Causation: being under 25 directly causes the increase in claim frequency.
The first statement can be assessed from conditional probabilities. The second requires a credible explanation of what would happen if the causal factor could be changed while other relevant conditions remained comparable.
Conditional probability answers “how often does \(A\) occur inside group \(B\)?” It does not answer “what would happen to \(A\) if we intervened and changed \(B\)?”
Several mechanisms can create an association without a direct causal effect:
- Confounding: a third variable influences both events. For example, driving experience may be related to both age and claim frequency.
- Selection bias: the observed data come from a selected group. Hospital patients, approved borrowers, or insured drivers may not represent the full population.
- Reverse direction: \(A\) may influence \(B\), rather than \(B\) influencing \(A\).
- Coincidence or instability: an apparent difference may arise from small samples or random variation.
Example: umbrellas and road accidents
Suppose accident probability is higher on days when more umbrellas are observed. It would be unreasonable to conclude that umbrellas cause accidents. Rain is a plausible confounder: it increases umbrella use and makes roads more hazardous.
Example: approved-loan data
A bank observes default only among approved applicants. Within that selected group, two financial characteristics may appear weakly associated even if they are strongly related in the full applicant population. Conditioning on approval changes the population and can create or hide relationships.
Use conditional probabilities to describe and predict patterns. Use causal language only when the study design and assumptions justify ruling out alternative explanations.
R implementation
# Policyholder table: rows are age groups, columns are claim status
tab <- matrix(c(36, 44, 24, 96), nrow = 2, byrow = TRUE,
dimnames = list(age = c("Under 30", "30+"),
status = c("Claim", "No claim")))
tab## status
## age Claim No claim
## Under 30 36 44
## 30+ 24 96## [1] 0.3## [1] 0.45## [1] 0.4242424Exercises: conditional probability
Exercise 1 — Contingency table. In the policyholder table above, find (a) \(\mathbb{P}(U\mid C)\); (b) \(\mathbb{P}(C\mid U^c)\); and (c) \(\mathbb{P}(U\cap C)\).
Show solution to Exercise 1
- Among 60 claim filers, 36 are under 30, so \(\mathbb{P}(U\mid C)=36/60=0.60\). (b) Among the 120 policyholders aged 30 or older, 24 filed claims, so \(\mathbb{P}(C\mid U^c)=24/120=0.20\). (c) \(\mathbb{P}(U\cap C)=36/200=0.18\).
Exercise 2 — Cards. A card is selected from a standard deck. Given that it is a face card, find the probability that it is a king.
Show solution to Exercise 2
There are 12 face cards—4 jacks, 4 queens, and 4 kings. Therefore \(\mathbb{P}(\text{king}\mid\text{face card})=4/12=1/3\).Exercise 3 — Claim severity. Suppose \(\mathbb{P}(C)=0.15\) and \(\mathbb{P}(L\mid C)=0.20\), where \(L\subseteq C\) denotes a large claim. Find \(\mathbb{P}(L)\).
Show solution to Exercise 3
By the multiplication rule, \(\mathbb{P}(L)=\mathbb{P}(C)\mathbb{P}(L\mid C)=0.15(0.20)=0.03\).Exercise 4 — Sampling claims. A file contains 6 fraudulent and 14 legitimate claims. Three are sampled without replacement. Find the probability that all three are legitimate.
Show solution to Exercise 4
\[\mathbb{P}(L_1\cap L_2\cap L_3)=\frac{14}{20}\frac{13}{19}\frac{12}{18}=\frac{91}{285}\approx0.3193.\]Exercise 5 — Reverse the condition. Suppose \(\mathbb{P}(A)=0.30\), \(\mathbb{P}(B)=0.50\), and \(\mathbb{P}(A\mid B)=0.20\). Find \(\mathbb{P}(A\cap B)\) and \(\mathbb{P}(B\mid A)\).
Show solution to Exercise 5
\(\mathbb{P}(A\cap B)=0.50(0.20)=0.10\). Therefore \(\mathbb{P}(B\mid A)=0.10/0.30=1/3\). This demonstrates that reversing the condition changes the denominator.Exercise 6 — Interpretation. A report states that \(\mathbb{P}(D\mid S)=0.06\), where \(D\) is default and \(S\) is a stressed market. Explain the probability in words and identify the reference group.
Show solution to Exercise 6
Among periods classified as stressed markets, 6% are associated with default. The reference group is the set of stressed-market periods, not all periods.4.3.6 Law of total probability and Bayes’ theorem
Many real populations contain distinct risk groups. Splitting the population into such groups lets us compute an overall probability as a weighted average.
If \(F_1,\ldots,F_k\) partition the sample space, then
\[\mathbb{P}(A)=\sum_{i=1}^k\mathbb{P}(A\mid F_i)\mathbb{P}(F_i).\]
Bayes’ theorem reverses the conditioning direction:
\[\mathbb{P}(F_j\mid A)=\frac{\mathbb{P}(A\mid F_j)\mathbb{P}(F_j)}{\sum_i\mathbb{P}(A\mid F_i)\mathbb{P}(F_i)}.\]
The total-probability formula moves from group-specific risks to an overall risk. Bayes’ theorem moves in the opposite direction: after observing \(A\), it revises how plausible each group is.
A reliable way to organize a problem is to separate three ingredients:
- the prior \(\mathbb{P}(F_i)\): how common group \(F_i\) is before observing new information;
- the likelihood \(\mathbb{P}(A\mid F_i)\): how likely the evidence is inside that group;
- the posterior \(\mathbb{P}(F_i\mid A)\): the revised group probability after observing the evidence.
The denominator in Bayes’ theorem is the total probability of the observed evidence. It adds every mutually exclusive route through which the evidence could occur.
A highly accurate signal can still have a modest posterior probability when the event of interest is rare. This is the base-rate effect: the prior probability matters alongside sensitivity and false-positive rate.
Risk classification. Suppose 30% of policyholders are accident-prone. Their one-year accident probability is 0.40, compared with 0.20 for other policyholders.
\[\mathbb{P}(A)=0.40(0.30)+0.20(0.70)=0.26,\]
and after an accident,
\[\mathbb{P}(\text{prone}\mid A)=\frac{0.40(0.30)}{0.26}\approx0.4615.\]
prior <- .30
p_accident <- .40 * prior + .20 * (1 - prior)
posterior <- .40 * prior / p_accident
c(accident = p_accident, prone_given_accident = posterior)## accident prone_given_accident
## 0.2600000 0.4615385Credit-warning example. A model flags 80% of firms that will default and 10% of firms that will not. With a 2% one-year default rate,
\[\mathbb{P}(D\mid +)=\frac{0.80(0.02)}{0.80(0.02)+0.10(0.98)}\approx0.1404.\]
The flag is informative, but default remains uncommon because its prior probability is small.
bayes_binary <- function(prior, sensitivity, false_positive) {
sensitivity * prior /
(sensitivity * prior + false_positive * (1 - prior))
}
bayes_binary(.02, .80, .10)## [1] 0.1403509Exercises: total probability and Bayes’ theorem
Exercise 1 — Two risk classes. An insurer classifies 35% of drivers as high risk. High-risk drivers file a claim with probability 0.18, while other drivers file a claim with probability 0.06. Find (a) the overall claim probability and (b) the probability that a claimant is high risk.
Show solution to Exercise 1
By total probability, \[\mathbb{P}(C)=0.35(0.18)+0.65(0.06)=0.102.\] Bayes’ theorem gives \[\mathbb{P}(H\mid C)=\frac{0.35(0.18)}{0.102}=\frac{21}{34}\approx0.6176.\]Exercise 2 — Diagnostic test. A disease affects 1% of a population. A test is positive for 95% of diseased people and 4% of non-diseased people. Find the probability that a person with a positive test has the disease.
Show solution to Exercise 2
\[\mathbb{P}(+)=0.95(0.01)+0.04(0.99)=0.0491.\] Thus \[\mathbb{P}(D\mid +)=\frac{0.95(0.01)}{0.0491}\approx0.1935.\] The posterior is well below 95% because the disease is rare and false positives arise from the much larger healthy group.Exercise 3 — Manufacturing lines. Lines A, B, and C produce 50%, 30%, and 20% of a company’s items. Their defect rates are 1%, 2%, and 4%. Find (a) the overall defect rate and (b) the probability that a defective item came from line C.
Show solution to Exercise 3
\[\mathbb{P}(D)=0.50(0.01)+0.30(0.02)+0.20(0.04)=0.019.\] Therefore \[\mathbb{P}(C\mid D)=\frac{0.20(0.04)}{0.019}=\frac8{19}\approx0.4211.\]Exercise 4 — Credit alert. A portfolio has a 3% default rate. An alert is triggered for 85% of defaults and 12% of non-defaults. Find (a) the probability of an alert; (b) the probability of default given an alert; and (c) the probability of default given no alert.
Show solution to Exercise 4
\[\mathbb{P}(+)=0.85(0.03)+0.12(0.97)=0.1419.\] Hence \[\mathbb{P}(D\mid +)=\frac{0.85(0.03)}{0.1419}\approx0.1797.\] Also \(\mathbb{P}(-)=0.8581\) and \[\mathbb{P}(D\mid -)=\frac{0.15(0.03)}{0.8581}\approx0.00524.\]Exercise 5 — Choosing the direction. Explain why sensitivity \(\mathbb{P}(+\mid D)\) is not the same quantity as the positive predictive value \(\mathbb{P}(D\mid +)\).
Show solution to Exercise 5
Sensitivity uses diseased individuals as its reference group and asks how many test positive. Positive predictive value uses positive tests as its reference group and asks how many truly have the disease. Their denominators differ, and the second quantity also depends on the disease prevalence.4.4 Independence
Independence means that learning whether one event occurred does not change the probability of the other. It is a property of the joint probability model, not simply a statement that two events look unrelated.
Events \(A\) and \(B\) are independent when
\[\mathbb{P}(A\cap B)=\mathbb{P}(A)\mathbb{P}(B).\]
If \(\mathbb{P}(B)>0\), this is equivalent to \(\mathbb{P}(A\mid B)=\mathbb{P}(A)\).
For events of positive probability, the following statements are equivalent:
\[ \mathbb{P}(A\cap B)=\mathbb{P}(A)\mathbb{P}(B), \]
\[ \mathbb{P}(A\mid B)=\mathbb{P}(A), \qquad \mathbb{P}(B\mid A)=\mathbb{P}(B). \]
To check independence, compute the intersection from the supplied information and compare it with the product of the marginal probabilities. A numerical equality is required; independence should not be assumed merely because the events concern different subjects.
Mutually exclusive events with positive probabilities cannot be independent. If \(A\cap B=\varnothing\), then \(\mathbb{P}(A\cap B)=0\), but \(\mathbb{P}(A)\mathbb{P}(B)>0\).
If market-crash and issuer-default probabilities are 0.05 and 0.03, independence predicts a joint probability of 0.0015. An observed/modelled joint probability of 0.006 indicates dependence.
4.4.1 Pairwise and mutual independence
For three events \(A,B,C\), pairwise independence requires independence of each pair. Mutual independence additionally requires
\[\mathbb{P}(A\cap B\cap C)=\mathbb{P}(A)\mathbb{P}(B)\mathbb{P}(C).\]
Pairwise independence does not automatically imply mutual independence.
Example: two coin tosses
For two fair coin tosses, let \(A\) be “the first toss is heads,” \(B\) “the second toss is heads,” and \(C\) “the two tosses match.” Each event has probability \(1/2\), and every pair has intersection probability \(1/4\), so the events are pairwise independent. However,
\[\mathbb{P}(A\cap B\cap C)=\mathbb{P}(\{HH\})=\frac14\ne\frac18,\]
so they are not mutually independent.
Exercises: independence
Exercise 1. Suppose \(\mathbb{P}(A)=0.40\), \(\mathbb{P}(B)=0.30\), and \(\mathbb{P}(A\cap B)=0.12\). Are \(A\) and \(B\) independent?
Show solution to Exercise 1
Yes. Since \(0.40(0.30)=0.12=\mathbb{P}(A\cap B)\), the defining equality holds.Exercise 2. Suppose \(\mathbb{P}(A)=0.50\), \(\mathbb{P}(B)=0.20\), and \(\mathbb{P}(A\cup B)=0.60\). Determine whether \(A\) and \(B\) are independent.
Show solution to Exercise 2
The addition rule gives \[\mathbb{P}(A\cap B)=0.50+0.20-0.60=0.10.\] Because \(\mathbb{P}(A)\mathbb{P}(B)=0.10\), the events are independent.Exercise 3. Events \(A\) and \(B\) are disjoint with \(\mathbb{P}(A)=0.25\) and \(\mathbb{P}(B)=0.40\). Are they independent?
Show solution to Exercise 3
No. Disjointness gives \(\mathbb{P}(A\cap B)=0\), while \(\mathbb{P}(A)\mathbb{P}(B)=0.10\). Positive-probability disjoint events are dependent.Exercise 4. A claim occurs with probability 0.10 and a market decline with probability 0.20. Assuming independence, find the probability of (a) both events; (b) neither event; and (c) at least one event.
Show solution to Exercise 4
- \(0.10(0.20)=0.02\). The complements are also independent, so (b) \(0.90(0.80)=0.72\). Therefore (c) \(1-0.72=0.28\).
Exercise 5. Three components fail independently with probabilities 0.02, 0.03, and 0.05. Find the probability that at least one fails.
Show solution to Exercise 5
Count the complement: \[1-(0.98)(0.97)(0.95)=1-0.90307=0.09693.\]Exercise 6. If \(A\) and \(B\) are independent, prove that \(A^c\) and \(B\) are independent.
Show solution to Exercise 6
Because \(A\cap B\) and \(A^c\cap B\) partition \(B\), \[\mathbb{P}(A^c\cap B)=\mathbb{P}(B)-\mathbb{P}(A\cap B) =\mathbb{P}(B)-\mathbb{P}(A)\mathbb{P}(B) =\mathbb{P}(A^c)\mathbb{P}(B).\] Thus \(A^c\) and \(B\) are independent.4.5 Case study: Who survived the Titanic disaster?
4.5.1 Purpose and context
The Titanic dataset records 2,201 passengers and crew classified by travel class, sex, age group, and survival status. It is an aggregated version of historical passenger records. The purpose of this case study is to translate a real contingency table into events and use probability rules to investigate how survival differed across groups.
This is an observational historical dataset. The probabilities describe recorded associations; they do not by themselves establish why one group had a different survival rate.
4.5.2 Guided questions
Let \(S\) denote survival, \(F\) being female, \(C\) being a child, and \(K_1\) traveling in first class.
- How many people are represented, and what is \(\mathbb{P}(S)\)?
- Find \(\mathbb{P}(S^c)\) and verify the complement rule.
- Compute \(\mathbb{P}(S\mid F)\), \(\mathbb{P}(S\mid F^c)\), \(\mathbb{P}(S\mid C)\), and \(\mathbb{P}(S\mid K_1)\).
- Compute \(\mathbb{P}(F\cap S)\) and verify \(\mathbb{P}(F\cap S)=\mathbb{P}(S\mid F)\mathbb{P}(F)\).
- Are sex and survival independent? Support your answer numerically.
- Use Bayes’ theorem to find \(\mathbb{P}(F\mid S)\). Interpret the direction of conditioning.
- Compute \(\mathbb{P}(F\cup S)\) by inclusion-exclusion and verify it directly from the table.
- Which limitations should be mentioned before generalizing these findings?
4.5.3 Guided solution
There are 2,201 recorded people and 711 survivors, giving \(\mathbb{P}(S)=0.3230\) and \(\mathbb{P}(S^c)=0.6770\). Survival was 0.7319 among females and 0.2120 among males. It was 0.5229 among children and 0.6246 among first-class travelers.
Sex and survival are not independent: \(\mathbb{P}(S\mid F)=0.7319\) is very different from \(\mathbb{P}(S)=0.3230\). Bayes’ theorem gives \(\mathbb{P}(F\mid S)=0.5162\): about 51.6% of survivors in this table were female. This is not the same question as the survival probability among females.
titanic <- as.data.frame(Titanic)
N <- sum(titanic$Freq)
P_event <- function(condition) sum(titanic$Freq[condition]) / N
P_survive <- P_event(titanic$Survived == "Yes")
P_female <- P_event(titanic$Sex == "Female")
P_female_and_survive <- P_event(
titanic$Sex == "Female" & titanic$Survived == "Yes"
)
conditional_rate <- function(group) {
sum(titanic$Freq[group & titanic$Survived == "Yes"]) /
sum(titanic$Freq[group])
}
results <- c(
N = N,
P_survive = P_survive,
P_not_survive = 1 - P_survive,
P_survive_given_female = conditional_rate(titanic$Sex == "Female"),
P_survive_given_male = conditional_rate(titanic$Sex == "Male"),
P_survive_given_child = conditional_rate(titanic$Age == "Child"),
P_survive_given_first = conditional_rate(titanic$Class == "1st"),
P_female_and_survive = P_female_and_survive,
product_rule_check = conditional_rate(titanic$Sex == "Female") * P_female,
P_female_given_survive = P_female_and_survive / P_survive,
P_female_or_survive = P_female + P_survive - P_female_and_survive
)
round(results, 4)## N P_survive P_not_survive
## 2201.0000 0.3230 0.6770
## P_survive_given_female P_survive_given_male P_survive_given_child
## 0.7319 0.2120 0.5229
## P_survive_given_first P_female_and_survive product_rule_check
## 0.6246 0.1563 0.1563
## P_female_given_survive P_female_or_survive
## 0.4838 0.3803## [1] 0.3803The case study illustrates why conditional probabilities require a clearly stated denominator. “Female among survivors” and “survived among females” use the same intersection but normalize by different groups.