Historically, the notion of independence has played a prominent role in probability. If events form an independent class, much less information is required to determine probabilities of Boolean combinations and calculations are correspondingly easier. In this unit, we give a precise formulation of the concept of independence in the probability sense. As in the case of all concepts which attempt to incorporate intuitive notions, the consequences must be evaluated for evidence that these ideas have been captured successfully.
There are many situations in which we have an “operational independence.”
Supose a deck of playing cards is shuffled and a card is selected at random then replaced with reshuffling. A second card picked on a repeated try should not be affected by the first choice.
If customers come into a well stocked shop at different times, each unaware of the choice made by the others, the the item purchased by one should not be affected by the choice made by the other.
If two students are taking exams in different courses, the grade one makes should not affect the grade made by the other.
The list of examples could be extended indefinitely. In each case, we should expect to model the events as independent in some way. How should we incorporate the concept in our developing model of probability?
We take our clue from the examples above. Pairs of events are considered. The “operational independence” described indicates that knowledge that one of the events has occured does not affect the likelihood that the other will occur. For a pair of events {A,B}, this is the condition
Occurrence of the event A is not “conditioned by” occurrence of the event B. Our basic interpretation is that P(A) indicates of the likelihood of the occurrence of event A. The development of conditional probability in the module Conditional Probability, leads to the interpretation of P(A|B) as the likelihood that A will occur on a trial, given knowledge that B has occurred. If such knowledge of the occurrence of B does not affect the likelihood of the occurrence of A, we should be inclined to think of the events A and B as being independent in a probability sense.
We take our clue from the condition P(A|B)=P(A). Property (CP4) for conditional probability (in the case of equality) yields sixteen equivalent conditions as follows.
| P ( A | B ) = P ( A ) | P ( B | A ) = P ( B ) | P ( A B ) = P ( A ) P ( B ) |
|
|
|
|
|
|
|
|
|
|
|
|
|
These conditions are equivalent in the sense that if any one holds, then all hold. We may chose any one of these as the defining condition and consider the others as equivalents for the defining condition. Because of its simplicity and symmetry with respect to the two events, we adopt the product rule in the upper right hand corner of the table.
Definition. The pair {A,B} of events is said to be (stochastically) independent iff the following product rule holds:
Remark. Although the product rule is adopted as the basis for definition, in many applications the assumptions leading to independence may be formulated more naturally in terms of one or another of the equivalent expressions. We are free to do this, for the effect of assuming any one condition is to assume them all.
The equivalences in the right-hand column of the upper portion of the table may be expressed as a replacement rule, which we augment and extend below:
If the pair 
independent, so is any pair obtained by taking the complement of
either or both of the events.
We note two relevant facts
Suppose event N has probability zero (is a null event). Then for any event A, we have 0≤P(AN)≤P(N)=0=P(A)P(N), so that the product rule holds. Thus {N,A} is an independent pair for any event A.
If event S has probability one (is an almost sure event), then its complement
Sc is a null event. By the replacement rule and the fact just established, 
is independent, so {S,A} is independent.
The replacement rule may thus be extended to:
If the pair 
independent, so is any pair obtained by replacing either or
both of the events by their complements or by a null event or by an almost sure event.
Unless at least one of the events has probability one or zero, a pair cannot be both independent and mutually exclusive. Intuitively, if the pair is mutually exclusive, then the occurrence of one requires that the other does not occur. Formally: Suppose 0<P(A)<1 and 0<P(B)<1. {A,B} mutually exclusive implies P(AB)=P(∅)=0≠P(A)P(B). {A,B} independent implies P(AB)=P(A)P(B)>0=P(∅)
Independence is not a property of events. Two non mutually exclusive events may be independent under one probability measure, but may not be independent for another. This can be seen by considering various probability distributions on a Venn diagram or minterm map.
Extension of the concept of independence to an arbitrary class of events utilizes the product rule.
Definition. A class of events is said to be (stochastically) independent iff the product rule holds for every finite subclass of two or more events in the class.
A class 
is independent iff all four of the following product rules hold
If any one or more of these product expressions fail, the class is not independent. A similar situation holds for a class of four events: the product rule must hold for every pair, for every triple, and for the whole class. Note that we say “not independent” or “nonindependent” rather than dependent. The reason for this becomes clearer in dealing with independent random variables.
We consider some classical exmples of nonindependent classes
Suppose 
is a partition, with each 
. Let
Then the class {A,B,C} has P(A)=P(B)=P(C)=1/2 and is pairwise independent, but not independent, since
Consider the class {A,B,C,D} with AD=BD=∅, C=AB⋁D, P(A)=P(B)=1/4, P(AB)=1/64, and P(D)=15/64. Use of a minterm maps shows these assignments are consistent. Elementary calculations show the product rule applies to the class {A,B,C} but no two of these three events forms an independent pair.
As noted above, the replacement rule holds for any pair of events. It is easy to show, although somewhat cumbersome to write out, that if the rule holds for any finite number k of events in an independent class, it holds for any k+1 of them. By the principle of mathematical induction, the rule must hold for any finite subclass. We may extend the replacement rule as follows.
If a class is independent, we may replace any of the sets by its complement, by a null event, or by an almost sure event, and the resulting class is also independent. Such replacements may be made for any number of the sets in the class. One immediate and important consequence is the following.
If 
is an independent class and the the class
of individual probabilities is known, then
the probability of every minterm may be calculated.
Suppose the class 
is independent with respective probabilities
P(A)=0.3, P(B)=0.6, and P(C)=0.5. Then
is independent and 
is independent and 
Similarly, the probabilities of the other six minterms, in order, are 0.21, 0.21, 0.06,
0.06, 0.09, and 0.09. With these minterm probabilities, the probability of any Boolean
combination of 
, and C may be calculated
In general, eight appropriate probabilities must be specified to determine the minterm probabilities for a class of three events. In the independent case, three appropriate probabilities are sufficient.
Suppose 
is independent with P(A∪BC)=0.51, 
,
and P(A)=0.30. Then 
and
With each of the basic probabilities determined, we may calculate the minterm probabilities, hence the probability of any Boolean combination of the events.