1 Coalescent Models
23
the occurrence of a coalescent event in the interval (0, t) may be modeled using the
first two of the key properties listed in Sect. 1.4 but over the corresponding, rescaled
interval (0, (Donnelly and Tavaré 1995).
Equivalently, one can imagine taking a standard gene genealogy, such as the one
in Fig. 1.1, then stretching or shrinking its coalescent intervals accordingly, so that
they become proportionately longer (or, respectively, shorter) when the population
size was larger (respectively, shorter). Alternatively, one may model changes in
population size as proportional changes in the mutation parameter θ over time.
Based on these considerations, for simple types of changes in population size, it
is possible to obtain analytical expressions for some quantities of interest (Slatkin
and Hudson 1991; Polanski and Kimmel 2003; Wakeley and Hey 1997).
Figure 1.4b shows the expected site-frequency spectrum for a sample of size
n = 20 from a population which was much smaller in the past than it is now.
Specifically, the population grew 100-fold instantaneously at time t = 0.2 in the
past, measured on the coalescent timescale based on the current population size. In
terms of the scaled mutation rate, between the present and time t = 0.2, the mutation
parameter was θ = 1, while before time t = 0.2 in the past, the mutation parameter
was θ = 0.01.
In this situation, only a small fraction of mutations will occur during the more
ancient coalescent intervals of the gene genealogy. These are the mutations that
would have produced high-frequency SNPs. For example, on average for n = 20,
there will be seven ancestral genetic lineages at time t = 0.2. These more ancient
intervals, with from seven down to two ancestral genetic lineages, are the only ones
in which mutations can create SNPs contributing to site frequencies ξ 14 through
ξ 19 . Figure 1.6b shows a similar scenario for the gene genealogy of a sample of size
n = 6, illustrating the dramatic compression of ancient coalescent intervals under
population growth.
Because more ancient mutations are disproportionately the source of highfrequency SNPs, population growth causes an excess of low-frequency SNPs (Fig.
1.4b) compared to the standard neutral coalescent (Fig. 1.4a). Population decline
causes an excess of high-frequency SNPs (not shown). However, as long as the
branching structure of the gene genealogy is determined by randomly joining
pairs of ancestral lineages, the site-frequency spectrum will always be a convex
decreasing function of the mutant count (Sargsyan and Wakeley 2008). Thus, the
most extreme excess of high-frequency SNPs that population decline or any series
of changes in population size can produce under the coalescent model is a flat sitefrequency spectrum.
1.5.2 Population Subdivision and Migration
The great simplicity of the standard neutral coalescent follows from the exchangeability of the genetic lineages ancestral to the sample (Kingman 1982c) which holds
only under neutrality for well-mixed populations. Whenever lineages carry labels,
such as allelic types when there is selection or locations when there is geographic
23
the occurrence of a coalescent event in the interval (0, t) may be modeled using the
first two of the key properties listed in Sect. 1.4 but over the corresponding, rescaled
interval (0, (Donnelly and Tavaré 1995).
Equivalently, one can imagine taking a standard gene genealogy, such as the one
in Fig. 1.1, then stretching or shrinking its coalescent intervals accordingly, so that
they become proportionately longer (or, respectively, shorter) when the population
size was larger (respectively, shorter). Alternatively, one may model changes in
population size as proportional changes in the mutation parameter θ over time.
Based on these considerations, for simple types of changes in population size, it
is possible to obtain analytical expressions for some quantities of interest (Slatkin
and Hudson 1991; Polanski and Kimmel 2003; Wakeley and Hey 1997).
Figure 1.4b shows the expected site-frequency spectrum for a sample of size
n = 20 from a population which was much smaller in the past than it is now.
Specifically, the population grew 100-fold instantaneously at time t = 0.2 in the
past, measured on the coalescent timescale based on the current population size. In
terms of the scaled mutation rate, between the present and time t = 0.2, the mutation
parameter was θ = 1, while before time t = 0.2 in the past, the mutation parameter
was θ = 0.01.
In this situation, only a small fraction of mutations will occur during the more
ancient coalescent intervals of the gene genealogy. These are the mutations that
would have produced high-frequency SNPs. For example, on average for n = 20,
there will be seven ancestral genetic lineages at time t = 0.2. These more ancient
intervals, with from seven down to two ancestral genetic lineages, are the only ones
in which mutations can create SNPs contributing to site frequencies ξ 14 through
ξ 19 . Figure 1.6b shows a similar scenario for the gene genealogy of a sample of size
n = 6, illustrating the dramatic compression of ancient coalescent intervals under
population growth.
Because more ancient mutations are disproportionately the source of highfrequency SNPs, population growth causes an excess of low-frequency SNPs (Fig.
1.4b) compared to the standard neutral coalescent (Fig. 1.4a). Population decline
causes an excess of high-frequency SNPs (not shown). However, as long as the
branching structure of the gene genealogy is determined by randomly joining
pairs of ancestral lineages, the site-frequency spectrum will always be a convex
decreasing function of the mutant count (Sargsyan and Wakeley 2008). Thus, the
most extreme excess of high-frequency SNPs that population decline or any series
of changes in population size can produce under the coalescent model is a flat sitefrequency spectrum.
1.5.2 Population Subdivision and Migration
The great simplicity of the standard neutral coalescent follows from the exchangeability of the genetic lineages ancestral to the sample (Kingman 1982c) which holds
only under neutrality for well-mixed populations. Whenever lineages carry labels,
such as allelic types when there is selection or locations when there is geographic