112
R. E. Graff et al.
testing is to limit the number of association tests executed. For example, in a study
of multiple SNPs and a single phenotype, one might consider running only a single
test per SNP. If the number of SNPs is sufficiently large, however, one will have
to further adjust for the number of tests performed. Doing so is helpful to address
issues of multiple comparisons but it is also important to note that the significance
cutoffs described below are somewhat arbitrary and do not reflect the potential
clinical or biological importance of an association (Witte et al. 1996).
5.5.8.1 Bonferroni and Number of Effective Independent Tests
The most straightforward and the most stringent means by which to correct for
multiple testing is the Bonferroni adjustment. It adjusts the conventional type I error
rate of 0.05 to 0.05/k, where k is the total number of tests performed. The adjustment
assumes that the hypothesis tests are independent, thereby making this approach
quite conservative in the context of correlated tests.
To make the strategy more applicable to the scenario of SNPs in LD, one can
estimate the effective number of independent SNPs in lieu of the total number of
SNPs in the Bonferroni adjustment (Nyholt 2004). Because the number of effective
independent SNPs will always be less than or equal to the total number of genotyped
SNPs, this approach is less conservative than the standard Bonferroni correction. For
GWAS, the generally accepted alpha-level for statistical significance based on the
effective number of genome-wide tests is 5 × 10 −8 . This concept of genome-wide
significance should be used only when hypotheses are tested on the genomic scale.
It is not appropriate for candidate gene studies or replication studies, for which the
number of effective independent tests is substantially lower.
5.5.8.2 Permutation Testing
Permutation testing is a more computationally intensive method with which to
adjust for multiple testing. It is less conservative than the Bonferroni adjustment
because it incorporates the correlation between genotypes and/or phenotypes. It
does so by randomly shuffling phenotypes in the dataset, effectively removing any
association between phenotypes and genotypes, while maintaining the correlation
among genotypes resulting from LD within an individual, and then testing for
association again. Random reassignment of the data and association testing is
repeated some prespecified number of times (generally in the thousands), and all
of the test statistics for the associations of interest are computed for each permuted
dataset. A permuted P value can then be obtained by comparing the original test
statistic to the distribution of test statistics from the permuted datasets. Several
statistical packages implement permutation testing, though the most commonly used
is PLINK (Purcell et al. 2007).
5.5.8.3 False Discovery Rate
Another method to account for multiple testing is the false discovery rate
(Benjamini and Hochberg 1995; Brzyski et al. 2017). Rather than control the
family-wise error rate as does the Bonferroni adjustment, the false discovery rate
controls the expected proportion of false discoveries among significant results.
R. E. Graff et al.
testing is to limit the number of association tests executed. For example, in a study
of multiple SNPs and a single phenotype, one might consider running only a single
test per SNP. If the number of SNPs is sufficiently large, however, one will have
to further adjust for the number of tests performed. Doing so is helpful to address
issues of multiple comparisons but it is also important to note that the significance
cutoffs described below are somewhat arbitrary and do not reflect the potential
clinical or biological importance of an association (Witte et al. 1996).
5.5.8.1 Bonferroni and Number of Effective Independent Tests
The most straightforward and the most stringent means by which to correct for
multiple testing is the Bonferroni adjustment. It adjusts the conventional type I error
rate of 0.05 to 0.05/k, where k is the total number of tests performed. The adjustment
assumes that the hypothesis tests are independent, thereby making this approach
quite conservative in the context of correlated tests.
To make the strategy more applicable to the scenario of SNPs in LD, one can
estimate the effective number of independent SNPs in lieu of the total number of
SNPs in the Bonferroni adjustment (Nyholt 2004). Because the number of effective
independent SNPs will always be less than or equal to the total number of genotyped
SNPs, this approach is less conservative than the standard Bonferroni correction. For
GWAS, the generally accepted alpha-level for statistical significance based on the
effective number of genome-wide tests is 5 × 10 −8 . This concept of genome-wide
significance should be used only when hypotheses are tested on the genomic scale.
It is not appropriate for candidate gene studies or replication studies, for which the
number of effective independent tests is substantially lower.
5.5.8.2 Permutation Testing
Permutation testing is a more computationally intensive method with which to
adjust for multiple testing. It is less conservative than the Bonferroni adjustment
because it incorporates the correlation between genotypes and/or phenotypes. It
does so by randomly shuffling phenotypes in the dataset, effectively removing any
association between phenotypes and genotypes, while maintaining the correlation
among genotypes resulting from LD within an individual, and then testing for
association again. Random reassignment of the data and association testing is
repeated some prespecified number of times (generally in the thousands), and all
of the test statistics for the associations of interest are computed for each permuted
dataset. A permuted P value can then be obtained by comparing the original test
statistic to the distribution of test statistics from the permuted datasets. Several
statistical packages implement permutation testing, though the most commonly used
is PLINK (Purcell et al. 2007).
5.5.8.3 False Discovery Rate
Another method to account for multiple testing is the false discovery rate
(Benjamini and Hochberg 1995; Brzyski et al. 2017). Rather than control the
family-wise error rate as does the Bonferroni adjustment, the false discovery rate
controls the expected proportion of false discoveries among significant results.