Advances in Phytochemistry, Textile and Renewable Energy Research for
Industrial Growth – Nzila et al. (Eds)
© 2022 Copyright the Author(s), ISBN: 978-1-032-11871-0
Open Access: www.taylorfrancis.com, CC BY-NC-ND 4.0 license
Bayesian inference for simple and generalized linear models: Comparing
INLA and McMC
J. Darkwah
Moi University, Eldoret, Kenya
ABSTRACT: Markov chain Monte Carlo (McMC) is a traditional technique in Bayesian inference. Lately,
Integrated Nested Laplace Approximations (INLA) has gained popularity as another technique for Bayesian
inference. This paper compares the performance of these techniques in terms of accuracy, execution time, and
computational burden in simple and generalized linear models. At the end of the simulation study, INLA produced
estimates similar to those of the McMC technique. This observation was evident in the estimates of the fixed
parameters of the models. Though random effects of the generalized linear model were not considered in this
paper, those of the simple linear model were considered and the estimates by the two techniques were found to
be closely identical, leading to the conclusion that INLA is as computationally efficient as McMC. Furthermore,
INLA took a shorter time in approximating parameters than McMC. Finally, McMC was found to be more
computationally intensive than INLA.
1 INTRODUCTION
The two main approaches in inferential statistics are
the Frequentist and the Bayesian approaches. Though,
if procedures are carefully followed, both approaches
produce identical results, they vary mainly in their conceptualization of probability (Fox, 2015). While the
Frequentist sees probability as a limiting frequency
of an experiment repeated infinitely, the Bayesian, on
the other hand, sees it as a subjective quantity which
depends on the availability of information (Wakefield,
2013). Applying these approaches in any simulation
study requires mathematical models, and in statistics these models are largely classified as either a
simple linear model or a generalized linear model.
While in simple linear models, the response variable assumes a Gaussian distribution, the response
variable of a generalized linear model assumes a nonGaussian conditional distribution (Fox, 2015). This
paper focuses on Bayesian inference for simple and
generalized linear models, comparing the traditional
Markov chain Monte Carlo (McMC) technique with
the relatively new Integrated Nested Laplace Approximation (INLA) technique in terms of computational
burden, accuracy and time of execution.
2 LITERATURE REVIEW
Bayesian inference is basically determining parameters of a model by sampling from its posterior marginal
and this is accomplished mostly by the McMC technique but with several challenges (Gamerman, D. &
Lopes, H.F, 2006). One of the challenges is the convergence of the Markov chain, which mostly takes
quite some time. Another challenge is the fact that
the McMC process is computationally intensive. These
challenges result from the complex form that the posterior distribution takes, making it not easy to sample
from (Givens, G. H., & Hoeting, J. A., 2012).
For the past two decades, several software for implementing McMC have been developed. Some of them
are OpenBUGS (Lunn, D., Spiegelhalter, D., Thomas,
A. & Best, N., 2009), JAGS (Plummer, 2003), and
CARBayes (Lee, 2013). While most of these software
rely on Monte Carlo integration for the estimation
of parameters, INLA, which is another technique in
Bayesian inference, computes very accurate approximations of the posterior marginal in a fraction of the
time used by McMC and its software, and it does this
by numerical integration (Rue, H. & Held, L., 2005)
(Rue, H., Martino, & S., Chopin, N., 2009).
In 2010, there was a study titled “Posterior and
Cross-validatory Predictive Checks: A Comparison of
McMC and INLA in Statistical Modeling and Regression.” This study compared cross-validatory checks
between INLA and McMC (Held, L., Schrodle, B. &
Rue, H., 2010). (Carroll, R., Lawson, A. B., Faes, C.
Kirby, R. S., Aregay, M., & Watjou, K., 2015), and
also compared INLA and OpenBUGS for hierarchical
Poisson modeling in disease mapping. It concluded
that INLA underperformed in estimating the parameters of the random effects when compared with
OpenBUGS in disease mapping. This study compares
the performance of McMC and INLA in terms of
accuracy, time of execution, and computational burden
62
DOI 10.1201/9781003221968-8
Industrial Growth – Nzila et al. (Eds)
© 2022 Copyright the Author(s), ISBN: 978-1-032-11871-0
Open Access: www.taylorfrancis.com, CC BY-NC-ND 4.0 license
Bayesian inference for simple and generalized linear models: Comparing
INLA and McMC
J. Darkwah
Moi University, Eldoret, Kenya
ABSTRACT: Markov chain Monte Carlo (McMC) is a traditional technique in Bayesian inference. Lately,
Integrated Nested Laplace Approximations (INLA) has gained popularity as another technique for Bayesian
inference. This paper compares the performance of these techniques in terms of accuracy, execution time, and
computational burden in simple and generalized linear models. At the end of the simulation study, INLA produced
estimates similar to those of the McMC technique. This observation was evident in the estimates of the fixed
parameters of the models. Though random effects of the generalized linear model were not considered in this
paper, those of the simple linear model were considered and the estimates by the two techniques were found to
be closely identical, leading to the conclusion that INLA is as computationally efficient as McMC. Furthermore,
INLA took a shorter time in approximating parameters than McMC. Finally, McMC was found to be more
computationally intensive than INLA.
1 INTRODUCTION
The two main approaches in inferential statistics are
the Frequentist and the Bayesian approaches. Though,
if procedures are carefully followed, both approaches
produce identical results, they vary mainly in their conceptualization of probability (Fox, 2015). While the
Frequentist sees probability as a limiting frequency
of an experiment repeated infinitely, the Bayesian, on
the other hand, sees it as a subjective quantity which
depends on the availability of information (Wakefield,
2013). Applying these approaches in any simulation
study requires mathematical models, and in statistics these models are largely classified as either a
simple linear model or a generalized linear model.
While in simple linear models, the response variable assumes a Gaussian distribution, the response
variable of a generalized linear model assumes a nonGaussian conditional distribution (Fox, 2015). This
paper focuses on Bayesian inference for simple and
generalized linear models, comparing the traditional
Markov chain Monte Carlo (McMC) technique with
the relatively new Integrated Nested Laplace Approximation (INLA) technique in terms of computational
burden, accuracy and time of execution.
2 LITERATURE REVIEW
Bayesian inference is basically determining parameters of a model by sampling from its posterior marginal
and this is accomplished mostly by the McMC technique but with several challenges (Gamerman, D. &
Lopes, H.F, 2006). One of the challenges is the convergence of the Markov chain, which mostly takes
quite some time. Another challenge is the fact that
the McMC process is computationally intensive. These
challenges result from the complex form that the posterior distribution takes, making it not easy to sample
from (Givens, G. H., & Hoeting, J. A., 2012).
For the past two decades, several software for implementing McMC have been developed. Some of them
are OpenBUGS (Lunn, D., Spiegelhalter, D., Thomas,
A. & Best, N., 2009), JAGS (Plummer, 2003), and
CARBayes (Lee, 2013). While most of these software
rely on Monte Carlo integration for the estimation
of parameters, INLA, which is another technique in
Bayesian inference, computes very accurate approximations of the posterior marginal in a fraction of the
time used by McMC and its software, and it does this
by numerical integration (Rue, H. & Held, L., 2005)
(Rue, H., Martino, & S., Chopin, N., 2009).
In 2010, there was a study titled “Posterior and
Cross-validatory Predictive Checks: A Comparison of
McMC and INLA in Statistical Modeling and Regression.” This study compared cross-validatory checks
between INLA and McMC (Held, L., Schrodle, B. &
Rue, H., 2010). (Carroll, R., Lawson, A. B., Faes, C.
Kirby, R. S., Aregay, M., & Watjou, K., 2015), and
also compared INLA and OpenBUGS for hierarchical
Poisson modeling in disease mapping. It concluded
that INLA underperformed in estimating the parameters of the random effects when compared with
OpenBUGS in disease mapping. This study compares
the performance of McMC and INLA in terms of
accuracy, time of execution, and computational burden
62
DOI 10.1201/9781003221968-8