70
S. Kiran and D. R. Micheal
Here
f
y; θ 1 , . . . , θ j
is the probability density function and
f
y 1 , . . . , y N ; θ 1 , . . . , θ j
is the joint density function of the random samples.
The joint density function is also the likelihood function and is represented as L(θ ).
Likelihood function can be written as
L(θ ) = L
θ 1 , . . . , θ j
=
N
i=1
f
y i ; θ 1 , . . . , θ j
.
For a discrete random variable, the likelihood function is the probability of the
joint occurrence of y 1 , y 2 , . . . , y N . The values of θ that maximizes the likelihood
function also maximize the probability density function f
y; θ 1 , θ 2 , . . . , θ j
. The
condition for maximum value of L(θ ) is given by
∂ L(θ )
∂θ
= 0.
The solution to this system of equations ˆ
θ gives the maximum likelihood estimate
of the parameter θ . In the maximum likelihood estimation method, the log-likelihood
function ln[L(θ )] is maximized instead of L(θ ). Maximizing the logarithm of a function is equivalent to maximizing the function itself as the logarithm is a continuous
and monotonically increasing function. For further details, readers are referred to
[10, 11].
3.2 Goodness of Fit Tests
Inferences on probability distribution of population are made based on parameter
estimates calculated from small representative samples. The goodness-of-fit test
provides a statistical hypothesis on the theoretical probability distribution functions
fit to the observed sample points and whether there are any notable differences
between theoretical and empirical data points. Nonparametric tests for goodness fit
like χ
2 , Kolmogorov–Smirnov (KS) and Anderson–Darling (AD) involve testing
of hypotheses within confidence limits. χ
2 measures the departure of the observed
value from the expected value. KS is based on the maximum difference between the
empirical values and expected values of cumulative distributions. AD gives more
weightage to the distribution tails where the maximum and minimum values of the
sample data points impact the quality of curve fitting. In general, KS and AD check
for the normality of data points. The log-likelihood ratio test compares the fit of
two models using the maximum values of the likelihood function of the models.
Higher the value, better is the fit. The Akaike information criterion (AIC), based
on AIC score, is calculated by measuring the model performance of various probability distribution functions for the same samples. The Bayesian information criterion
(BIC) is based on the likelihood function and resolves the problem of over-fitting by
S. Kiran and D. R. Micheal
Here
f
y; θ 1 , . . . , θ j
is the probability density function and
f
y 1 , . . . , y N ; θ 1 , . . . , θ j
is the joint density function of the random samples.
The joint density function is also the likelihood function and is represented as L(θ ).
Likelihood function can be written as
L(θ ) = L
θ 1 , . . . , θ j
=
N
i=1
f
y i ; θ 1 , . . . , θ j
.
For a discrete random variable, the likelihood function is the probability of the
joint occurrence of y 1 , y 2 , . . . , y N . The values of θ that maximizes the likelihood
function also maximize the probability density function f
y; θ 1 , θ 2 , . . . , θ j
. The
condition for maximum value of L(θ ) is given by
∂ L(θ )
∂θ
= 0.
The solution to this system of equations ˆ
θ gives the maximum likelihood estimate
of the parameter θ . In the maximum likelihood estimation method, the log-likelihood
function ln[L(θ )] is maximized instead of L(θ ). Maximizing the logarithm of a function is equivalent to maximizing the function itself as the logarithm is a continuous
and monotonically increasing function. For further details, readers are referred to
[10, 11].
3.2 Goodness of Fit Tests
Inferences on probability distribution of population are made based on parameter
estimates calculated from small representative samples. The goodness-of-fit test
provides a statistical hypothesis on the theoretical probability distribution functions
fit to the observed sample points and whether there are any notable differences
between theoretical and empirical data points. Nonparametric tests for goodness fit
like χ
2 , Kolmogorov–Smirnov (KS) and Anderson–Darling (AD) involve testing
of hypotheses within confidence limits. χ
2 measures the departure of the observed
value from the expected value. KS is based on the maximum difference between the
empirical values and expected values of cumulative distributions. AD gives more
weightage to the distribution tails where the maximum and minimum values of the
sample data points impact the quality of curve fitting. In general, KS and AD check
for the normality of data points. The log-likelihood ratio test compares the fit of
two models using the maximum values of the likelihood function of the models.
Higher the value, better is the fit. The Akaike information criterion (AIC), based
on AIC score, is calculated by measuring the model performance of various probability distribution functions for the same samples. The Bayesian information criterion
(BIC) is based on the likelihood function and resolves the problem of over-fitting by