Bayesian Inferences for Two Parameter Weibull Distribution

IOSR Journal of Mathematics (IOSR-JM)
e-ISSN: 2278-5728, p-ISSN: 2319-765X. Volume 11, Issue 1 Ver. II (Jan - Feb. 2015), PP 24-33
www.iosrjournals.org
DOI: 10.9790/5728-11122433 www.iosrjournals.org 24 | Page
Bayesian Inferences for Two Parameter Weibull Distribution
Kipkoech W. Cheruiyot1
, Abel Ouko2
, Emily Kirimi3
1
Maasai Mara University, Kenya
Department of Mathematics and Physical Sciences
2
The East African University, Kenya
Department of Mathematics
3
Technical University of Kenya
Department of Mathematics and Statistics
Abstract: In this paper, Bayesian estimation using diffuse (vague) priors is carried out for the parameters of a
two parameter Weibull distribution. Expressions for the marginal posterior densities in this case are not
available in closed form. Approximate Bayesian methods based on Lindley (1980) formula and Tierney and
Kadane (1986) Laplace approach are used to obtain expressions for posterior densities. A comparison based on
posterior and asymptotic variances is done using simulated data. The results obtained indicate that, the
posterior variances for scale parameter  obtained by Laplace method are smaller than both the Lindley
approximation and asymptotic variances of their MLE counterparts.
Keywords: Weibull distribution, Lindley approximation, Laplace approximation, Maximum Likelihood
Estimates
I. Introduction
The Weibull distribution is one of the most widely used distributions in reliability and survival analysis
because of various shapes assumed by the probability density functions (p.d.f) and the hazard function. The
Weibull distribution has been used effectively in analyzing lifetime data particularly when the data are censored
which is very common in survival data and life testing experiments. The Weibull distribution was derived from
the problem of material strength and it has been widely used as a lifetime model. Weibull distribution
corresponds to a family of distribution that covers a wide range of distributions that goes from the normal model
to the exponential model making it applicable in different areas for instance in fatigue life, strength of materials,
genetic research, quality control and reliability analysis. The probability density function (p.d.f) for the two
parameter Weibull distribution is given by
1
( , , ) expT
x x
f x
 

 
  

     
     
     
0x 
(1.1)
II. Likelihood Based Estimation of Parameters of Weibull Distribution.
2.1 The Maximum Likelihood Estimation
Let 1 2, ,..., nX X X be independent random samples of size n from Weibull distribution with the p.d.f given by
(1.1). Differentiating with respect to  and  and equating to zero we obtain 

and 

0
dl
d
 
1
ln ln( ) ( ) ln( ) 0i i
n n
x x
i
i i
n
n x 
 
 
     (2.1)
0
dl
d
 
1
( ) 0i
n
x
i
n 

 
  
   (2.2)
From (2.2) we obtained the maximum likelihood estimates of  as
1
1 n
i
in
x



 
 
 
 (2.3)
Substituting (2.3) into (2.1), yields an expression in terms of 

only as given by

1
1
1
ln
1 1
ln
n
i i n
i
in
i
i
i
x x
x
n
x









 



(2.4)
The maximum likelihood estimate for 

is obtained from (2.4) with the aid of standard iterative procedures.
2.2 Variance and Covariance Estimates
The asymptotic variance-covariance matrix of 

and 

are obtained by inverting information matrix with
elements that are negatives of expected values of second order derivatives of logarithms of the likelihood
functions. Cohen (1965) suggested that in the present situation it is appropriate to approximate the expected
values by their maximum likelihood estimates. Accordingly, we have as the approximate variance-covariance
matrix with elements
12 2
2
2 2
2
var( ) cov( )
cov( ) var( )
l l
l l
    
  
  

  
  
  
    
      
   
    
   
When  and  are independent, the covariance of the above matrix is zero.
When  is known the asymptotic variances for 

is obtained by
2
2
ˆ
var( )
ˆn



 

III. Bayesian Estimation of Parameters of Weibull Distribution
3.1 Bayesian Approach.
Let 1 2, ,..., nX X X be a random sample from a population with a two parameter Weibull distribution given by
( / , )f x   =


1
( )
x 


exp {-( ) }
x 
 (3.1)
The likelihood function is the given by
1
11
( , ; ) exp{ ( ) }i
n n
xn n
i
ii
L x x  
    

 
The log likelihood function is given by
1 1
log log log ( 1) log ( )i
n n
x
i
i i
l L n n x 
   
 
      
Suppose that we are ignorant about the parameters ( ,  ) so that the diffuse (vague) prior used is
1
( , )  

 
  
  (3.2)
The joint posterior distribution is then given by
( , / ) ( / ) ( , )g x L x     
11
1
( , / ) exp{ ( ) }i
n n
xn n
ii
g x  
   



 
  
 
 (3.3)
The marginal p.d.f of x is given by
 1
0 0
11
( ) exp{ ( ) }i
n n
xn n
ii
f x d d 
    
 


   (3.4)
Similarly, the marginal posterior p.d.f’s of  and  are required in order to compute the corresponding
posterior expectations of  and  as

( / ) ( )
( / ) ( / )
( / ) ( )
L x
E x g x
L x
     
   
    
 
  
 



(3.5)
and
( / ) ( )
( / ) ( / )
( / ) ( )
L x
E x g x
L x
     
   
    
 
  
 



(3.6)
respectively
Since the Bayes estimate for  and  involve evaluating ratios of two mathematically intractable integrals,
appropriate Bayesian approximations are applied. Assuming  is known, the likelihood function for  is given
by
1
( / ) exp{ ( ) }i
n
xn
i
L x  
 

 
(3.7)
The log likelihood function for  is given by
1
log log ( )i
n
x
i
l L n 
 

    
Since  is known, the prior density for  is given by
1
( ) 

 
  
 
The posterior density for  is given by
( / ) ( / ) ( )g x L x   
1
1
( / ) ( ) exp{ }
n
n
i
i
g x x  
  

 

  
Therefore the posterior expectation for  is obtained by
( / ) ( / )E x g x d    
( / ) ( )
( / )
( / ) ( )
L x
E x
L x
    

   





(3.8)
3.2 Laplace Approximation
Since the Bayes estimate of  involve ratio of two mathematically intractable integrals, Tierney and Kadane,
(1986) proposed to estimate (3.8) as follows
( / )
nL
nL
e d
E x
e d







(3.9)
where
log ( )
l
L
n
  
1
1
1
log( ) log ( )i
n
x
n
i

 
 
    (3.10)
and
log ( ) log
l
L
n
  
  

1
1
1
log log( ) log ( )i
n
x
n
i

  
 
     (3.11)
The posterior mode 

of L is obtained by differentiating L with respect to  once and equating to zero, that is,
0
dL
d

1
1
( ) 0i
n
x
in


 
   
   
Giving 

in terms of  as
1
1(1 )
n
i
i
x
n






 
  
 

(3.12)
The posterior mode (local maximum) for L
is obtained by differentiating L
with respect to 
and equating
to zero to get
1
1
1
log log( ) log ( )i
n
x
n
i

  
 
    
1
( ) 0i
n
x
in


 
  
  
Giving 

in terms of  as
1
1
1 n
i
i
x
n





 
  
 

(3.13)
The
2
 and
2
 
are equal to the minus the inverse of the second derivative of the log posterior density at its
mode given by
2
"
1
( )L



 
2
2 2 2
1
1 ( 1)
( )i
n
x
i
d L
d n


  
   
 
  
"
2 2
1
1 ( 1)
( ) ( )i
n
x
i
L
n


  

 


 
  
Hence
2
( 1)
12
1
( 1)
n
in
i
x




 


 
 
 


(3.14)
Also
2
1
2
( 2)
( 1)
n
i
i
n x
d L
d
n
 

   





 

 


2
1
"
1
( )L




 

2
12
1 2
1 n
i
i
x
n




 
 
 

(3.15)
Thus, the Laplace approximation of (3.8) is given by
( / ) ( )exp{ ( ( ) ( ))}E x n L L
  

 
 
  (3.16)
where
1
1(1 )
n
i
i
x
n






 
  
 

and
1
1
1 n
i
i
x
n





 
  
 

2
2
( / ) ( )
( / )
( / ) ( )
L x
E x
L x
    

   





(3.17)
2
( / )
nL
nL
e d
E x
e d







where
1
1 1
log( ) log ( )i
n
x
i
L
n

 
 
   
and
2
1
1 1
log log( ) log ( )i
n
x
i
L
n

  



    
The posterior mode of L and L
are given by
1
1
1(1 )
n
i
i
x
n






 
  
 

(3.18)
and
1
1
1( 1)
n
i
i
x
n






 
  
 

(3.19)
respectively
The
2
1 and
2
1 
of L and L
respectively are given by
2
1
"
1
( )L



 
"
2 2
1
1 ( 1)
( ) ( )i
n
x
i
L
n


  

 


 
  
Hence

2
( 1)
12
1
( 1)
n
in
i
x




 


 
 
 


(3.20)
And
2
1
"
1
1
( )L





 
"
1 2
1
1 ( 1)
( )i
n
x
i
L
n


  
  



    
Hence
2
12
1
( 1)
( 1)
n
i
i
x
n



 

 
  


(3.21)
Thus, the Laplace approximation of (3.17) is
2
2 1 1( / ) ( )exp{ ( ( ) ( ))}E x n L L
  

 
 
  (3.22)
Hence the posterior variance of  is
2 2
( / ) ( / ) [ ( / )]V x E x E x  
  
  (3.23)
3.3 Lindley (1980) Approximation
Lindley (1980) developed a multidimensional linear Bayes estimate of an arbitrary function as an
approximation of an asymptotic expansion of the ratio of two integrals which cannot be expressed in a closed
form given by
 
( ) ( )exp ( )
( ) /
( )exp ( )
U V L d
E U x
V L d
   

  





Expanding ( )L  and ( ) ( )U V  by Taylor’s series about

the MLE of , Lindley (1980) Bayesian
approximation of two parameter case is given by
  21 1
( ) / ( 2 ) of order n and smaller.
2 2
ij j j ijkl ijkl
i j i j k l
E U x U U U L U terms

  

 
     
 
 
     11 11 12 12 21 21 22 22 1 1 11 2 21 2 2 22 1 12
2 2 2
30 1 11 2 11 12 21 1 11 12 2 11 22 12 12 2 22 21 1 22 11 21
2
03 1 12 22 2 22
1
2
( ) [3 ( 2 )] [3 ( 2 )]1
2 ( )
U U U U U U U U U
L U U L U U L U U
L U U
         
            
  
        
        
  
   
(3.24)
all evaluated at MLE of  (see, Lindley, 1980). For two parameter Weibull distribution we have,
The MLE of ( , )   is ( , )  
  

( ) log ( ) log ( )V       
i j
ij i j
L
L
 



 
2
; and Ui j ij
U U U
U U
   
  
  
   

( , )th
ij i j  element in the inverse of matrix { }ijL evaluated at ( , ), , 1,2i j  
  
 
The quantities 'ijL s are the higher order derivatives of log-likelihood function given by
10
1
( )i
n
x
i
l n
L
 

  

   


(3.25)
2
2
( 1)
202 2
1
( )i
n
x
i
l n
L  


 



  


(3.26)
3
3
( 1)( 2)
303 3
1
2
( )i
n
x
i
l n
L   


 
 


   


(3.27)
2
1
11
1 1
( ) ln( ) ( )i i i
n n
x x x
i i
l n
L 
    
    

    
 
 
(3.28)
2 2 2
3
( 1) ( 1)
212 2
1 1 1
( ) ( ) ln( ) ( )i i i i
n n n
x x x x
i i i
l n
L    
     
  
 
  

    
 
  
(3.29)
   2
2 2 2 2
4 2 2
4 22
222 2
1 1 1 1 1
( ) ( ) ln( ) ( ) ln( ) ( ) ln( ) ( ) ln( )i i i i i i i i i
n n n n n
x x x x x x x x x
i i i i i
l
L       
            
      

      
 
    
(3.30)
01ln ln( ) ( ) ln( )i i
n n
x x
i
i i
l n
n x L
 
 

    

 
(3.31)
 
2 2
022 2
1
( ) ln( )i i
n
x x
i
l n
L
 
  

   

 (3.32)
   
3 3
033 3
1
2
ln( )i i
n
x x
i
l n
L

 
  

  

 (3.33)
 
3 2
2
122
1 1
( ) ln( ) ( ) ln( )i i i i
n n
x x x x
i i
l
L 
     
   

  
 
  (3.34)
Fisher’s information matrix is given by
20 11
11 02
L L
I
L L
 
  
 
Thus
20 111
11 0220 02 11 11
1 L L
I
L LL L L L
  
   
  
11 12
21 22
 
 
 
  
  (3.35)
where
 2
2
( 1) 2 1
20 02 11 11 2 2
1 1 1 1
( ) (ln( )) ( ) ln( ) ( )i i i i i i
n n n n
x x x x x x
i i i i
n n n
L L L L

    
       

  

   
       
               
        
   
Next, we have
log ( ) ln( )     
Hence
1 2
1 1
and
 
 
   
 
     
 
Since  is known, we let
U 

such that
1 21 ; 0 and 0 ; i,j=1,2ijU U U
 
 
 
    
 
The quantities 'ijL s are the higher order derivatives of log-likelihood function. Because  is known, the
following derivatives are used to obtain Bayes estimates of 
3
3
( 1)( 2)
303 3
1
2
( )i
n
x
i
l n
L   


 
 


   


( , )th
ij i j  is obtained by inverting minus second derivative of log likelihood function with respect to 
evaluated at ( ), 1,2i 
 
 
Therefore, Bayes estimate of  using (3.24) is given by
 211
30 11
1
( / )
2
E x L

  

 
     (3.36)
Then
 2 2 211
30 11 11( / ) 2E x L

     

 
      (3.37)
Hence the posterior variance for  is given by
 
22
( / ) / ( / )Var x E x E x      (3.38)
IV. Results
Simulation experiments were carried out using R software to compare the performance of the Bayes
and MLE estimates of the two parameter Weibull distribution. We assumed the shape parameter  is known.
We performed simulation experiment with different sample sizes (10, 30, 50 and 100), drawn from a Weibull
distribution for different values of shape parameter  (1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5 and 6). We specified
the true value of the scale parameter to be fixed at one ( 1)  for all the sample sizes. We computed
estimates based on two different Bayesian methods, that is, Tierney and Kadane, (1986) Laplace approximation
method, Lindley (1980) approximation and the method of Maximum likelihood estimation.
Figure 1: Graph of Estimates of  against Lambda ( ) for a Sample of n=10 for affixed 1 

Figure 2: Graph of Estimates of  against Lambda ( ) for a sample of n=100 for affixed 1 
Figure 3: Graph of variance for sample size 10 against values of Lambda (  ).
Figure 4: Graph of variance for sample size 100 against values of Lambda ( ).

V. Discussions
Figure 1 show the estimates of  under varying size of , Bayes estimates obtained by Lindley (1980)
and Tierney and Kadane, (1986) Laplace approximation are found to be larger than the MLE counter parts.
Lindley (1980) approximation is found to overestimate the scale parameter ; however the three methods
demonstrate the tendency for their estimates and variances to perform better for larger sample size. This is
shown in Figure 2 when the sample size is increased to 100. As the sample size increases the MLE and Bayes
estimates becomes more consistent and accurate.
Figure 3 show the variances of the estimates of  for varying size of shape parameter for a sample of
10. It is observed that posterior variances of estimates of  are smaller than the asymptotic variances of MLE
hence more precise and accurate. However, both variances tend to converge to zero as the value of  get larger.
For the sample of 10 Tierney and Kadane, (1986) Laplace approximation is seen to perform slightly better than
Lindley (1980), since it stabilizes faster.
Figure 4 show that Tierney and Kadane, (1986) Laplace approximation performed better for larger
sample of 100. The variance of MLE are observed to stabilize faster when the sample size and shape parameter
 increase.
It is also observed that the Bayesian method generally performed better for both small and larger value
of  than the MLE counter parts. Lindley (1980) and Laplace methods tend to perform almost similarly for
smaller  . But the Tierney and Kadane, (1986) Laplace approximation is found to produce better results than
both MLE and Lindley (1980) for larger  and larger samples sizes.
VI. Conclusion
We have shown Bayesian techniques for estimating the scale parameter of the two parameter Weibull
distribution which produces estimates with smaller variances than the MLE. Tierney and Kadane, (1986)
Laplace approximation which requires the second derivatives in its computation is found to be more accurate
than the Lindley (1980) which requires third derivatives in its computation. This is in line with Tierney et al
(1989) findings, that Laplace method is more accurate than the third derivative method of Lindley (1980). Even
though the two Bayesian methods are better than the MLE counter parts, they have their own limitations.
Lindley (1980) approximation requires existence of MLE in its computation. This appears as if it is an
adjustment to the MLE to reduce variability. On the other hand, Laplace approximation requires existence of a
unimodal distribution in its computation, hence difficult to use in cases of a multi modal distribution.
VII. Recommendation
In this study, it is noted that the posterior variances of Bayes estimates are smaller than asymptotic
variances. Comparing the two Bayesian methods, Tierney and Kadane, (1986) Laplace approximation method
has smaller variance than the Lindley (1980) approximation technique hence more precise and accurate. Laplace
approximation does not require explicit third order derivatives in its computation which are required in Lindley
(1980) approximation method hence simple to compute. We therefore recommend further work to be done on
two parameter Weibull distribution when both scale and shape parameters are unknown to investigate accuracy
of the two Bayesian methods.
References
[1]. Cohen, A.C. (1965): Maximum Likelihood Estimation in the Weibull Distribution Based on Complete and Censored Samples.
Technometrics, 7, 579-588.
[2]. Lindley, D.V. (1980). Approximate Bayesian Method, Trabajos Estadistica, 31, 223-237.
[3]. Tierney L, Kass, R.E. and Kadane, J.B. (1989): Fully exponential Laplace approximations to expectations and variances of non-
positive functions. Journal of American Statistical Association, 84, 710-716.
[4]. Tierney L. and Kadane, J.B. (1986): Accurate Approximations for Posterior Moments and Marginal Densities. Journal of American
Statistical Association, 81, 82-86.

Bayesian Inferences for Two Parameter Weibull Distribution

More Related Content

What's hot (18)

Viewers also liked (20)

Similar to Bayesian Inferences for Two Parameter Weibull Distribution (20)

More from IOSR Journals (20)

Recently uploaded (20)

Bayesian Inferences for Two Parameter Weibull Distribution