SlideShare a Scribd company logo

A COMPARISON STUDY OF ESTIMATION METHODS FOR GENERALIZED JELINSKI-MORANDA MODEL BASED ON VARIOUS SIMULATED PATTERNS

ijseajournal, profile picture
ijseajournal

The paper compares three estimation methods for the generalized Jelinski-Moranda (GJ-M) model, focusing on maximum likelihood estimation, nonlinear least squares, and weighted nonlinear least squares. It derives mathematical formulas, conducts simulations to evaluate the model’s accuracy, and analyzes multiple generated sub-models to enhance software reliability modeling. Findings indicate significant results and methodologies that address limitations in selecting appropriate reliability models based on varied input data scenarios.

Education
1 of 22
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 DOI: 10.5121/ijsea.2016.7303 27 A COMPARISON STUDY OF ESTIMATION METHODS FOR GENERALIZED JELINSKI-MORANDA MODEL BASED ON VARIOUS SIMULATED PATTERNS Lutfiah Ismail Al turk 1 and Eftekhar Gabel Alsolami2 lturk@kau.edu.sa ,ekalsuami@uj.edu.sa 1 Statistics Department,King Abdulaziz University,Kingdom of Saudi Arabia 2 Mathematics Department,University of Jeddah,Kingdom of Saudi Arabia ABSTRACT In this paper three methods for estimating the parameters of the generalized Jelinski-Moranda (GJ- M)model are compared. The needed mathematical formulas for resolving the estimates are derived. Because of the lack of various real data with changed input data size different simulation scenarios are given to help achieving our goals. Illustrative algorithms for the simulation studies are given. First, the accuracy of the GJ-M model’s estimators is checked based on two evaluation criteria. Moreover, several generated models from the GJ-M general formula are evaluated based on three different methods of comparison. Useful results for the software reliability modelling area are concluded. KEYWORDS Generalized Jelinski-Moranda (GJ-M) model, Simulations, Maximum likelihood estimation, Nonlinear least squares estimation, Weighted nonlinear least squares estimation. 1. INTRODUCTION The main interest of many costumers and developers is software reliability. Software reliability model is a mathematical formula that describes the software failure process as a function of factors, failure time and one of reliability measures. In fact, statistical modeling for quantifying software reliability is a useful approach but before using this approach careful attention should be paid to select the appropriate model that can best quantify and predict reliability. Selecting inappropriate model can be considered as one of this approach limitations. In the ot her hand, studying several models with different failure rate behavior among the huge numbers of proposed reliability models during the last 45 years could help to avoid this limitation. The classical Jelinski– Moranda (J-M) model [ Jelinski and Moranda (1972)] is one of the most elementary models, andit forms the basis of further software reliability models either as a modifications or as an extensions. This model assumes that the failure rate is proportional to the number of remaining faults and each fault contributes the same to the failure rate.The generalized Jelinski-Moranda (GJ-M) model [Al turk and Alsolami (2016)] which also assumes that fault detection and correction begins when a program contains N faults and all the faults have the same rateφcould offer several reliability sub-models with various failure rate behaviour based on its shape parameter β. Those generated sub-models could be checked for a given system with less effort
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 28 and time, the J-M model is a special case of this general model when β = 1. The purpose of this paper is to study the accuracy of the GJ-M model’s three estimators using the maximum likelihood (ML), nonlinear least squares (NLS), weighted nonlinear least squares (WNLS)estimation methods. Another purpose is to compare several generated models as special cases from the GJ-M model based also on those three chosen estimation methods. Two studies based on several simulated Pattern are conducted to achieve our purposes. Generating various simulated patterns will help us to study various situations that could not be offered by the limited available real data sets. Moreover, simulating different size of failure time data will help to investigate the changing in the input data size. The rest of this paper is arranged as follows: Section 2 describes the MLE, NLSE, and WNLSE approaches for the GJ-M model. Section 3 illustrates the evaluation criteria that will be used in our evaluation. Simulation studies will be discussed in Sections 4. In the end, Section 5 is the conclusion of this paper. 2. PARAMETER ESTIMATION OF THE GENERLIZED JELINSKI-MORANDA (GJ-M) MODEL The GJ-M model which is suggested By Al turk and Alsolami in (2016) assumes that the amount of debugging time between fault occurrences has a Weibull distribution. The faults rate is proportional to the number of faults remaining multiplying by the term ηt୧ ஗ିଵ , and each fault discovered is immediately removed thus reducing the number of faults by one. According to these assumptions, the probability density function (pdf) of the GJ-M is defined as follows: fሺt୧ሻ = φሾN − ሺi − 1ሻሿβt୧ ஒିଵ eି஦ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ (1) Where i : is the fault index. φ:is a proportionality constant. N: is the number of initial faults present in system. β:is the shape parameter. t୧: is the i୲୦ time interval between detection of ሺi − 1ሻୱ୲ and i୲୦ faults. The reliability of this model can be obtained as follows: Rሺt୧ሻ = eି஦ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ (2) Additionally, the mean time to failure can be represented by expression: Eሺt୧ሻ = MTTFሺt୧ሻ = ଵ ஒ ሼφሾN − ሺi − 1ሻሿሽ ି భ ಊΓቀ ଵ ஒ ቁ (3)
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 29 While, the variance is given by: varሺt୧ሻ = Eሺt୧ ଶሻ − ൫Eሺt୧ሻ൯ ଶ = Γ ቀ ଶ ஒ + 1ቁ ሾφሺN − ሺi − 1ሻሻሿ మ ಊ − Γଶ ቀ ଵ ஒ + 1ቁ ሾφሺN − ሺi − 1ሻሻሿ మ ಊ = ୻ቀ మ ಊ ାଵቁି୻మቀ భ ಊ ାଵቁ ሾ஦ሺ୒ିሺ୧ିଵሻሻሿ మ ಊ = ୻ቀ మ ಊ ାଵቁି୻మቀ భ ಊ ାଵቁ ሾ஦ሺ୒ିሺ୧ିଵሻሻሿ మ ಊ (4) More details and measures of reliability for the GJ-M model can be found in [Al turk and Alsolami (2016)] 2.1. Maximum Likelihood Estimation (MLE) Method Parameter estimation is of primary importance in software reliability prediction. The maximum likelihood estimation (MLE) method is the most important traditional and widely used estimation technique. This technique has several properties including consistency, efficiency and asymptotic normality. For the purpose of estimating the unknown three parameters N, φ, and β of the GJ-M model using MLE method and based on the failure time data set ሼtଵ, tଶ, … , t୬; n > 0ሽ, the likelihood function will be defined as follows: LሺN, φ, βሻ = β୬ φ୬ ∏ ሾN − ሺi − 1ሻሿt୧ ஒିଵ eି஦∑ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ୬ ୧ୀଵ (5) To simplify the computation, the natural logarithm of both sides will be taken as follows: ln LሺN, φ, βሻ = nln ሺηሻ + nlnሺφሻ + ෍ ln ሾN − ሺi − 1ሻሿ ୬ ୧ୀଵ + β ෍ ln ୬ ୧ୀଵ t୧ − ∑ ln t୧ ୬ ୧ୀଵ − φ ∑ ሾN − ሺi − 1ሻሿt୧ ஒ୬ ୧ୀଵ (6) By taking the first partial derivative of Equation (6) with respect to φ , N and β, we obtain: ப୐ሺ୒,஦,ஒሻ ப஦ = ୬ ஦ − ෍ ሾN − ሺi − 1ሻሿt୧ ஒ ୬ ୧ୀଵ ಢైሺొ,ಞ,ಊሻ ಢొ = ෍ 1 ሾN − ሺi − 1ሻሿ ୬ ୧ୀଵ − φ ෍ t୧ ஒ ୬ ୧ୀଵ
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 30 ಢైሺొ,ಞ,ಊሻ ಢಊ = n β + ෍ lnt୧ ୬ ୧ୀଵ − φ ෍ሾN − ሺi − 1ሻሿt୧ ஒ ୬ ୧ୀଵ lnt୧ Then by settingಢైሺొ,ಞ,ಊሻ ಢಞ = 0, ಢైሺొ,ಞ,ಊሻ ಢొ = 0, and ಢైሺొ,ಞ,ಊሻ ಢಊ = 0, the ML estimates of the GJ-M model’s three parameters should satisfy the following equations: φෝ = ౤ ∑ ሾొషሺ౟షభሻሿ౪౟ ಊ౤ ౟సభ (7) ୬ ∑ ୲౟ ಊ౤ ౟సభ ∑ ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ౤ ౟సభ = ∑ ଵ ሾ୒ିሺ୧ିଵሻሿ ୬ ୧ୀଵ (8) ୬ ஒ + ∑ lnt୧ ୬ ୧ୀଵ = ୬ ∑ ሺ୒ିሺ୧ିଵሻሻ୲౟ ಊ౤ ౟సభ ୪୬୲౟ ∑ ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ౤ ౟సభ (9) 2.2. Nonlinear Least Squares Estimation (NLSE) Method The nonlinear least squares NLS estimates are obtained by minimizing the following objective function: ψ୒୐ୗሺN, φ, βሻ = ∑ ሾt୧ − ┌ሺ భ ಊ ାଵሻ ሾ஦ሺ୒ି୧ାଵሻሿ భ ಊ ሿଶ୬ ୧ୀଵ (10) Taking the first partial derivative of Equation (10), which assumes that each observation has an equal weight, with respect to φ and N, and β and equate the obtained equations to zero [ಢಠొై౏ሺొ,ಞ,ಊሻ ಢಞ = 0, ಢಠಠొై౏ሺొ,ಞ,ಊሻ ಢొ = 0, ಢಠొై౏ሺొ,ಞ,ಊሻ ಢಊ = 0] we obtain: φෝ = ൥Γ ቀ ଵ ஒ + 1ቁ ∑ ଵ ሾሺ୒ି୧ାଵሻሿ మ ಊ ୬ ୧ୀଵ ∑ ୲౟ ሾሺ୒ି୧ାଵሻሿ భ ಊ ୬ ୧ୀଵ൘ ൩ ஒ (11) ∑ ୲౟ ሺ୒ି୧ାଵሻ భ ಊ శభ ୬ ୧ୀଵ = ┌ሺ భ ಊ ାଵሻ ஦ෝ భ ಊ ∑ ଵ ሺ୒ି୧ାଵሻ మ ಊ శభ ୬ ୧ୀଵ (12) Where ┌ᇱሺzሻ = ‫׬‬ dtሺlntሻt୸ିଵஶ ଴ eି୲
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 31 The NLS estimates of N and β can be obtained by solving the Equations (12 and 13). Then by substituting the obtained estimated values of N and β into Equation (11), we get the NLS estimate ofφ. 2.3.Weighted Nonlinear Least Squares Estimation (WNLSE) Method As seen from Equation (4), the variance of the GJ-M model varies with the inter-failure time data, accordingly using the WNLSE method may give more accurate estimates of the model’s parameters. The WNLS estimates will be obtained by minimizing the following objective function: ψ୛୒୐ୗሺN, φ, βሻ = ∑ w୧ሾt୧ − ┌ሺ భ ಊ ାଵሻ ሾ஦ሺ୒ି୧ାଵሻሿ భ ಊ ሿଶ୬ ୧ୀଵ (14) Taking the first partial derivative of the above function with respect to φ, N and β, and equate the obtained equations to zero, we obtain Equations (15, 16 and 17): ∂ψ୛୒୐ୗሺN, φ, βሻ ∂φ = 0 φෝ = ൥Γ ቀ ଵ ஒ + 1ቁ ∑ ୵౟ ሾሺ୒ି୧ାଵሻሿ మ ಊ ୬ ୧ୀଵ ∑ ୵౟୲౟ ሾሺ୒ି୧ାଵሻሿ భ ಊ ୬ ୧ୀଵ൘ ൩ ஒ (15) ∂ψ୛୒୐ୗሺN, φ, βሻ ∂N = 0 ∑ ୵౟୲౟ ሺ୒ି୧ାଵሻ భ ಊ శభ ୬ ୧ୀଵ = ┌ሺ భ ಊ ାଵሻ ஦ෝ భ ಊ ∑ ୵౟ ሺ୒ି୧ାଵሻ మ ಊ శభ ୬ ୧ୀଵ (16) ∂ψ୛୒୐ୗሺN, φ, βሻ ∂β = 0 ┌ᇱ ൬ 1 β + 1൰ ቐ෍ w୧t୧ ሺN − i + 1ሻ భ ಊ − ┌ ቀ ଵ ஒ + 1ቁ φෝ భ ಊ ෍ w୧ ሺN − i + 1ሻ మ ಊ ୬ ୧ୀଵ ୬ ୧ୀଵ ቑ = ┌ቀ భ ಊ ାଵቁ ஒమ ൝∑ ୵౟୲౟ሾ୪୬஦ෝା୪୬ሺ୒ି୧ାଵሻሿ ሺ୒ି୧ାଵሻ భ ಊ − ┌ቀ భ ಊ ାଵቁ ஦ෝ భ ಊ ∑ ୵౟ሾ୪୬஦ෝା୪୬ሺ୒ି୧ାଵሻሿ ሺ୒ି୧ାଵሻ మ ಊ ୬ ୧ୀଵ ୬ ୧ୀଵ ൡ (17)
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 32 Where ┌ᇱሺzሻ = ‫׬‬ dtሺlntሻt୸ିଵஶ ଴ eି୲ By solving the Equations (16 and 17) using Gauss-Newton method N෡ and β෠ can be obtained, then φෝ can be found by substituting N෡ and β෠ in Equation (15). 3. EVALUATION CRITERIA The mean of square errors (MSE), the root mean of square errors (RMSE), the mean absolute errors (MAE) and the mean absolute percentage error (MAPE) criteria are used for the evaluation purpose in our simulation studies. The lower the criteria value, the better performance we get. The formulas of those criteria are shown in Table (1). Table 1: Some evaluation criteria. Criteria Name Criteria formula MSE [Zhanget al. (2003)]. 1 n − k ෍ሺy୧ − yො୧ሻଶ ୬ ୧ୀଵ MAPE [Gentry et al. (1995)] ∑ ൬ │୷౟ି୷ෝ౟│ ୷౟ ൰୬ ୧ୀଵ n × 100% RMSE [Chai and Draxler (2014)]. ඩ 1 n − k ෍ሾy୧ − yො୧ሿଶ ୬ ୧ୀଵ MAE [Chai and Draxler (2014)]. 1 n − k ෍ │y୧ − yො୧│ ୬ ୧ୀଵ Where i : is thefault index. yො୧ : is the predicted value. y୧:is the true value. n: the sample size of the data k: the number of parameters.
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 33 4. SIMULATION STUDY Simulation study was conducted to assess the performance of the GJ-M model under different simulated patterns of software systems. A wide variety of situations were generated by setting different values of the GJ-M model’s three parameters and various sample sizes.In fact, our simulation is designed for two motivations: first to compare the precision of the estimators; then to assess the performance of the different generated sub- models of the GJ-M model, particularly, to focus on their performance for predicting software reliability. 4.1 Simulation algorithms Two simulation algorithms has been designed and coded in R programming language version 3.2.3 to help achieving our motivations: Algorithm 1: For Part I Step 1: Generate 15, 30, 50, and 100 independent uniform U (0,1) random variables. Step 2: For each sample size and using the following equation: t୧ = ሺln ሺuሻ φሾN − ሺi − 1ሻሿ⁄ ሻ భ ಊ, the following four simulation scenarios will be generated: φ = 0.5, N = 150, and β = 0.5, 1, 2. φ = 0.5, N = 250, and β = 0.5, 1, 2. φ = 1.5, N = 150, and β = 0.5, 1, 2. φ = 1.5, N = 250, and β = 0.5, 1, 2. Step 3: Define initial values of the model’s parameters and use Equation (6) and package nlminb to obtain the maximum likelihood (ML) estimates of parameters. Step 4: Define initial values of the model’s parameters and use Equation (10) and package minpack.lm to obtain nonlinear least square (NLS) estimates of parameters. Step 5: Define initial values of the model’s parameters, set w୧ = 1/i where i = 1, 2, …, n and use Equation (14) and package minpack.lm to obtain the weighted nonlinear least squares (WNLS) estimates of parameters. Step 6: Use the MSE and MAPE formulas in Table (1) to check the accuracy of (ML, NLS, and WNLS) estimates. Step 7: For each sample size in Step 1, (Step 1 to 6) will be repeated a large number of times (5000 times in our simulation study) and the average of the parameters estimates and assessment criteria will be computed. Algorithm 2: For Part II Step 1: Generate 100 independent uniform U (0,1) random variables. Step 2: Generate three simulated data sets with the following assumptions: φ = 0.01 , N = 150 and β = 3.5 φ = 0.01 , N = 150, β = 5 φ = 0.01 , N = 150, β = 8.5, and using the following equation:
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 34 t୧ = ሺln ሺuሻ φሾN − ሺi − 1ሻሿ⁄ ሻ భ ಊ Step 3: Set [φ = 0.01, N = 150, and β = 0.5, 1, 1.5, 2, 2.5, and 3] to generate six sub-models as special cases of the GJ-M model. Step 4: Estimate the generated models’ parameters based on MLE, NLSE, and NWLSE estimation methods, to accomplish this step values of the model’s parameters are initialized, set w୧ = 1/i where i = 1, 2, …, n, Equation (6, 10, and 14) will be used, and nlminb and minpack.lm packages will be utilized. Step 5: Select the best fit model among the six generated models based on three selection methods MSE, RMSE, and AME using their mathematical formulas in Table (1). 4.2 Simulation results and discussion The results of the simulation experiments are summarized in Tables (2, 3, 4, and 5), and will be discussed in this section. Part I: Studying and comparing the accuracy of the ML, NLS, WNLS estimates Table (2) demonstrates the average values of the parameter estimates for the MLE, NLSE, and WNLSE methods along with their average MSE and MAPE over the 5000 simulations, under four simulated situations: [φ = 0.5, N = 150, β = 0.5, 1, 2],ሾ φ = 1.5, N = 150, β = 0.5, 1, 2], [φ = 0.5, N = 250, β = 0.5, 1, 2], and [φ = 1.5, N = 250, β = 0.5, 1, 2] with considering different sample sizes [n = 15, 30, 50, and 100]. Each sub-table includes 12 comparison cases for each of the three parameters. In Table (2.a): 4 cases illustrate that the NLS estimates for the parameter φ perform the best and are the closer to the true values, while 8 cases illustrate that the WNLSE method give the best estimates values, none of the cases are for the sake of the MLE method. For the parameter N, 8 cases are for the sake of the NLS method, 5 cases are for the sake of WNLS, and again none of the cases are for the sake of MLE methods. Regarding to the parameter β, the NLSE and WNLSE perform the same with approximately zero values of the average of the two evaluation criteria while the MLE method has larger average values of the MSE and MAPE criteria Moreover, it obvious that the average values of MSE and MAPE values in this table decreases as the sample sizes increases. For the simulated scenario in Table (2.b) we assumed the parameter φ which represents the contribution by each failure in failure intensity to be greater than 1, for this scenario the NLSE and WNLSE methods behave the same with a slight difference for the sake of WNLSE method and again the MLE method takes the third rank. For the parameter N the best performance is from the NLSE method while the WNLS and the MLE methods take the second and third rank respectively and also there is no noticeably difference between the obtained results from the NLSE and WNLSE methods. Lastly, the average values of the MSE and MAPE for β seems to be comparatively the same for NLSE and WNLSE methods and smaller than the MLE method. With respect to assuming larger φ value in this pattern, this assumption gives smaller criteria values for
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 35 the estimates of the three parameters in most of the cases. Also, it is noticeable that the criteria values decreases as the sample sizes increases. While for on the simulation situation in Table (2.c),which assumed larger number of initial faults present in system, the overall results show that the average values of the two evaluation criteria tends to be the smallest for the NLSE method when estimating the parameters φ and N, and tends to be the same with WNLSE when estimating the parameter β. The MLE method has the larger values of the average evaluation criteria for all cases. Moreover, assuming larger N value in this pattern gives larger average values of the MSE and MAPE criteria. Also, if the sample size is increased, the values of evaluation criteria are lower. For the simulated pattern in Table (2.d) it is again as the results above, the NLS estimator seems to perform better than the WNLS and ML estimators with respect to the average values of the MSE and MAPE criteria for the parameters φ and N with a minor difference in the results of the NLSE and WNLSE methods, but for the parameter β the NLSE and WNLS behavethe same as they have smaller MSE and MAPE than the MLE method, also for in this pattern by increasing the sample size lower criteria values are obtained. According to the results in this part some concluding points could be summarized as follows: The overall results show that the average values of MSE and MAPE tends to get smaller when the sample sizes increases so the larger the sample size the more information about the failure process and more reliable results we have and this indicates the need for big software reliability real data to make reliable release decisions. The NLSE and WNLSE methods give more consistent estimate values than the MLE method for all the three GJ-M model’s parameters under all our simulated patterns. The results indicate that there is no major difference between the accuracy of the NLSE and WNLSE methods, their average values of MSE and MAPE either close or the same in most of our considered cases. Assuming larger value of the failure rate contributed by each fault gives smaller average values of MSE and MAPE in most of the selected cases. Maybe this indicates that removing faults with larger contribution will give more enhancements to the accuracy of the obtained estimates. Assuming larger value of initial faults present in system gives larger average values of MSE and MAPE in all the studied cases. Maybe this indicates that having greater start number of faults in a given system will negatively affect the accuracy of its obtained estimates. Part II: Investigate the performance of some of the sub-models as special cases of the GJ-M model Tables (2, 3, and 4)present results of simulation numerical experiments to compare the performance of the six special cases of the GJ-M model. The six sub-models were generated by assuming: ሾφ = 0.5, N = 150, β = 0.5, 1, 1.5, 2, 2.5, and 3]. For this comparison three data sets were simulated by assuming φ = 0.5, N = 150 and the following assumptions for the shape parameter for each data set: Data 1: β = 3.5, Data 2: β = 5.5, and Data 3: β = 8. The estimates were found by using the MLE, NLSE, and WNLSE methods. The performance of the models was
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 36 evaluated based on the MSE, RMSE, andAME criteria. According to the results in Tables (3, 4, and 5) the following points are concluded: According to our study, The ML estimates are unstable as they have big values of the evaluation criteria compared to the true values while the NLE and WNLSE methods give more accurate estimates. In most of the cases, the best performance is obtained when using the WNLSE methods. Model 6 is the best fit model for simulated Data (1, 2, and 3) using the three estimation methods and based on to the MSE, RMSE, and AME criteria. Table 2: The average values of ML, NLS, andWNLS estimates along with average MSE and MAPE ofeach special case of the GJ-M Modelfrom 5000 simulations of sample sizes n = (15,30,50, 100). Table (2.a): Scenario 1;ሾφ = 0.5, N = 150, and β = 0.5, 1, 2ሿ. Bold numbers indicates the best estimates accuracy.
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 37 )(Table (2.a): Continued
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 38 )(Table (2.a): Continued.
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 39 (Table (2.b): Scenario 2; φ = 1.5, N = 150, and β = 0.5, 1, 2ሻ
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 40 (Table (2.b): Continued).
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 41 (Table (2.c): Scenario 3: φ = 0.5, N = 250, and β = 0.5, 1, 2ሻ.
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 42 Table (2.c): Continued
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 43 Table (2.d): Scenario 4: ሾφ = 1.5, N = 250, and β = 0.5, 1, 2ሿ
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 44 Table (2.d): Continued
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 45 Table 3: The MSE of six special cases of the G J-M model based on three simulated data sets. Bold numbers indicates the best model performance.
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 46 Table 4: The RMSE of six special cases of the G J-M model based on three simulated data sets. Bold numbers indicates the best model performance.
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 47 Table 5: The AME of six special cases of the G J-M model based on three simulated data sets. Bold numbers indicates the best model performance. 5. CONCULDING REMARKS The GJ-M model is a general formula of the very well-known Jelinski-Moranda model (1972).This general formula can introduce different failure rate behavior sub-models to help suit more different situations. In this paper, we have explored the performance of three method of estimation for the GJ-M model through simulating many failure system scenarios. Furthermore, we have investigated the performance of some of the sub-models as special cases of the GJ-M model.We have made some conclusions and recommendations based on our simulation results: The NLSE and WNLSE methods perform well, both of them give efficient and accurate estimates in all our studied cases and we also see that they do better than the MLE method.
International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 48 Because selecting appropriate weighting function is acrucial concern of the WNLSE method it has value to try more weight functions either optimal or empirical for more enhancement of the performance of the WNLSE method. Generating several sub-models of the GJ-M model increases the possibility of finding the best fit model that suit several situations. It is worth to consider the WNLSE method with the reliability models considering that they have non-constant variance with testing time. REFERENCES [1] Chai, T., and Draxler, R. ,Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature, Geoscientific Model Development. 7, 1247- 1250. doi:10.5194/gmd-7-1247-2014. [2] Gentry,Travis W., Bogdan M. Wilamowski, and Larry R. Weatherford, " A Comparison of Traditional Forecasting Techniques and Neural Networks", presented at ANNIE'95 - Artificial Neural Networks in Engineering, St. Louis, Missouri, USA, November 12-15, 1995; also in Intelligent Engineering Systems Through Artificial Neural Networks vol 5, pp. 765-760, ed. C. H. Dagli, M. Akay, C. L. P Chen, B. R. Fernandez, J. Gosh, ASME PRESS, New York 1995. [3] Jelinski Z. and Moranda P. B. , Software reliability research, Statistical computer performance evaluation (Edited by W. Freiberger), Academic Press, New York,(1972), pp. 465-497. [4] Liu J., Liu Y., Xu M., Parameter Estimation of Jelinski-Moranda Model Based on Weighted Nonlinear Least Squares and Heteroscedasticity, arXiv preprint arXiv:1503.00094, 2015, pp.1-17 [5] Lutfiah Ismail Al turk andEftekhar Gabel Alsolami, Jelinski-Moranda, Software Reliability Growth Model: A Brief Literature and Modification, International Journal of Software Engineering and Applications (IJSEA), Vol.7, No.2, March 2016, pp.33-44. [6] Zhang, X., Teng, X. and Pham, H., Considering Fault Removal Efficiency in Software Reliability Assessment, IEEE Transactions on Systems, Man and Cybernetics-part A, Vol.33, No.1, 2003; 114- 120. AUTHOR Lutfiah Ismail Al turkis currently working as Associate Professor of Mathematical Statistics in Statistics Department at Faculty of Sciences, King AbdulAziz University, Saudi Arabia. Lutfiah Ismail Al turk obtained her B.Sc degree in Statistics and Computer Science from Faculty of Sciences, King AbdulAziz University in 1993 and M.Sc (Mathematical statistics) degree from Statistics Department, Faculty of Sciences, King AbdulAziz University in 1999. She received her Ph.D in Mathematical Statistics from university of Surrey, UK in 2007. Her current research interests include Software reliability modeling and Statistical Machine Learning. Eftekhar Gabel Alsolami is presently Teaching Assistant in Mathematics Department at Faculty of Sciences, University of Jeddah, Saudi Arabia. Eftekhar Gabel Alsolami obtained her B.Sc degree in Statistics from Faculty of Sciences, King AbdulAziz University in 2011.

More Related Content

PDF
PREDICTING CLASS-IMBALANCED BUSINESS RISK USING RESAMPLING, REGULARIZATION, A...
IJMIT JOURNAL
 
PPTX
Statistical Modeling in 3D: Explaining, Predicting, Describing
Galit Shmueli
 
PDF
A Prediction Model for Taiwan Tourism Industry Stock Index
ijcsit
 
PDF
Application of Semiparametric Non-Linear Model on Panel Data with Very Small ...
IOSRJM
 
PDF
Regression, theil’s and mlp forecasting models of stock index
IAEME Publication
 
PDF
Regression, theil’s and mlp forecasting models of stock index
iaemedu
 
PPTX
Gbs1
Ghousiya Banu.S
 
PDF
Klibel5 econ 9_
KLIBEL
 
PREDICTING CLASS-IMBALANCED BUSINESS RISK USING RESAMPLING, REGULARIZATION, A...
IJMIT JOURNAL
 
Statistical Modeling in 3D: Explaining, Predicting, Describing
Galit Shmueli
 
A Prediction Model for Taiwan Tourism Industry Stock Index
ijcsit
 
Application of Semiparametric Non-Linear Model on Panel Data with Very Small ...
IOSRJM
 
Regression, theil’s and mlp forecasting models of stock index
IAEME Publication
 
Regression, theil’s and mlp forecasting models of stock index
iaemedu
 
Gbs1
Ghousiya Banu.S
 
Klibel5 econ 9_
KLIBEL
 

What's hot (16)

PDF
Christopher Johnson Bachelor's Thesis
BagpipesJohnson
 
PDF
Statistical Modeling: The Two Cultures
Christoph Molnar
 
PDF
The picture fuzzy distance measure in controlling network power consumption
Journal of Fuzzy Extension and Applications
 
PPT
Simple (and Simplistic) Introduction to Econometrics and Linear Regression
Philip Tiongson
 
PDF
Ali, Redescending M-estimator
Muhammad Ali
 
PDF
BOOTSTRAPPING TO EVALUATE RESPONSE MODELS: A SAS® MACRO
Anthony Kilili
 
PDF
Penalized Regressions with Different Tuning Parameter Choosing Criteria and t...
CSCJournals
 
PDF
Rule-Based Mamdani-Type Fuzzy Modeling of Perceived Stress, And Cortisol Resp...
IJERA Editor
 
PPTX
Missing Data and data imputation techniques
Omar F. Althuwaynee
 
DOCX
Regression project
MansiChowkkar
 
PDF
Exploratory data analysis
gokulprasath06
 
PDF
ON FEATURE SELECTION ALGORITHMS AND FEATURE SELECTION STABILITY MEASURES: A C...
ijcsit
 
PDF
Biostatistics Workshop: Missing Data
HopkinsCFAR
 
PDF
Missing data handling
QuantUniversity
 
PDF
PERFORMANCE ANALYSIS OF HYBRID FORECASTING MODEL IN STOCK MARKET FORECASTING
IJMIT JOURNAL
 
PDF
Feature selection using modified particle swarm optimisation for face recogni...
eSAT Journals
 
Christopher Johnson Bachelor's Thesis
BagpipesJohnson
 
Statistical Modeling: The Two Cultures
Christoph Molnar
 
The picture fuzzy distance measure in controlling network power consumption
Journal of Fuzzy Extension and Applications
 
Simple (and Simplistic) Introduction to Econometrics and Linear Regression
Philip Tiongson
 
Ali, Redescending M-estimator
Muhammad Ali
 
BOOTSTRAPPING TO EVALUATE RESPONSE MODELS: A SAS® MACRO
Anthony Kilili
 
Penalized Regressions with Different Tuning Parameter Choosing Criteria and t...
CSCJournals
 
Rule-Based Mamdani-Type Fuzzy Modeling of Perceived Stress, And Cortisol Resp...
IJERA Editor
 
Missing Data and data imputation techniques
Omar F. Althuwaynee
 
Regression project
MansiChowkkar
 
Exploratory data analysis
gokulprasath06
 
ON FEATURE SELECTION ALGORITHMS AND FEATURE SELECTION STABILITY MEASURES: A C...
ijcsit
 
Biostatistics Workshop: Missing Data
HopkinsCFAR
 
Missing data handling
QuantUniversity
 
PERFORMANCE ANALYSIS OF HYBRID FORECASTING MODEL IN STOCK MARKET FORECASTING
IJMIT JOURNAL
 
Feature selection using modified particle swarm optimisation for face recogni...
eSAT Journals
 
Ad

Similar to A COMPARISON STUDY OF ESTIMATION METHODS FOR GENERALIZED JELINSKI-MORANDA MODEL BASED ON VARIOUS SIMULATED PATTERNS (20)

PDF
JELINSKI-MORANDA SOFTWARE RELIABILITY GROWTH MODEL: A BRIEF LITERATURE AND MO...
ijseajournal
 
PDF
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
Editor IJCATR
 
PDF
Software reliability models error seeding model and failure model-iv
Gurbakash Phonsa
 
PDF
Pareto Type II Based Software Reliability Growth Model
Waqas Tariq
 
PDF
Software reliability prediction
Mirza Mohymen
 
PDF
IRJET- A Study on Software Reliability Models
IRJET Journal
 
PDF
Optimal Selection of Software Reliability Growth Model-A Study
IJEEE
 
PDF
A survey of predicting software reliability using machine learning methods
IAESIJAI
 
PDF
A Simplified Model for Evaluating Software Reliability at the Developmental S...
Waqas Tariq
 
PDF
Reliability growth models
Roy Antony Arnold G
 
PDF
A Novel Approach to Derive the Average-Case Behavior of Distributed Embedded ...
ijccmsjournal
 
PDF
50120130405032
IAEME Publication
 
PDF
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
ijceronline
 
PDF
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
ijceronline
 
PDF
A03720106
inventionjournals
 
PPT
Nii shonan-meeting-gsrm-20141021 - コピー
Hironori Washizaki
 
PDF
DEFECT PREDICTION USING ORDER STATISTICS
IAEME Publication
 
PDF
Developing software analyzers tool using software reliability growth model
IAEME Publication
 
PDF
Developing software analyzers tool using software reliability growth model
IAEME Publication
 
PDF
DETECTION OF RELIABLE SOFTWARE USING SPRT ON TIME DOMAIN DATA
IJCSEA Journal
 
JELINSKI-MORANDA SOFTWARE RELIABILITY GROWTH MODEL: A BRIEF LITERATURE AND MO...
ijseajournal
 
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
Editor IJCATR
 
Software reliability models error seeding model and failure model-iv
Gurbakash Phonsa
 
Pareto Type II Based Software Reliability Growth Model
Waqas Tariq
 
Software reliability prediction
Mirza Mohymen
 
IRJET- A Study on Software Reliability Models
IRJET Journal
 
Optimal Selection of Software Reliability Growth Model-A Study
IJEEE
 
A survey of predicting software reliability using machine learning methods
IAESIJAI
 
A Simplified Model for Evaluating Software Reliability at the Developmental S...
Waqas Tariq
 
Reliability growth models
Roy Antony Arnold G
 
A Novel Approach to Derive the Average-Case Behavior of Distributed Embedded ...
ijccmsjournal
 
50120130405032
IAEME Publication
 
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
ijceronline
 
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
ijceronline
 
A03720106
inventionjournals
 
Nii shonan-meeting-gsrm-20141021 - コピー
Hironori Washizaki
 
DEFECT PREDICTION USING ORDER STATISTICS
IAEME Publication
 
Developing software analyzers tool using software reliability growth model
IAEME Publication
 
Developing software analyzers tool using software reliability growth model
IAEME Publication
 
DETECTION OF RELIABLE SOFTWARE USING SPRT ON TIME DOMAIN DATA
IJCSEA Journal
 
Ad

Recently uploaded (20)

PDF
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
Chris Sistrunk
 
PDF
Pre independence Education in Inndia.pdf
goofy5603
 
PDF
O7-L3 Supply Chain Operations - ICLT Program
labyankof
 
PPTX
PPH.pptx obstetrics and gynecology in nursing
SrideviDevaraj5
 
PDF
Classroom Observation Tools for Teachers
Nicaramirez
 
PDF
Abdominal Access Techniques with Prof. Dr. R K Mishra
mohitsuren827
 
PPTX
Pharma ospi slides which help in ospi learning
rameetkumar304
 
PDF
01-Introduction-to-Information-Management.pdf
PrinceIbarra
 
PDF
Computing-Curriculum for Schools in Ghana
abigailanane203
 
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
PDF
RMMM.pdf make it easy to upload and study
sajedul2
 
PDF
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Academy of Grassroots Studies and Research of India (AGRASRI)
 
PPTX
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
chetansingh379583
 
PDF
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
PPTX
Institutional Correction lecture only . . .
limvtheghins
 
PDF
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
PDF
Sports Quiz easy sports quiz sports quiz
nikunj2757
 
PPTX
IMMUNITY IMMUNITY refers to protection against infection, and the immune syst...
shilpa939953
 
PDF
Insiders guide to clinical Medicine.pdf
AbhishekPatil315128
 
PPTX
master seminar digital applications in india
ananyaray35
 
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
Chris Sistrunk
 
Pre independence Education in Inndia.pdf
goofy5603
 
O7-L3 Supply Chain Operations - ICLT Program
labyankof
 
PPH.pptx obstetrics and gynecology in nursing
SrideviDevaraj5
 
Classroom Observation Tools for Teachers
Nicaramirez
 
Abdominal Access Techniques with Prof. Dr. R K Mishra
mohitsuren827
 
Pharma ospi slides which help in ospi learning
rameetkumar304
 
01-Introduction-to-Information-Management.pdf
PrinceIbarra
 
Computing-Curriculum for Schools in Ghana
abigailanane203
 
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
RMMM.pdf make it easy to upload and study
sajedul2
 
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Academy of Grassroots Studies and Research of India (AGRASRI)
 
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
chetansingh379583
 
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
Institutional Correction lecture only . . .
limvtheghins
 
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
Sports Quiz easy sports quiz sports quiz
nikunj2757
 
IMMUNITY IMMUNITY refers to protection against infection, and the immune syst...
shilpa939953
 
Insiders guide to clinical Medicine.pdf
AbhishekPatil315128
 
master seminar digital applications in india
ananyaray35
 

A COMPARISON STUDY OF ESTIMATION METHODS FOR GENERALIZED JELINSKI-MORANDA MODEL BASED ON VARIOUS SIMULATED PATTERNS

  • 1. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 DOI: 10.5121/ijsea.2016.7303 27 A COMPARISON STUDY OF ESTIMATION METHODS FOR GENERALIZED JELINSKI-MORANDA MODEL BASED ON VARIOUS SIMULATED PATTERNS Lutfiah Ismail Al turk 1 and Eftekhar Gabel Alsolami2 lturk@kau.edu.sa ,ekalsuami@uj.edu.sa 1 Statistics Department,King Abdulaziz University,Kingdom of Saudi Arabia 2 Mathematics Department,University of Jeddah,Kingdom of Saudi Arabia ABSTRACT In this paper three methods for estimating the parameters of the generalized Jelinski-Moranda (GJ- M)model are compared. The needed mathematical formulas for resolving the estimates are derived. Because of the lack of various real data with changed input data size different simulation scenarios are given to help achieving our goals. Illustrative algorithms for the simulation studies are given. First, the accuracy of the GJ-M model’s estimators is checked based on two evaluation criteria. Moreover, several generated models from the GJ-M general formula are evaluated based on three different methods of comparison. Useful results for the software reliability modelling area are concluded. KEYWORDS Generalized Jelinski-Moranda (GJ-M) model, Simulations, Maximum likelihood estimation, Nonlinear least squares estimation, Weighted nonlinear least squares estimation. 1. INTRODUCTION The main interest of many costumers and developers is software reliability. Software reliability model is a mathematical formula that describes the software failure process as a function of factors, failure time and one of reliability measures. In fact, statistical modeling for quantifying software reliability is a useful approach but before using this approach careful attention should be paid to select the appropriate model that can best quantify and predict reliability. Selecting inappropriate model can be considered as one of this approach limitations. In the ot her hand, studying several models with different failure rate behavior among the huge numbers of proposed reliability models during the last 45 years could help to avoid this limitation. The classical Jelinski– Moranda (J-M) model [ Jelinski and Moranda (1972)] is one of the most elementary models, andit forms the basis of further software reliability models either as a modifications or as an extensions. This model assumes that the failure rate is proportional to the number of remaining faults and each fault contributes the same to the failure rate.The generalized Jelinski-Moranda (GJ-M) model [Al turk and Alsolami (2016)] which also assumes that fault detection and correction begins when a program contains N faults and all the faults have the same rateφcould offer several reliability sub-models with various failure rate behaviour based on its shape parameter β. Those generated sub-models could be checked for a given system with less effort
  • 2. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 28 and time, the J-M model is a special case of this general model when β = 1. The purpose of this paper is to study the accuracy of the GJ-M model’s three estimators using the maximum likelihood (ML), nonlinear least squares (NLS), weighted nonlinear least squares (WNLS)estimation methods. Another purpose is to compare several generated models as special cases from the GJ-M model based also on those three chosen estimation methods. Two studies based on several simulated Pattern are conducted to achieve our purposes. Generating various simulated patterns will help us to study various situations that could not be offered by the limited available real data sets. Moreover, simulating different size of failure time data will help to investigate the changing in the input data size. The rest of this paper is arranged as follows: Section 2 describes the MLE, NLSE, and WNLSE approaches for the GJ-M model. Section 3 illustrates the evaluation criteria that will be used in our evaluation. Simulation studies will be discussed in Sections 4. In the end, Section 5 is the conclusion of this paper. 2. PARAMETER ESTIMATION OF THE GENERLIZED JELINSKI-MORANDA (GJ-M) MODEL The GJ-M model which is suggested By Al turk and Alsolami in (2016) assumes that the amount of debugging time between fault occurrences has a Weibull distribution. The faults rate is proportional to the number of faults remaining multiplying by the term ηt୧ ஗ିଵ , and each fault discovered is immediately removed thus reducing the number of faults by one. According to these assumptions, the probability density function (pdf) of the GJ-M is defined as follows: fሺt୧ሻ = φሾN − ሺi − 1ሻሿβt୧ ஒିଵ eି஦ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ (1) Where i : is the fault index. φ:is a proportionality constant. N: is the number of initial faults present in system. β:is the shape parameter. t୧: is the i୲୦ time interval between detection of ሺi − 1ሻୱ୲ and i୲୦ faults. The reliability of this model can be obtained as follows: Rሺt୧ሻ = eି஦ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ (2) Additionally, the mean time to failure can be represented by expression: Eሺt୧ሻ = MTTFሺt୧ሻ = ଵ ஒ ሼφሾN − ሺi − 1ሻሿሽ ି భ ಊΓቀ ଵ ஒ ቁ (3)
  • 3. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 29 While, the variance is given by: varሺt୧ሻ = Eሺt୧ ଶሻ − ൫Eሺt୧ሻ൯ ଶ = Γ ቀ ଶ ஒ + 1ቁ ሾφሺN − ሺi − 1ሻሻሿ మ ಊ − Γଶ ቀ ଵ ஒ + 1ቁ ሾφሺN − ሺi − 1ሻሻሿ మ ಊ = ୻ቀ మ ಊ ାଵቁି୻మቀ భ ಊ ାଵቁ ሾ஦ሺ୒ିሺ୧ିଵሻሻሿ మ ಊ = ୻ቀ మ ಊ ାଵቁି୻మቀ భ ಊ ାଵቁ ሾ஦ሺ୒ିሺ୧ିଵሻሻሿ మ ಊ (4) More details and measures of reliability for the GJ-M model can be found in [Al turk and Alsolami (2016)] 2.1. Maximum Likelihood Estimation (MLE) Method Parameter estimation is of primary importance in software reliability prediction. The maximum likelihood estimation (MLE) method is the most important traditional and widely used estimation technique. This technique has several properties including consistency, efficiency and asymptotic normality. For the purpose of estimating the unknown three parameters N, φ, and β of the GJ-M model using MLE method and based on the failure time data set ሼtଵ, tଶ, … , t୬; n > 0ሽ, the likelihood function will be defined as follows: LሺN, φ, βሻ = β୬ φ୬ ∏ ሾN − ሺi − 1ሻሿt୧ ஒିଵ eି஦∑ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ୬ ୧ୀଵ (5) To simplify the computation, the natural logarithm of both sides will be taken as follows: ln LሺN, φ, βሻ = nln ሺηሻ + nlnሺφሻ + ෍ ln ሾN − ሺi − 1ሻሿ ୬ ୧ୀଵ + β ෍ ln ୬ ୧ୀଵ t୧ − ∑ ln t୧ ୬ ୧ୀଵ − φ ∑ ሾN − ሺi − 1ሻሿt୧ ஒ୬ ୧ୀଵ (6) By taking the first partial derivative of Equation (6) with respect to φ , N and β, we obtain: ப୐ሺ୒,஦,ஒሻ ப஦ = ୬ ஦ − ෍ ሾN − ሺi − 1ሻሿt୧ ஒ ୬ ୧ୀଵ ಢైሺొ,ಞ,ಊሻ ಢొ = ෍ 1 ሾN − ሺi − 1ሻሿ ୬ ୧ୀଵ − φ ෍ t୧ ஒ ୬ ୧ୀଵ
  • 4. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 30 ಢైሺొ,ಞ,ಊሻ ಢಊ = n β + ෍ lnt୧ ୬ ୧ୀଵ − φ ෍ሾN − ሺi − 1ሻሿt୧ ஒ ୬ ୧ୀଵ lnt୧ Then by settingಢైሺొ,ಞ,ಊሻ ಢಞ = 0, ಢైሺొ,ಞ,ಊሻ ಢొ = 0, and ಢైሺొ,ಞ,ಊሻ ಢಊ = 0, the ML estimates of the GJ-M model’s three parameters should satisfy the following equations: φෝ = ౤ ∑ ሾొషሺ౟షభሻሿ౪౟ ಊ౤ ౟సభ (7) ୬ ∑ ୲౟ ಊ౤ ౟సభ ∑ ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ౤ ౟సభ = ∑ ଵ ሾ୒ିሺ୧ିଵሻሿ ୬ ୧ୀଵ (8) ୬ ஒ + ∑ lnt୧ ୬ ୧ୀଵ = ୬ ∑ ሺ୒ିሺ୧ିଵሻሻ୲౟ ಊ౤ ౟సభ ୪୬୲౟ ∑ ሾ୒ିሺ୧ିଵሻሿ୲౟ ಊ౤ ౟సభ (9) 2.2. Nonlinear Least Squares Estimation (NLSE) Method The nonlinear least squares NLS estimates are obtained by minimizing the following objective function: ψ୒୐ୗሺN, φ, βሻ = ∑ ሾt୧ − ┌ሺ భ ಊ ାଵሻ ሾ஦ሺ୒ି୧ାଵሻሿ భ ಊ ሿଶ୬ ୧ୀଵ (10) Taking the first partial derivative of Equation (10), which assumes that each observation has an equal weight, with respect to φ and N, and β and equate the obtained equations to zero [ಢಠొై౏ሺొ,ಞ,ಊሻ ಢಞ = 0, ಢಠಠొై౏ሺొ,ಞ,ಊሻ ಢొ = 0, ಢಠొై౏ሺొ,ಞ,ಊሻ ಢಊ = 0] we obtain: φෝ = ൥Γ ቀ ଵ ஒ + 1ቁ ∑ ଵ ሾሺ୒ି୧ାଵሻሿ మ ಊ ୬ ୧ୀଵ ∑ ୲౟ ሾሺ୒ି୧ାଵሻሿ భ ಊ ୬ ୧ୀଵ൘ ൩ ஒ (11) ∑ ୲౟ ሺ୒ି୧ାଵሻ భ ಊ శభ ୬ ୧ୀଵ = ┌ሺ భ ಊ ାଵሻ ஦ෝ భ ಊ ∑ ଵ ሺ୒ି୧ାଵሻ మ ಊ శభ ୬ ୧ୀଵ (12) Where ┌ᇱሺzሻ = ‫׬‬ dtሺlntሻt୸ିଵஶ ଴ eି୲
  • 5. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 31 The NLS estimates of N and β can be obtained by solving the Equations (12 and 13). Then by substituting the obtained estimated values of N and β into Equation (11), we get the NLS estimate ofφ. 2.3.Weighted Nonlinear Least Squares Estimation (WNLSE) Method As seen from Equation (4), the variance of the GJ-M model varies with the inter-failure time data, accordingly using the WNLSE method may give more accurate estimates of the model’s parameters. The WNLS estimates will be obtained by minimizing the following objective function: ψ୛୒୐ୗሺN, φ, βሻ = ∑ w୧ሾt୧ − ┌ሺ భ ಊ ାଵሻ ሾ஦ሺ୒ି୧ାଵሻሿ భ ಊ ሿଶ୬ ୧ୀଵ (14) Taking the first partial derivative of the above function with respect to φ, N and β, and equate the obtained equations to zero, we obtain Equations (15, 16 and 17): ∂ψ୛୒୐ୗሺN, φ, βሻ ∂φ = 0 φෝ = ൥Γ ቀ ଵ ஒ + 1ቁ ∑ ୵౟ ሾሺ୒ି୧ାଵሻሿ మ ಊ ୬ ୧ୀଵ ∑ ୵౟୲౟ ሾሺ୒ି୧ାଵሻሿ భ ಊ ୬ ୧ୀଵ൘ ൩ ஒ (15) ∂ψ୛୒୐ୗሺN, φ, βሻ ∂N = 0 ∑ ୵౟୲౟ ሺ୒ି୧ାଵሻ భ ಊ శభ ୬ ୧ୀଵ = ┌ሺ భ ಊ ାଵሻ ஦ෝ భ ಊ ∑ ୵౟ ሺ୒ି୧ାଵሻ మ ಊ శభ ୬ ୧ୀଵ (16) ∂ψ୛୒୐ୗሺN, φ, βሻ ∂β = 0 ┌ᇱ ൬ 1 β + 1൰ ቐ෍ w୧t୧ ሺN − i + 1ሻ భ ಊ − ┌ ቀ ଵ ஒ + 1ቁ φෝ భ ಊ ෍ w୧ ሺN − i + 1ሻ మ ಊ ୬ ୧ୀଵ ୬ ୧ୀଵ ቑ = ┌ቀ భ ಊ ାଵቁ ஒమ ൝∑ ୵౟୲౟ሾ୪୬஦ෝା୪୬ሺ୒ି୧ାଵሻሿ ሺ୒ି୧ାଵሻ భ ಊ − ┌ቀ భ ಊ ାଵቁ ஦ෝ భ ಊ ∑ ୵౟ሾ୪୬஦ෝା୪୬ሺ୒ି୧ାଵሻሿ ሺ୒ି୧ାଵሻ మ ಊ ୬ ୧ୀଵ ୬ ୧ୀଵ ൡ (17)
  • 6. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 32 Where ┌ᇱሺzሻ = ‫׬‬ dtሺlntሻt୸ିଵஶ ଴ eି୲ By solving the Equations (16 and 17) using Gauss-Newton method N෡ and β෠ can be obtained, then φෝ can be found by substituting N෡ and β෠ in Equation (15). 3. EVALUATION CRITERIA The mean of square errors (MSE), the root mean of square errors (RMSE), the mean absolute errors (MAE) and the mean absolute percentage error (MAPE) criteria are used for the evaluation purpose in our simulation studies. The lower the criteria value, the better performance we get. The formulas of those criteria are shown in Table (1). Table 1: Some evaluation criteria. Criteria Name Criteria formula MSE [Zhanget al. (2003)]. 1 n − k ෍ሺy୧ − yො୧ሻଶ ୬ ୧ୀଵ MAPE [Gentry et al. (1995)] ∑ ൬ │୷౟ି୷ෝ౟│ ୷౟ ൰୬ ୧ୀଵ n × 100% RMSE [Chai and Draxler (2014)]. ඩ 1 n − k ෍ሾy୧ − yො୧ሿଶ ୬ ୧ୀଵ MAE [Chai and Draxler (2014)]. 1 n − k ෍ │y୧ − yො୧│ ୬ ୧ୀଵ Where i : is thefault index. yො୧ : is the predicted value. y୧:is the true value. n: the sample size of the data k: the number of parameters.
  • 7. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 33 4. SIMULATION STUDY Simulation study was conducted to assess the performance of the GJ-M model under different simulated patterns of software systems. A wide variety of situations were generated by setting different values of the GJ-M model’s three parameters and various sample sizes.In fact, our simulation is designed for two motivations: first to compare the precision of the estimators; then to assess the performance of the different generated sub- models of the GJ-M model, particularly, to focus on their performance for predicting software reliability. 4.1 Simulation algorithms Two simulation algorithms has been designed and coded in R programming language version 3.2.3 to help achieving our motivations: Algorithm 1: For Part I Step 1: Generate 15, 30, 50, and 100 independent uniform U (0,1) random variables. Step 2: For each sample size and using the following equation: t୧ = ሺln ሺuሻ φሾN − ሺi − 1ሻሿ⁄ ሻ భ ಊ, the following four simulation scenarios will be generated: φ = 0.5, N = 150, and β = 0.5, 1, 2. φ = 0.5, N = 250, and β = 0.5, 1, 2. φ = 1.5, N = 150, and β = 0.5, 1, 2. φ = 1.5, N = 250, and β = 0.5, 1, 2. Step 3: Define initial values of the model’s parameters and use Equation (6) and package nlminb to obtain the maximum likelihood (ML) estimates of parameters. Step 4: Define initial values of the model’s parameters and use Equation (10) and package minpack.lm to obtain nonlinear least square (NLS) estimates of parameters. Step 5: Define initial values of the model’s parameters, set w୧ = 1/i where i = 1, 2, …, n and use Equation (14) and package minpack.lm to obtain the weighted nonlinear least squares (WNLS) estimates of parameters. Step 6: Use the MSE and MAPE formulas in Table (1) to check the accuracy of (ML, NLS, and WNLS) estimates. Step 7: For each sample size in Step 1, (Step 1 to 6) will be repeated a large number of times (5000 times in our simulation study) and the average of the parameters estimates and assessment criteria will be computed. Algorithm 2: For Part II Step 1: Generate 100 independent uniform U (0,1) random variables. Step 2: Generate three simulated data sets with the following assumptions: φ = 0.01 , N = 150 and β = 3.5 φ = 0.01 , N = 150, β = 5 φ = 0.01 , N = 150, β = 8.5, and using the following equation:
  • 8. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 34 t୧ = ሺln ሺuሻ φሾN − ሺi − 1ሻሿ⁄ ሻ భ ಊ Step 3: Set [φ = 0.01, N = 150, and β = 0.5, 1, 1.5, 2, 2.5, and 3] to generate six sub-models as special cases of the GJ-M model. Step 4: Estimate the generated models’ parameters based on MLE, NLSE, and NWLSE estimation methods, to accomplish this step values of the model’s parameters are initialized, set w୧ = 1/i where i = 1, 2, …, n, Equation (6, 10, and 14) will be used, and nlminb and minpack.lm packages will be utilized. Step 5: Select the best fit model among the six generated models based on three selection methods MSE, RMSE, and AME using their mathematical formulas in Table (1). 4.2 Simulation results and discussion The results of the simulation experiments are summarized in Tables (2, 3, 4, and 5), and will be discussed in this section. Part I: Studying and comparing the accuracy of the ML, NLS, WNLS estimates Table (2) demonstrates the average values of the parameter estimates for the MLE, NLSE, and WNLSE methods along with their average MSE and MAPE over the 5000 simulations, under four simulated situations: [φ = 0.5, N = 150, β = 0.5, 1, 2],ሾ φ = 1.5, N = 150, β = 0.5, 1, 2], [φ = 0.5, N = 250, β = 0.5, 1, 2], and [φ = 1.5, N = 250, β = 0.5, 1, 2] with considering different sample sizes [n = 15, 30, 50, and 100]. Each sub-table includes 12 comparison cases for each of the three parameters. In Table (2.a): 4 cases illustrate that the NLS estimates for the parameter φ perform the best and are the closer to the true values, while 8 cases illustrate that the WNLSE method give the best estimates values, none of the cases are for the sake of the MLE method. For the parameter N, 8 cases are for the sake of the NLS method, 5 cases are for the sake of WNLS, and again none of the cases are for the sake of MLE methods. Regarding to the parameter β, the NLSE and WNLSE perform the same with approximately zero values of the average of the two evaluation criteria while the MLE method has larger average values of the MSE and MAPE criteria Moreover, it obvious that the average values of MSE and MAPE values in this table decreases as the sample sizes increases. For the simulated scenario in Table (2.b) we assumed the parameter φ which represents the contribution by each failure in failure intensity to be greater than 1, for this scenario the NLSE and WNLSE methods behave the same with a slight difference for the sake of WNLSE method and again the MLE method takes the third rank. For the parameter N the best performance is from the NLSE method while the WNLS and the MLE methods take the second and third rank respectively and also there is no noticeably difference between the obtained results from the NLSE and WNLSE methods. Lastly, the average values of the MSE and MAPE for β seems to be comparatively the same for NLSE and WNLSE methods and smaller than the MLE method. With respect to assuming larger φ value in this pattern, this assumption gives smaller criteria values for
  • 9. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 35 the estimates of the three parameters in most of the cases. Also, it is noticeable that the criteria values decreases as the sample sizes increases. While for on the simulation situation in Table (2.c),which assumed larger number of initial faults present in system, the overall results show that the average values of the two evaluation criteria tends to be the smallest for the NLSE method when estimating the parameters φ and N, and tends to be the same with WNLSE when estimating the parameter β. The MLE method has the larger values of the average evaluation criteria for all cases. Moreover, assuming larger N value in this pattern gives larger average values of the MSE and MAPE criteria. Also, if the sample size is increased, the values of evaluation criteria are lower. For the simulated pattern in Table (2.d) it is again as the results above, the NLS estimator seems to perform better than the WNLS and ML estimators with respect to the average values of the MSE and MAPE criteria for the parameters φ and N with a minor difference in the results of the NLSE and WNLSE methods, but for the parameter β the NLSE and WNLS behavethe same as they have smaller MSE and MAPE than the MLE method, also for in this pattern by increasing the sample size lower criteria values are obtained. According to the results in this part some concluding points could be summarized as follows: The overall results show that the average values of MSE and MAPE tends to get smaller when the sample sizes increases so the larger the sample size the more information about the failure process and more reliable results we have and this indicates the need for big software reliability real data to make reliable release decisions. The NLSE and WNLSE methods give more consistent estimate values than the MLE method for all the three GJ-M model’s parameters under all our simulated patterns. The results indicate that there is no major difference between the accuracy of the NLSE and WNLSE methods, their average values of MSE and MAPE either close or the same in most of our considered cases. Assuming larger value of the failure rate contributed by each fault gives smaller average values of MSE and MAPE in most of the selected cases. Maybe this indicates that removing faults with larger contribution will give more enhancements to the accuracy of the obtained estimates. Assuming larger value of initial faults present in system gives larger average values of MSE and MAPE in all the studied cases. Maybe this indicates that having greater start number of faults in a given system will negatively affect the accuracy of its obtained estimates. Part II: Investigate the performance of some of the sub-models as special cases of the GJ-M model Tables (2, 3, and 4)present results of simulation numerical experiments to compare the performance of the six special cases of the GJ-M model. The six sub-models were generated by assuming: ሾφ = 0.5, N = 150, β = 0.5, 1, 1.5, 2, 2.5, and 3]. For this comparison three data sets were simulated by assuming φ = 0.5, N = 150 and the following assumptions for the shape parameter for each data set: Data 1: β = 3.5, Data 2: β = 5.5, and Data 3: β = 8. The estimates were found by using the MLE, NLSE, and WNLSE methods. The performance of the models was
  • 10. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 36 evaluated based on the MSE, RMSE, andAME criteria. According to the results in Tables (3, 4, and 5) the following points are concluded: According to our study, The ML estimates are unstable as they have big values of the evaluation criteria compared to the true values while the NLE and WNLSE methods give more accurate estimates. In most of the cases, the best performance is obtained when using the WNLSE methods. Model 6 is the best fit model for simulated Data (1, 2, and 3) using the three estimation methods and based on to the MSE, RMSE, and AME criteria. Table 2: The average values of ML, NLS, andWNLS estimates along with average MSE and MAPE ofeach special case of the GJ-M Modelfrom 5000 simulations of sample sizes n = (15,30,50, 100). Table (2.a): Scenario 1;ሾφ = 0.5, N = 150, and β = 0.5, 1, 2ሿ. Bold numbers indicates the best estimates accuracy.
  • 11. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 37 )(Table (2.a): Continued
  • 12. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 38 )(Table (2.a): Continued.
  • 13. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 39 (Table (2.b): Scenario 2; φ = 1.5, N = 150, and β = 0.5, 1, 2ሻ
  • 14. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 40 (Table (2.b): Continued).
  • 15. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 41 (Table (2.c): Scenario 3: φ = 0.5, N = 250, and β = 0.5, 1, 2ሻ.
  • 16. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 42 Table (2.c): Continued
  • 17. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 43 Table (2.d): Scenario 4: ሾφ = 1.5, N = 250, and β = 0.5, 1, 2ሿ
  • 18. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 44 Table (2.d): Continued
  • 19. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 45 Table 3: The MSE of six special cases of the G J-M model based on three simulated data sets. Bold numbers indicates the best model performance.
  • 20. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 46 Table 4: The RMSE of six special cases of the G J-M model based on three simulated data sets. Bold numbers indicates the best model performance.
  • 21. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 47 Table 5: The AME of six special cases of the G J-M model based on three simulated data sets. Bold numbers indicates the best model performance. 5. CONCULDING REMARKS The GJ-M model is a general formula of the very well-known Jelinski-Moranda model (1972).This general formula can introduce different failure rate behavior sub-models to help suit more different situations. In this paper, we have explored the performance of three method of estimation for the GJ-M model through simulating many failure system scenarios. Furthermore, we have investigated the performance of some of the sub-models as special cases of the GJ-M model.We have made some conclusions and recommendations based on our simulation results: The NLSE and WNLSE methods perform well, both of them give efficient and accurate estimates in all our studied cases and we also see that they do better than the MLE method.
  • 22. International Journal of Software Engineering & Applications (IJSEA), Vol.7, No.3, May 2016 48 Because selecting appropriate weighting function is acrucial concern of the WNLSE method it has value to try more weight functions either optimal or empirical for more enhancement of the performance of the WNLSE method. Generating several sub-models of the GJ-M model increases the possibility of finding the best fit model that suit several situations. It is worth to consider the WNLSE method with the reliability models considering that they have non-constant variance with testing time. REFERENCES [1] Chai, T., and Draxler, R. ,Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature, Geoscientific Model Development. 7, 1247- 1250. doi:10.5194/gmd-7-1247-2014. [2] Gentry,Travis W., Bogdan M. Wilamowski, and Larry R. Weatherford, " A Comparison of Traditional Forecasting Techniques and Neural Networks", presented at ANNIE'95 - Artificial Neural Networks in Engineering, St. Louis, Missouri, USA, November 12-15, 1995; also in Intelligent Engineering Systems Through Artificial Neural Networks vol 5, pp. 765-760, ed. C. H. Dagli, M. Akay, C. L. P Chen, B. R. Fernandez, J. Gosh, ASME PRESS, New York 1995. [3] Jelinski Z. and Moranda P. B. , Software reliability research, Statistical computer performance evaluation (Edited by W. Freiberger), Academic Press, New York,(1972), pp. 465-497. [4] Liu J., Liu Y., Xu M., Parameter Estimation of Jelinski-Moranda Model Based on Weighted Nonlinear Least Squares and Heteroscedasticity, arXiv preprint arXiv:1503.00094, 2015, pp.1-17 [5] Lutfiah Ismail Al turk andEftekhar Gabel Alsolami, Jelinski-Moranda, Software Reliability Growth Model: A Brief Literature and Modification, International Journal of Software Engineering and Applications (IJSEA), Vol.7, No.2, March 2016, pp.33-44. [6] Zhang, X., Teng, X. and Pham, H., Considering Fault Removal Efficiency in Software Reliability Assessment, IEEE Transactions on Systems, Man and Cybernetics-part A, Vol.33, No.1, 2003; 114- 120. AUTHOR Lutfiah Ismail Al turkis currently working as Associate Professor of Mathematical Statistics in Statistics Department at Faculty of Sciences, King AbdulAziz University, Saudi Arabia. Lutfiah Ismail Al turk obtained her B.Sc degree in Statistics and Computer Science from Faculty of Sciences, King AbdulAziz University in 1993 and M.Sc (Mathematical statistics) degree from Statistics Department, Faculty of Sciences, King AbdulAziz University in 1999. She received her Ph.D in Mathematical Statistics from university of Surrey, UK in 2007. Her current research interests include Software reliability modeling and Statistical Machine Learning. Eftekhar Gabel Alsolami is presently Teaching Assistant in Mathematics Department at Faculty of Sciences, University of Jeddah, Saudi Arabia. Eftekhar Gabel Alsolami obtained her B.Sc degree in Statistics from Faculty of Sciences, King AbdulAziz University in 2011.