Validation examples AHP and ANP
- 1. Validating AHP and ANP in Exercises with Known Answers By Rozann Saaty Creative Decisions Foundation 4922 Ellsworth Avenue Pittsburgh, PA 15213 Email: rozann@creativedecisions.net Fax: 412-681-4510 1
- 2. Validation Examples • Fundamental Scale of the AHP • Single Pairwise Comparison Matrices • Hierarchical Validation Examples • Networks with Feedback and Dependence • Saaty Compatibility Index • Complex Model Validation with Benefits, Opportunities, Costs and Risks 2
- 3. Fundamental Scale of Absolute Numbers 1 Equal importance 3 Moderate importance of one over another 5 Strong or essential importance 7 Very strong or demonstrated importance 9 Extreme importance 2,4,6,8 For Intermediate values 1.1,1.2,…,1.9 Decimals for very close comparisons Use Reciprocals for Inverse Comparisons 3
- 4. Three Examples of Estimating Relative Values in a single pairwise matrix Relative areas of shapes Relative weights of objects Relative distances to cities 4
- 5. Estimating Relative Areas using AHP (one of the first validation exercises) B A E D C 5
- 6. One Individual’s Judgment Matrix for Estimating Relative Areas Circle Triangle Square Diamond Rec- Eigenvector Actual tangle (priority Relative vector of Sizes from relative Measuring sizes) Figures Circle 1 9 2.5 3 6 0.488 0.467 Triangle 1/9 1 1/5 1/3.5 1/1.5 0.049 0.046 Square 1/2.5 5 1 1.7 3 0.233 0.241 Diamond 1/3 3.5 1/1.7 1 1.5 0.148 0.149 Rectangle 1/6 1.5 1/3 1/1.5 1 0.082 0.097 *Only the judgments in bold must be made, the others are automatically determined 6
- 7. A Relative Priority Scale Applies to a Particular Group of Objects – if the Objects Change, the Priorities Change Relative Relative Priority Priority Scale Scale with Five with Figures Square removed Circle .467 .615 Triangle .046 .061 Square .241 xxxxxxxxxxxxx xxxxxxxxxxxxx Diamond .149 .196 Rectangle .097 .128 7
- 8. Relative Weights of Objects Weight Radio Typewriter Large Projector Small Eigenvector Actual Attache Attache Results Relative Case Weights Radio 1 1/5 1/3 1/4 4 0.09 0.10 Typewriter 5 1 2 2 8 0.40 0.39 Large 3 1/2 1 1/2 4 0.18 0.20 Attache Case Projector 4 1/2 2 1 7 0.29 0.27 Small 1/4 1/8 1/4 1/7 1 0.04 0.04 Attache Case 8
- 9. Distances of Cities from Philadelphia Comparison Cairo Tokyo Chicago San London Montreal Eigen- Distance to Relative of Distances Francisco vector Philadelph Distance from ia in miles Philadelphia Cairo 1 1/2 8 3 3 7 0.263 5,729 0.278 Tokyo 3 1 9 3 3 9 0.397 7,449 0.361 Chicago 1/8 1/9 1 1/6 1/5 2 0.033 660 0.032 San 1/3 1/3 6 1 1/3 6 0.116 2,732 0.132 Francisco London 1/3 1/3 5 3 1 6 0.164 3,658 0.177 Montreal 1/7 1/9 1/2 1/6 1/6 1 0.027 400 0.019 9
- 10. Validation with Hierarchies • An Investment Model • Barzilai’s Wrong Hierarchical Structures – we show how to do them correctly 10
- 11. An Investment Example using Arithmetic Calculating investment returns using arithmetic AHP needs to match these results if it is valid C1 C2 Total Normalized Interest Capital Gains Return Total Return Return Return (relative returns of the two investments) Investment A $3 $10 $13 13/34 = 0.382 Investment B $6 $15 $21 21/34 = 0.618 Totals $9 $25 $34 11
- 12. The Same Investment Example using AHP in the Wrong Way Obtain priorities by normalizing dollars in each column, add and re-normalize to get final results C1 C2 Sum of Normalized Interest Capital Normalized Sum Return Gains Return (norm.) Return (norm.) Investment 3/9 10/25 11/15 11/30=.367 A ≠.382 Investment 6/9 15/25 19/15 19/30=.633 B ≠.618 Totals 1 1 2 12
- 13. Using AHP the Right Way C1 C2 Total Weight and 9/34 25/34 Dollars Add to get the Correct Relative Priorities A 3/9×9/34=3/34 10/25×25/34=10/34 13 13/34 =0 .382 B 6/9×9/34=6/34 15/25×25/34=15/34 21 21/34 = 0.618 Dol- 9 25 34 lars 13
- 14. OBJECTIONS • I don’t want to go through all this nonsense to figure out correct weights for the criteria. • Ans: We are only doing this to show AHP can produce correct results with actual scales. In practice one would combine numbers from the same scale under a single criterion first using whatever formulas are appropriate. • Oh, very well, but what if you have intangibles? • Ans: We establish the importance of the criteria through pairwise comparisons. 14
- 15. Conclusion • In any validation exercise it is necessary to establish the weights of the criteria correctly so they will give the results you are assuming. You cannot assume criteria weights at the top of the hierarchy and results at the bottom that are in conflict. 15
- 16. Predicting International Chess Competition Outcome by Number of Games 16
- 17. Barzilai’s Hierarchical Validation Example The president and three vice-presidents of a company are analyzing their marketing options. The company produces and sells a single product for a fixed price through five stores in the city. Stores 1 and 2 are in the city’s West Side, store 3 is at City Centre, and stores 4 and 5 are in the East Side. They all agree to define the company’s value function as its total annual revenue where represents annual sales in millions of dollars in store i, i = 1,…,5. The company needs to choose between marketing strategies A and B. These strategies will result in annual revenue of P = (3, 3, 1, 1, 1) if strategy A is chosen and Q = (1, 1, 1, 3, 3) in millions of dollars if strategy B is implemented. Barzilai concludes that v(x) = x1+ x2 + x3 + x4 + x5 and since v(P) = v(Q) = 9 (millions of dollars) the two alternatives are equally preferred 17
- 18. Barzilai’s hypothesis: grouping the stores in different ways should give the same result if AHP is valid. What he finds: grouping the stores in different ways gives different results. Why? He does not understand that criteria weights are not arbitrary; they depend on the structure and the assumed results. 18
- 19. Three Structures (we show two here) Barzilai creates three different structures by grouping the five stores into territories in three ways, making the casual and unjustifiable assumption that the criteria and subcriteria weights have default equal values, regardless of the structure. 19
- 20. First Structure v1 1/3 1/3 1/3 Territories s1 s2 s3 1/2 1/2 1 1/2 1/2 Stores x1 x2 x3 x4 x5 Barzilai concludes that v(x) = (1/6, 1/6,1/3,1/6,1/6) = (1/6, 1/6,2/6,1/6,1/6) = 1+ 1 + 2 + 1 + 1 ≠ 9 (million dollars) Actually, these are priorities anyway, not dollars! 20
- 21. Right AHP way to do first structure V 1 s1=8/18=.444 s2=2/18=.111 s3=8/18=.444 8M 2M 8M Store S1 Store S2 Store S3 Store S4 Store S5 x1=4/8=.5 x2=4/8=.5 x3=2/2=1 x4=4/8=.5 x5=4/8=.5 4M 4M 2M 4M 4M P2=3/4=.75 P3=1/2=.5 P4=1/4=.25 P5=1/4=.25 P1=3/4=.75 3M 1M 1M 1M 3M Q3=1/2=.5 Q4=3/4=.75 Q5=3/4=.75 Q1=1/4=.25 Q2=1/4=.25 1M 3M 3M 1M 1M Priority (A) =0.444×0.5×0.75+0.444×0.5×0.75+ 0.333×1×0.5+0.444×0.5×0.25+0.444×0.5×0.25 = 0.50 Priority (B) = 0.444×0.5×0.25+0.444×0.5×0.25+ 21 0.333×1×0.5+0.444×0.5×0.75+0.444×0.5×0.75 = 0.50
- 22. Second Structure v2 1/2 1/2 Territories t1 t2 1/3 1/3 1/3 1/2 1/2 x1 x2 x3 x4 x5 Stores Here Barzilai concludes that v(x) = (1/6, 1/6,1/6,1/4,1/4) = (2/12, 2/12,2/12,3/12,3/12) = 2+ 2+ 2 + 3 + 3 ≠ 9 (million dollars) Again, confusing priorities with dollars. 22
- 23. How Do We Validate the ANP? • Re-do the Investment Example as an ANP model and show the results again validate expectations. • Estimate market share using a network (Walmart, Kmart and Target example) • Do a complex BOCR model (ANWR voting results) 23
- 24. Investment Example as ANP Model Table 4 Data for the Investment Problem Criteria Alternatives Criteria Cluster A1 Interest Return Capital Gains C2 First A2 Return C1 Capi- Inve Second Inter- tal stme Invest est Gains nt ment 1Cri- C1 - Alternatives teria Interest 0 0 3 6 Cluster C2 - Alternative A Alternative B Capital Gains 0 0 10 15 2Alt- ernat A - First ives inv. 3 10 0 0 B- Second inv. 6 15 0 0 24
- 25. ANP VIEWS Supermatrix 25
- 26. ANP RESULTS Final Priorities(same as Limit Matrix Hierarchical Results) (normalize by cluster to get final results) Investmt 1 .3824 Investmt 2 .6176 Interest .2647 Capital .7352 Gains 26
- 27. Walmart, Kmart and Target 27
- 28. Comparison with Actual Data Mass Merch- Market Share ANP Model andizers Actual (1998) Results Walmart .548 .573 (58Billion) Kmart .259 .221 (27.5 Billion) 20.3 Target .192 .206 (20.3 Billion) Closeness of Results: Saaty Compatibility Index: 1.011 28
- 29. Saaty Compatibility Index • Compute the Saaty Compatibility Index by forming two pairwise comparison matrices: A from the results and B from the actual relative values. • Transpose the A matrix and perform Hadamard matrix multiplication (cell-wise): AT x B. Sum each row and sum those results. Divide by n2. The result is the Saaty Compatibility Index. A value of 1.0 means the vectors exactly coincide. • The closer to 1.0, the better the result. 29
- 30. The Next Step in Validation • Validating multilevel ANP models. • The ANWR model for determining whether or not the US should allow drilling for oil in Alaska. • Results: 30
- 31. • In a recent poll, native Alaskans supported opening ANWR to oil and gas exploration. The question asked was “Do you believe oil and gas exploration should or should not be allowed within the ANWR Coastal Plain?” Poll Results Model Results 77.7% 22.3% 31