A Guide to Effective Benchmarking of Applications Development
Download as PPTX, PDF2 likes1,036 views
This document summarizes a webinar on effective benchmarking of applications development. Pam Morris from Total Metrics discusses benchmarking definitions and types, as well as why organizations benchmark, risks, and how to ensure valid results. She emphasizes establishing rigorous terms of reference and stakeholder agreement on strategic intent, metrics, scope, and more. The webinar aims to help organizations make informed decisions through benchmarking.
A Guide to Effective Benchmarking of Applications Development
- 1. A Guide to Effective Benchmarking of Applications Development October 30, 2012 4:00 PM - 5:30 PM Slide: 1 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 2. Pam Morris CEO Total Metrics Pam.Morris@totalmetrics.com Hosted by: Michael Milutis Director of Marketing Computer Aid, Inc. (CAI) Michael_Milutis@compaid.com Slide: 2 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 3. About Total Metrics • “Supporting organizations in their quest to optimize the cost effectiveness and efficiency of their software delivery” • Internationally based - Certified IFPUG and COSMIC Function Point and Metrics Consultants and Trainers – Europe, UK, USA, Australia • Provide metrics related tools, procedures, consulting and training - Thought leader in functional size measurement and developers of : • SCOPE Project Sizing Software™ – Function point Counting, Portfolio Metrics Management, Industry Benchmarking Analytics. • FP Outline™ Wizard for Estimating Application and Project Functional Size • For organizations serious about measuring Functional Size • Multi-lingual Korean, Chinese, Japanese, Portuguese, Spanish , German, Dutch, Italian, French, English etc. • NEW Release FREEdownloads see WWW.totalmetrics.com Slide: 3 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 4. CAI Achieves IT Operational Excellence Slide: 4 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 5. PDU Credits Available for this Webinar • The PMI has accredited this webinar with PDUs • You will be eligible to receive 1.0 PDU credits • Your PDU email will be sent to you within 24 hours Slide: 5 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 6. Online Webinar Recordings NOW AVAILABLE • Anytime Access • Hundreds of Topics Visit: www.ITMPI.org/library Slide: 6 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 7. Enjoy the benefits of ITMPI Membership JOIN TODAY! • UNLIMITED Free Webinar Recordings • UNLIMITED Free PDU Credits • Hundreds of Topics Visit: www.ITMPI.org/subscribe Slide: 7 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 8. Overview • What is a Benchmark? • Why Benchmark? • What are the Risks? • Ensuring Success - Terms of Reference Slide: 8 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 9. Benchmarking Definitions “Benchmarking is the process of continuously comparing and measuring an organisation with business leaders anywhere in the world to gain information which will help the organisation take action to improve its performance.” American Productivity and Quality Centre Benchmarking Management Guide Cambridge, Mass., 1993, Productivity Press “Activity of comparing objects of interest to each other or against a benchmark to evaluate characteristic(s)” ISO /IEC FCD 29155-1, Software and Systems Engineering – IT Project Performance Benchmarking Framework – Part 1: Concepts & Definitions “Benchmarking is the process of determining a metric baseline for an organisational or functional unit for the purpose of comparison” Total Metrics – Glossary of Terms Slide: 9 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 10. Types of Benchmarks • Internal Benchmark – Benchmark against a group of commonly usable and authorised benchmarks based upon data sourced from within the target organisation. – Procedure of comparing and measuring one‟s own performance within an organisation • External Benchmark – Benchmark against a group of commonly usable benchmarks based upon data sourced from external organisations and issued by external repository providers. – Procedure of comparing and measuring an organisation with other organisations Slide: 10 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 11. Why Benchmark? Answer: To be able to make informed decisions! – Are we more or less effective than our competitors? (Should we offshore?) – Are we more or less effective than industry? (Should we upgrade our skills? Technology? Or should we outsource?) – Is our IT consuming more money than it should ? (Should we change suppliers?) – Are we improving at the rate required by our contract? (Do we need to prepare for likely penalties?) – Are we achieving ROI on investments in tools and technology? (Do we need to review our IT investment strategy?) – Are all organization units performing at the same level? (Which units do we keep or cut?) Slide: 11 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 12. What are the Risks? Answer: Incorrect strategic decisions based on invalid numbers! » Critical to get the Benchmarking Right! Slide: 12 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 13. How do we to ensure Valid Benchmark Results? Answer: Establish rigorous ‘Terms of Reference’ and get stakeholder agreement on the Benchmark’s: • Strategic Intent • Type • Performance Metrics • Standards for Measures • Scope • Frequency • Peers for Comparison • Report Requirements • Dispute Resolution Process Slide: 13 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 14. Strategic Intent Common Understanding: Why are we Benchmarking? • Strategy is determined by: – What results are required – When - within what time frame – How results will be used Slide: 14 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 15. Common purposes for Benchmarks - External • Client initiated Benchmarks – Establish current IT efficiency and effectiveness compared to Industry norms – should we look at outsourcing? – Evaluate supplier performance against Industry standards – should we change suppliers? – Set competitive range for performance baseline and targets – how can we monitor that we are getting value for money from our suppliers? • Supplier initiated Benchmarks – Establish competitive position for productivity, pricing and service levels – how can we ensure that we are in a competitive bidding position? Slide: 15 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 16. Common purposes for Benchmarks - Internal • In-house initiated Benchmarks – Which operational units are the most efficient? - Should we review where work is allocated ? – Which operational have “Best Practice” processes? - How can we implement practices in all units? – Did we achieve the planned benefits of implementing new tools, techniques or technologies? - Can we justify the expenditure? – How consistent is our development productivity rate? - Can we be confident in using it to estimate planned projects? – Is our M&S support productivity improving? - Are we doing more with less staff than Industry norms? Slide: 16 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 17. Example : M&S Support efficiency improvement Slide: 17 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 18. Comparisons – Internal? External? Both? – Internal • To demonstrate improvement trends over time for the organisations internal processes, or – External • To compare internal results with external independent organisational units, or Industry as a whole. Eg. ISBSG Slide: 18 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 19. What will we measure? • Cost-effectiveness of the process – are we getting „value‟ for money invested? • Efficiency of the process – how „productively‟ is our software being developed or maintained? • Speed of Delivery – how „quickly‟ can we deliver software product or „solve a problem‟? • Quality of the product – how „good‟ is the software product or service we deliver? • Quality of the process – how much time and money was wasted in „rework‟? • Customer Satisfaction – how well does the delivery of our software products and related services meet and or exceed their customer‟s expectations? Slide: 19 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 20. Comparable Measures? • Establish Rules for Comparison: • Units of measurement – Day? Hrs? • Measurement scope – which processes , whose effort, which FPs? • Data accuracy • Validation requirements Slide: 20 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 21. Industry Data Collection Standards • Glossary of Terms • Data Collection Standards • Development and Enhancement (D&E) • Maintenance and Support (M&S) • ISO/IEC 29155. Systems and software engineering - Information technology project performance benchmarking framework Slide: 21 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 22. Industry Benchmarking Standards IT project performance benchmarking framework overview International standard (ISO/IEC 29155-1. Systems and software engineering -- Information technology project performance benchmarking framework -- Part 1: Concepts and definitions. ) Slide: 22 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 23. Scope - What is ‘in’ and what is ‘out’ • Comparing “apples to apples” • Decide on the inclusion and exclusion criteria for projects and applications into the Benchmarking Set • Delivery Options • Types of Requirements • Resourcing Profiles • Technology Profiles • Size Profiles Slide: 23 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 24. Scope - What is ‘in’ and what is ‘out’ • Aim for homogeneous data sets for comparison • User Base • Functional Domains • Project Classifications Slide: 24 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 25. How often should we compare? Collect data and compare at a rate that is useful and meaningful! • Demonstrating Trends • Allocating Projects to Benchmarking Periods • Usefulness of the Result • Statistical Validity of the Result Slide: 25 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 26. Comparable External data • Organisational Type • Different User Priorities • Quality of the External Dataset • Filtering of Submission Data in an External Dataset Slide: 26 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 27. Benchmarking Report • Structure and content • Level of granularity • Presentation technology • Confidentiality and Audience • Review Process and Acceptance criteria • Feedback process Slide: 27 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 28. Dispute Resolution Process • Who is responsible for: – resolution costs – liability payments – consequential loss • Who will arbitrate? Slide: 28 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 29. Summary • Benchmarking results provide important input into tactical and strategic decisions • Benchmarking can be done relatively cost effectively using Industry data • BUT ….Careful upfront planning is the most important step! – Get the planning right and – The rest is easy (…..well easier!!) Slide: 29 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 30. Other Considerations • Submitting to the ISBSG Data Repository – Maintenance and Support – Development and Enhancement • Benchmarking Internally • Benchmarking against ISBSG Slide: 30 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 31. Thank You! EMAIL Admin@Totalmetrics.com Total Metrics TO GET THE FULL PAPER ON Level 1, 191 Riversdale Road WHICH THIS PRESENTATION Hawthorn WAS BASED. Victoria 3122 Australia Phone +61 (3) 9882 7611 Admin@Totalmetrics.com www.totalmetrics.com Slide: 31 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 32. Questions? Slide: 32 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 33. CAI Sponsors Proudly Sponsors The IT Metrics & Productivity Institute • IT and Software Knowledge Center: WWW.ITMPI.ORG • Weekly Webinars Hosted by Industry Leaders: WWW.ITMPI.ORG / WEBINARS • Access Webinar Recordings Anytime at WWW.ITMPI.ORG / LIBRARY • Enjoy the Benefits of ITMPI Membership at WWW.ITMPI.ORG / SUBSCRIBE • Follow Us on TWITTER at WWW.TWITTER.COM / ITMPI • Follow Us on FACEBOOK at WWW.ITMPI.ORG/ FACEBOOK • Join Our Network on LINKED IN at WWW.ITMPI.ORG/ LINKEDIN • Find Out About Our CONFERENCES at WWW.ITMPI.ORG/ EVENTS Slide: 33 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 34. Subscribe to the AITS Newsletter • Daily articles in your mailbox on improving the management of IT. Subscribe at www.AITS.org • Listen to our “Thought Radio” podcasts on iTunes or at www.AITS.org Slide: 34 12/27/2012 Webinar Sponsored by Computer Aid, Inc.
- 35. Pam Morris CEO Total Metrics Pam.Morris@totalmetrics.com Hosted by: Michael Milutis Director of Marketing Computer Aid, Inc. (CAI) Michael_Milutis@compaid.com Slide: 35 12/27/2012 Webinar Sponsored by Computer Aid, Inc.