Prediction Markets In Project and Program Management
Download as PPTX, PDF1 like1,262 views
The document discusses prediction markets as a tool for program and project management, emphasizing the 'wisdom of crowds' phenomenon to gauge consensus on future events. It outlines key characteristics of prediction markets, how they differ from traditional surveys, and provides examples of questions relevant to various project knowledge areas. The document concludes that while prediction markets can enhance decision-making across all project realms, they may not be suitable for every project or population.
Prediction Markets In Project and Program Management
- 1. Prediction Markets for Program and Project Management PMI WDC Tools Seminar Tom Erickson 17 April 2012
- 2. Agenda • The Wisdom of Crowds • Prediction Markets • An Audience Participation Demo • Prediction Market Characteristics • A Real Prediction Market • Prediction Markets for Knowledge Areas • Lessons Learned 4/17/2012 Tom Erickson, 2012 All Rights Reserved 2
- 3. “The Wisdom of Crowds” 1 • How Many Jelly Beans? • Average of answers from a large population will be more accurate than most of the individual answers. • A diverse collection of independently-deciding individuals is likely to make certain types of decisions and predictions better than individuals or 2 even experts . 1. “The Wisdom of Crowds”, James Surowiecki, Doubleday, 2004 4/17/2012 3 2. http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds
- 4. What If. . .? 4/17/2012 Tom Erickson, 2012 All Rights Reserved 4
- 5. The Problem I Need A Prediction Market! 4/17/2012 Tom Erickson, 2012 All Rights Reserved 5
- 6. 3 What Is a Prediction Market? We can look to a popular television show to explain what prediction markets are: In “Who Wants to be a Millionaire,” when contestants get stuck, they have three lifelines: » 50/50 Eliminate two of the four answers » Phone a friend Call whoever you want to get help » Ask the audience (93% accurate) Use the highest percentage answer from the audience A prediction market is like “asking the audience.” 4/17/2012 Inkling Markets, Inc. Used with Permission 6
- 7. The Nebit IPL • Potentially earth-shaking new consumer product • Initial, simultaneous, multi-site initial product launch (IPL) pending • Initial product runs successful (more or less) • Mahogany row has high expectations • What could go wrong? Vote your confidence in the IPL‟s success. 4/17/2012 Tom Erickson, 2012 All Rights Reserved 7
- 8. The Nebit IPL Prediction Market 4/17/2012 Tom Erickson, 2012 All Rights Reserved 8
- 9. Prediction Market Characteristics • Prediction market systems: – Can assess the consensus of large (selected) populations efficiently, on a real-time basis – Are usually web-based – Pose questions about future events – Can look and feel like stock markets, complete with „play money‟, buys and sells – Can assess multiple future outcomes in parallel; i.e., they can include many questions on many topics at the same time – Often include tools for statistical and trend analysis – Frequently allow users to pose their own questions 4/17/2012 Tom Erickson, 2012 All Rights Reserved 9
- 10. Surveys vs Prediction Markets3 Typically if a company wants to understand people‟s opinions they conduct a survey or poll. A prediction market is different from a survey or poll. • Surveys and polls • A prediction market asks typically ask what a what a person predicts will person wants to happen happen whether they want “Who will you vote for? the outcome to happen or not “Who do you think will be elected? • Surveys and polls are • A prediction market gives snapshots in time. A an ongoing view in to what person cannot change people think will happen their answer once they and allows people to have answered it change their mind based on external events or new information 4/17/2012 Inkling Markets, Inc. Used with Permission 10
- 11. Wise Crowds Criteria Description Each person should have private information even if Diversity of opinion it's just an eccentric interpretation of the known facts. People's opinions aren't determined by the opinions of Independence those around them. People are able to specialize and draw on local Decentralization knowledge. Some mechanism exists for turning private judgments Aggregation into a collective decision. 4/17/2012 “The Wisdom of Crowds”, James Surowiecki, Doubleday, 2004 11
- 12. This Just In. . . You are the shipping manager and you heard through the grapevine that one of your shippers has just won another major contract that will use all their spare capacity, and then some. 4/17/2012 Tom Erickson, 2012 All Rights Reserved 12
- 13. Example: Inkling Public Markets 4/17/2012 Inkling Markets, Inc. Used with permission. 13
- 14. News Flash During a production staff meeting the industrial engineers monitoring the production statistics reported a 20% improvement in throughput over the initial production run, but this news won‟t be reported up the chain until the next monthly PMR. 4/17/2012 Tom Erickson, 2012 All Rights Reserved 14
- 15. Project/Program Knowledge Areas The Standard for Project Management The Standard for Program Management Integration Management Integration Management Scope Management Scope Management Time Management Time Management Cost Management Cost Management Quality Management Quality Management Human Resource Management Human Resource Management Communications Management Communications Management Risk Management Risk Management Procurement Management Procurement Management Financial Management Stakeholder Management Program Governance 4/17/2012 Tom Erickson, 2012 All Rights Reserved 15
- 16. Prediction Market Questions Knowledge Area Prediction Market Questions • How many changes will the Charter go through during the project? •Will the Program Management Plan get signed Integration Management off before CDR? • Will all class 1 changes become resolved before IOC? • How much will requirements increase before PDR? Scope Management • How many errors will the audit find in the WBS? • How many errors will the functional configuration audit find? • What percent of the project activities will turn out to be on the critical path? Time Management • How many activities will be added to the schedule during final design? • Will FQT complete on schedule? 4/17/2012 Tom Erickson, 2012 All Rights Reserved 16
- 17. Prediction Market Questions Knowledge Area Prediction Market Questions • When will the overall program CPI exceed 5% of baseline? Cost Management • How long will it take to complete the next rolling wave budget? • Will the cost variance at IOC exceed 15%? • Will the customer approve the QMP in time for CDR? • Will Program XYZ successfully pass Phase 2 Quality Management PCA? • How many process escapes will occur during low rate initial production? • Will we meet the project start up 100-day staffing levels? • What percentage of the staff will complete time Human Resource Management accounting training before the next timecard audit? • How many key personnel will leave by FAT? 4/17/2012 Tom Erickson, 2012 All Rights Reserved 17
- 18. Prediction Market Questions Knowledge Area Prediction Market Questions • How many stakeholders will attend the PDR? • What will be the score from the next communications survey with the team? With the Communications Management stakeholders? • How many staff hours will it take to prepare for the next quarterly status review? • What will be the score of the next business resiliency exercise? • Where is the next unforeseen problem most Risk Management likely to occur? • How many opportune uncertainties will the staff identify in the next quarter? • Will the contract get signed before the deadline for ordering long-lead materiel passes? •How much will the contract mod backlog grow Procurement Management before CDR? • How many of our sole source suppliers will we lose during low rate initial production? 4/17/2012 Tom Erickson, 2012 All Rights Reserved 18
- 19. Prediction Market Questions Knowledge Area Prediction Market Questions • Will this year’s program ROIC exceed expectations? Financial Management • How much will our cost of capital change this fiscal year? • Which program will exceed expected cash flow? • Will the COO approve the stakeholder management/ communications plan? • How many stakeholders will decline the next Stakeholder Management semi-annual program review? • Which key stakeholders will be most difficult to schedule for one-on-ones? • Which program benefits expected this year will not appear? • Which functional area will generate the most Program Governance audit findings this year? • How much will quality escapes erode program profits in the next five years? 4/17/2012 Tom Erickson, 2012 All Rights Reserved 19
- 20. The Leak A member of the West Coast marketing team overhears one of the retailers at a conference describe The Nebit to a competitor. It is one week until IPL. 4/17/2012 Tom Erickson, 2012 All Rights Reserved 20
- 21. Lessons Learned • Need diverse mix of participants – Different levels – Different organizational areas • Traders must have some knowledge or frame of reference of at least 3-4 questions being asked in the marketplace • Maintain anonymity among peers • Don‟t try to force organizational structure on marketplace • Users will self-select what to trade in • People take the fantasy currency seriously 4/17/2012 Inkling Markets, Inc. Used with permission. 21
- 22. Lessons Learned (cont.) • Prizes don‟t work that well. Here are some alternatives: – Reward participants with exclusive access to people and information – Create a competitive environment between groups – Create a community of practice – Let people ask their own questions. Make the marketplace feel organic, not forced from above In addition to incentives, proper management of the marketplace is the #1 indicator of success vs. failure 4/17/2012 Inkling Markets, Inc. Used with permission. 22
- 23. The Nebit IPL Prediction Market What has been happening? 4/17/2012 Tom Erickson, 2012 All Rights Reserved 23
- 24. Final Results 4/17/2012 Tom Erickson, 2012 All Rights Reserved 24
- 25. Conclusions • Prediction Markets can provide the capability to efficiently engage the “wisdom of crowds” phenomenon. • Prediction Markets can support all Project/Program Knowledge Areas. • Prediction Markets are not suitable for all projects and populations. 4/17/2012 Tom Erickson, 2012 All Rights Reserved 25
- 26. Questions? 4/17/2012 26
- 27. Acknowledegments • I am grateful to Adam Siegel and the Inkling Markets team for allowing me to include their material in this presentation, as well as their past support and assistance while using their tools for demos and prototypes. 4/17/2012 Tom Erickson, 2012 All Rights Reserved 27
- 28. Contact Information • Tom Erickson: – terickson@apnbok.com – www.linkedin.com/in/tcerickson – 703-201-0021 • Adam Siegel: – adam@inklingmarkets.com – www.inklingmarkets.com – (773) 856-6087 4/17/2012 28
- 29. Backup 4/17/2012 29
- 30. Prediction Market Systems • Commercial Systems – Consensus Point – Gexid – Hollywood Stock Exchange (HSX) – Inkling Markets, Inc. – InTrade/TradeSports – Lumenogic – Nosco – Pro:kons • Open Source Systems – Conversocial – IdeaFutures – MarMix – Serotonin – USIFEX – Zocalo 4/17/2012 http://sefier.in/~irj/archives/long-review-market-prediction-software 30
Editor's Notes
- #3: It all began at a county fair in 1907. There was a contest to guess how much an ox’s meat would weight after butchering. Francis Galton, a polymath of the 19th century, was surprised to find that the medianof the crowds’ answers was within 9 pounds of the actual result. Later he found that the average (mean) of the individual answers was within 1 pound of the butchered weight! James Surowiecki, in a book published in 2004 built on this and other similar observations to describe the phenomenon we refer to today as the Wisdom of Crowds.
- #4: What if you could use the wisdom of crowds to predict future events on your projects?
- #5: The problem – crowds today are not nearly so well-behaved as they were at the turn of the 20th century. Crowds are bigger, spread over very large geographical areas, and can generate unimaginable quantities of data. Today, to tap the wisdom of crowds you need tools, and prediction markets are the answer.
- #8: You thought you could just sit there and pick up a PDU? We are going to conduct a prediction market of sorts during the presentation. (This is probably the first time at a tools session a presenter has told you to keep your cell phones turned on.)When you came in you found a sheet of paper on your chair describing a future fictitious event, and during the presentation I am going to ask you to express your opinion about the outcome. You will be allowed to change your mind as many times as you wish, and you will use your cell phone, or tablet (if you can get on the hotel WiFi or have cellular access) to provide your guesses. Here is the setup in a nutshell.
- #9: We are using a polling engine called Poll Anywhere that can accept cell phone text messages to guess the outcome. I have a poll set up for the scenario described on your handout. Here is how it works. If you gut tells you the IPL “. . . Will do well. Execs will be happy” you will send a text message to 22333 containing only the code 298815.Alternatively you can direct your phone’s browser to PollEV.com/Nebit (case sensitive) and select your choice. You can change your mind as many times as you wish during the presentation.Go ahead. If you have a gut feeling about how the IPL will turn out, text your choice to 22333, or connect with a browser to PollEv.com/Nebit (case sensitive) and pick your best guess. I’ll give you a few minutes before I proceed.(Whistling the Jeopardy theme. . .)If everyone has had a chance to post their initial thoughts, let me go on to describe some of the characteristics of a prediction market.
- #10: Almost all prediction markets share common characteristics. How they may implement these features may vary, but the concepts will be similar.The last item is important. Allowing users to pose their own questions has the double benefit of raising issues or potential problems that might not otherwise occur to management or the prediction market operators as well as providing users a reason to have a vested interest in the market.
- #11: A prediction market is not another type of survey.Surveys ask “what will you do”; prediction markets ask “what will happen”.Surveys are snapshots in time; prediction markets can be continuous.
- #12: Subsequent research on the idea has shown that you need particular characteristics for accurate results from the crowd.
- #13: Bulletin!Does this change your opinion about the success of the Nebit IPL? If so, take a moment to post your choice on the poll.O.K. So much for the theory. Let’s take a look at a real prediction market.
- #14: I will give you a quick guided tour of the Inkling Markets public marketplace. (If you view these slides in presentation mode and click on the graphic it will launch your default browser and open the Inkling Markets public marketplace.)
- #15: Uh-oh – another update from the front.This is good news, right? Does it change your intuition about the success of the Nebit IPL? I’ll give you a few minutes to express your opinion before I proceed.Moving on, I promised I would talk about prediction markets in the context of project and program management. In the next few slides I will show examples of prediction market questions for all project and program management knowledge areas.
- #16: Comparing the knowledge areas for the Standard for Project Management and the Standard for Program Management, we can see they are very similar. Program Management adds three knowledge areas, and as you would expect the process stakeholders and scope for the two standards won’t be the same, but for our purpose we can address both Standards together.
- #17: Here are typical questions you might want to include in a project or program management prediction market for Integration Management, Scope Management and Time Management.
- #18: Here are typical questions you might want to include in a project or program management prediction market for Cost Management, Quality Management and Human Resource Management.
- #19: Here are typical questions you might want to include in a project or program management prediction market for Communications Management, Risk Management and Procurement Management.
- #20: Here are typical questions you might want to include in program management prediction market for Financial Management, Stakeholder Management and Program Governance.
- #21: Looks like theNebit IPL is sailing into troubled waters.What does this bit of news do to your confidence in the success of the Nebit IPL? Take a moment to express your guess on the poll.
- #22: Successful prediction markets need to be managed. Here are several points to keep in mind as you plan for and operate a prediction market on your program.
- #23: Incentives are important to keep people engaged, but prizes usually aren’t the most effective.
- #24: This was the final result of the poll taken throughout the presentation.
- #26: Thank you for your time, and for your questions and comments. Does anyone want to explore any additional thoughts before we wrap up?
- #27: I must acknowledge the support Inkling Markets has provided, with material and advice, in preparing this presentation.
- #28: If you have additional questions feel free to contact me. If you want to know more about Inkling’s products, contact Adam or the team at Inkling Markets – they are the experts.
- #29: For your further reading or research here is a list of commercial and open source prediction market systems and software compiled recently at the URL provided.