Promise08 Wrapup
- 1. PROMISE 2008: wrap up The PROMISE effect: Do more together than separately Gary D. Boetticher Tim Menzies Tom Ostrand Guenther Ruhe
- 2. The other side of PROMISE PROMISE is not just an annual meeting, it is community. How much contact have you had with your fellow PROMISE-ians during the year? Are you exploiting the “PROMISE effect”? Do more together than separately.
- 3. Case study #1 Cukic & Menzies: new NSF award 2008 to 2011 “ Overcoming ceiling effects in defect predictors Surprisingly good reviews High praise for the connection to the PROMISE repository What are the broader impacts of the proposed activity? “… The work has significant broader impacts and the PROMISE repository that the PIs are a part of provides them a platform for making further contributions to the SE community. …” Summary Statement “… The PROMISE repository and the work of the PI represents a significant input to the proposal. …. “ So the repository is now a tool for access funding.
- 4. Case study #2 For several years, Ostrand, Weyuker, Bell at AT&T held the state of the art in defect localization (predicting the subset of the code with most errors) 2007: At PROMISE’07: Koru argues that defect distribution is logarithmic (I.e. more errors in the smaller modules) Sept 2008: Milton (WVU) tells Menzies that their new defect localizer is failing. Menzies says “try crawling the suspect modules in the Koru order” (smallest modules first) Jan’08: Milton shows the new localizer dramatically out-performs standard methods Mar’08: Milton hosted at AT&T for a week by Ostrand Tests the new localizer inside the firewalls at AT&T
- 5. Case study #3 TSE 2007, May, Kitchenham et.al. assess within-company (WC) data vs cross-company (CC) data for effort prediction. August ‘07 : Menzies (USA) realizes that the PROMISE repository could perform the same study, but for defect prediction. Crowd sources the PROMISE community for collaborators Sept ‘07: Bener&Turhan (Turkey) run the experiments Turkey makes the data, USA runs the statistics CC results: best PD results ever seen with this data (b ut very bad PF rates) Oct ‘07: Submission to TSE by Menzies, Bener, Turhan CC depreciated for defect prediction Feb 14: TSE decision: revise & resubmit May 1: Resubmission with 3 news data sets (Turkish whitegoods) Paper now much stronger PROMISE repository now has 3 more data sets Other Bener, Menzies, Turhan papers: two Defects’08, ASE’08
- 6. Case study 4,5,6…. Over to you… Any other stories to add of interactions between PROMISE-ians?
- 7. The PROMISE effect Do more together separately. Look around the room What ideas are in the air? What is the value-added of the person sitting next to you? What are you plans for collaboration over the next 7 months (in time for the next PROMISE cfp)? And we can use web-tools to foster this community: Chat rooms, discussion groups, wikis, code library
- 8. Proposal: a PROMISE chat room Two minutes chatting can sometimes beat 2 months of reading ?Meebo.com Web-based, platform independent
- 9. Proposal: a PROMISE discussion list Chatting is a hardly permanent record of a discussion Good to access what others have said in the past ?groups.google.com
- 10. Proposal: a PROMISE wiki Promisedata.org is read only (except for comments) Good for workshop admin and data storage Chat rooms and discussion groups good for discussion “volleys” Wikis are better for summarizing and synthesizing on-going discussions write, rewrite access ?wikispaces.org
- 11. Proposal: a shared PROMISE code library Reading is one thing Running code is another. Have a shared open-source, code base, complete with make files Able to reproduce prior results with one command. ?code.google.com ?a Debian package
- 12. Proposal: the PROMISE’09 awards Source award Awarded to the individuals or groups that contribute the most new data, with baseline results, to the repository. Service award Awarded to the individuals or groups that offer most support to others in the new groups/ chat room/ wiki. Science award Awarded to the 2009 paper that best repeats/extends/ interacts with prior results and/or authors of a past PROMISE paper.