![Problem AddressedEnd-user programmers’ code is buggyE.g., [Powell et al. 2008; EuSpRIG 2006]Valentina Grigoreanu (M.A.X.) 4](https://image.slidesharecdn.com/stratceletc6-valentinag-100419174518-phpapp01/85/StratCel-A-Strategy-Centric-Approach-to-the-Design-of-End-User-Debugging-Tools-4-320.jpg)
![Problem AddressedNot only is EUP code buggy…Problem amplified by:Tens of millions of end-user programmers [Scaffidi et al. 2005]Lack of training in software engineering techniques [Study Participants]Increasing size and complexity of spreadsheets [Whittaker 1999]Valentina Grigoreanu (M.A.X.) 5](https://image.slidesharecdn.com/stratceletc6-valentinag-100419174518-phpapp01/85/StratCel-A-Strategy-Centric-Approach-to-the-Design-of-End-User-Debugging-Tools-5-320.jpg)
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging Tools
- 1. Valentina Grigoreanu, Margaret Burnett, George RobertsonOregon State University & Microsoft Corporation04.13.10 – CHI 2010 TalkStrategy-Centric Approach:Design of End-User Debugging Tools Valentina Grigoreanu (M.A.X.) 1
- 2. AgendaProblem + Theoretical FindingsStratCel: New Strategy-Centric Debugging ToolTactics+MovesStratagemsStrategiesValentina Grigoreanu (M.A.X.) 2
- 3. Research Methods: Eight StudiesFactors to consider in future studies (Moves and Tactics)Generalizing across environments, populations, & study setups (Stratagems)A Sensemaking perspective on end-user debugging (Strategies)Designing end-user debugging tools (All four levels)Valentina Grigoreanu (M.A.X.) 3
- 4. Problem AddressedEnd-user programmers’ code is buggyE.g., [Powell et al. 2008; EuSpRIG 2006]Valentina Grigoreanu (M.A.X.) 4
- 5. Problem AddressedNot only is EUP code buggy…Problem amplified by:Tens of millions of end-user programmers [Scaffidi et al. 2005]Lack of training in software engineering techniques [Study Participants]Increasing size and complexity of spreadsheets [Whittaker 1999]Valentina Grigoreanu (M.A.X.) 5
- 6. Related WorkPrevious research / tools to improve end-user debugging:Automatic bug detectionMapping techniques from professional SEFeature usageA critical stone has been left unturned…Valentina Grigoreanu (M.A.X.) 6
- 7. CHI Paper/Dissertation RQHow can we design tools such that they support end-user programmers’ existingeffectivedebugging strategies?What are they?How are they used?Are there gender differences?Do the strategies generalize across EUP environments?How can tools be designed around strategies?What impact would strategy support have on success?Valentina Grigoreanu (M.A.X.) 7
- 8. DefinitionsHas Impact on…Iteratative improvements to Existing Features.New feature innovation.Overview. Stumbling Blocks. Checks.Note: All three of these are needed for strategy-centric design.Valentina Grigoreanu (M.A.X.) 8
- 10. The StratagemsCode InspectionControl FlowDataflowFeedback FollowingHelpProceeding as in Prior ExperienceSpatialSpecification CheckingTestingTo-do ListingValentina Grigoreanu (M.A.X.) 10
- 13. StratCel21 implications for designAt 3 levels of abstraction:Strategies (high)Stratagems (medium)Moves/tactics (low)Valentina Grigoreanu (M.A.X.) 13
- 15. 1. Strategies ImplicationObserved two strategiesComprehensive strategy:Thorough, found easy bugs quickly, lots of fixes within 30 minutesBut… Needed help keeping track of cells to return toSelective strategy:Fixed the hardest bug, found some of the easier ones in the processBut… Needed help keeping track of cells not checkedValentina Grigoreanu (M.A.X.) 15
- 16. Strategies ImplicationSounds a lot like support for the to-do listing stratagem might help:Observed in Forms/3, PowerShell, professional devs.Poorly supported in all: often requires repurposingShows promise for improving debugging successValentina Grigoreanu (M.A.X.) 16
- 17. StratCel:Strategies and To DosDoes not impose an orderValentina Grigoreanu (M.A.X.) 17
- 18. 2. Stratagems ImplicationSupport all strategems in the context of each to-do item (specification checking, testing, code inspection, etc.).Each provides additional relevant information for debugging.Valentina Grigoreanu (M.A.X.) 18
- 19. StratCel:Multi-Stratagem SupportValentina Grigoreanu (M.A.X.) 19
- 20. 3. Moves & Tactics ImplicationsHighlighting formula cellsToo many warnings is as bad as none at allInterworksheet dependenciesValentina Grigoreanu (M.A.X.) 20
- 21. Good Tactics, Bad MovesImplication 3Following dependencies between spreadsheetsHow many of our participants do you think understood this icon when they saw it? Implication for design: at least a tooltip, would be nice. Though a method for navigating to that/those cell(s) would be ideal.Valentina Grigoreanu (M.A.X.) 21
- 22. StratCel: Highlight FormulasValentina Grigoreanu (M.A.X.) 22
- 23. StratCel: Error CheckingValentina Grigoreanu (M.A.X.) 23
- 24. StratCel: Interworksheet DependenciesValentina Grigoreanu (M.A.X.) 24
- 25. Evaluation: Study Setup8 Control participants / 5 Treatment participantsExperience with formulasThink-aloud studyDebugged a grade-book spreadsheet in ExcelSeeded with 10 bugsOr so we thought... There were actually 12!Our StratCel participants found 2 more bugs that the researchers, the spreadsheet’s creator, and the Excel-only participants all had overlooked!Valentina Grigoreanu (M.A.X.) 25
- 26. StratCel Results:TriangulationWhy? Too much feedback is as bad as no feedback.
- 27. Support for selective and comprehensive strats:Selective: used status borders to easily find areas on which to focus attention.Comprehensive: walked through the entire to-do list item-by-item.Bugs Foundp<0.005Valentina Grigoreanu (M.A.X.) 26
- 28. StratCel Results:TriangulationWhy? Found significantly more bugs.
- 29. Contextually-provided information on additional stratagems:
- 30. Specification
- 31. Recently used function library
- 32. Each feature used by at least one participantBugs Fixedp<0.002The time factor:- Debugging is often a time-sensitive activity. On average, Treatment participants found and fixed bugs faster.
- 33. Time to first bug find: p<0.005
- 34. Time to first bug fix: p<0.003Valentina Grigoreanu (M.A.X.) 27
- 35. StratCel Results:TriangulationFoundCF: 4.5, TF: 9.0CM: 5.0, TM: 9.0Fixed:CF: 3.5, TF: 9CM: 2.5, TM: 8.5“I feel like [StratCel] would be extra useful for someone like me who, well, I can use Excel and I can figure it out, but I'm definitely not an expert at Excel. […] I think the only problems I had were with the Excel functions I hadn't learned. This is like a really good way of helping me keep track of what I've done and not get lost."Valentina Grigoreanu (M.A.X.) 28
- 37. Parallels to IR Research- This design approach can be used beyond end-user debugging.- [T]he goal has been formed in much IR research to have the system do the searching for us, but that is not the only alternative. It should also be possible to design search interfaces that harmonize with and make easy the prosecution of good search strategy, systems that make it easy for novices to move quickly into good searching because the system promotes it” [Bates 1990].Valentina Grigoreanu (M.A.X.) 30
- 38. StratCel Results:UsabilityWas StratCel intuitive – a.k.a. did we really support end-user programmers’ debugging strategies?"And this is straight-forward and makes a lot of sense. When you look at it, you know what it is. There are lots of tools, where you can tell that people said, 'well… there's just a workaround and you can just do it this way'. But this one, it just seemed very straightforward and it builds on everything from Excel.”“This was really helpful because it has a way to say these are all your formulas… These are the ones you need to go look at. And I like this part [the address field] which shows me where I can find all of the formulas, so I can see them. For example, on this one, I could see there was a gap for E16and I could go back and look specifically at that cell, because I expect it to be the same, and see what's going on.”Valentina Grigoreanu (M.A.X.) 31
- 39. StratCel Results:Real-World ApplicationsApplicable to real-world tasks:“I think this would be useful for my complex accounting spreadsheets. If you would like to share the tool, I would love to try it on those.”“Looking at [StratCel], I was thinking I have to have a way of tracking my [knitting] patterns. So things that… Ok. I have a pattern and I have steps I have to go through. And I need a way to track them.”"So, can I use your tool? You should sell this and make a million dollars!”Valentina Grigoreanu (M.A.X.) 32
- 40. Validated Design GuidelinesAutomatic error detection tools default: Value quality (low number of false-positives) over quantity (detecting more possible types of errors). Important information about cells (e.g., to-do status) should be overlaid onto the spreadsheet to give the user a quick overview of the to-do status of both individual cells and of the overall spreadsheet. Some users are comprehensive, whereas others will start by trying to fix apparent bugs right away. Support both, since both have pros and cons.Strategy-based tools should provide explicit support for to-do listing. To improve debugging of end-user programs, it helps to automatically generate a list of items to check so that all areas of the code are given equal attention. Information about the remaining strategems should be provided in the context of each to-do item to provide more information on which to base a bug fix. Viewing formulas related to an item (e.g., the consistent formulas, recently used formulas, or formulas used in files in a certain directory) may be particularly useful for improving debugging success.33
Editor's Notes
- #3: Theoretical findings to guide future changes: e.g., what strategies do spreadsheet users employ?Incremental improvements to Excel based on my studiesStratCel: A new tool I built to complement Excel’s support for debugging. I’ve got a provisional patent on StratCel – haven’t yet decided on the next step.
- #4: This CHI paper builds on a whole lot of other studies. So, before I tell you what StratCel is, I’ll give you some highlights from earlier studies.
- #22: They hovered over – nothing happenedThey clicked on it – it selects the cell below itHow do you look up help for something like this?More importantly, how do you smoothly navigate to those dependent cells?