CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product
- 2. Learnings from founding a Computer Vision Startup Research vs Developing a Product
- 3. Learnings from founding a Computer Vision Startup Challenge 1: Quality Stability Modularity Robustness Flexibility Flickr: sbisson
- 4. Learnings from founding a Computer Vision Startup Challenge 2: Time to market Effort Estimation Effort vs Reward Priorization Flickr: Kristian M Often in direct conflict with quality
- 5. Learnings from founding a Computer Vision Startup Challenge 3: Changing requirements A new competitor Customer feedback Flickr: 416style A new technology
- 6. Learnings from founding a Computer Vision Startup Flickr: geoffeg Challenge 4: User Experience
- 7. Learnings from founding a Computer Vision Startup Flickr: royskeane Challenge 5: Efficient Teamwork
- 8. Learnings from founding a Computer Vision Startup Summary Challenges Time To Market Quality Efficient Changing Teamwork Requirements User Experience
- 9. Learnings from founding a Computer Vision Startup Running the project Time To Market Quality Needed: a process that is Changing Efficient Teamwork - Flexible (Changing Requirements) Requirements - Quickly leads to results (Time to Market) User Experience - Little overhead (Efficient teamwork) - Shows problems early on (Time to Market and Quality)
- 10. Learnings from founding a Computer Vision Startup Traditional way of running software projects What if requirements change halfway through project? What if unexpected issues arise? waterfall model (image source: Wikipedia)
- 11. Learnings from founding a Computer Vision Startup A better way to run software projects Do short iterations with all phases: iterative process Iteration 1 Examples: - Scrum Iteration 2 - RUP (Rational Unified Process) - ... Iteration 3
- 12. Scrum et al. (Google Tech Talk) Learnings from founding a Computer Vision Startup Ken Schwaber http://www.youtube.com/watch?v=IyNPeTn8fpo
- 13. Learnings from founding a Computer Vision Startup Scrum
- 14. Learnings from founding a Computer Vision Startup Backlogs Requirements written as “User Stories” Example: “iPhone app user can log-in with username and password, in order to log history of requests” Effort for each story is estimated (how, see later) Prioritized list of stories in the Product Backlog. Re-prioritize regularly. For each sprint, just select top-n stories to Sprint Backlog
- 15. Learnings from founding a Computer Vision Startup Sprints Sprint = Iterations (2-4 weeks) Meetings Sprint Planning Meeting -> Team decides how much it can do in the Sprint (not the manager). Set a goal. Daily Scrum 15 minutes max (“timeboxed”). Each team member: What I am working on now? / what’s next? / problems? Check progress on burndown chart. If needed drop tasks. Sprint Review Meeting
- 16. Learnings from founding a Computer Vision Startup NO INTERRUPTIONS DURING SPRINT why?? “People need uninterrupted time to get things done” http://gettingreal.37signals.com/ch07_Alone_Time.php With programmers, it's especially hard. Productivity depends on being able to juggle a lot of little details in short term memory all at once http://www.joelonsoftware.com/articles/fog0000000068.html
- 17. Learnings from founding a Computer Vision Startup Estimations Product Backlog: Stories are estimated roughly Unit: Story points. A measure of “size” / ”complexity” Ex 1: 0, ½, 1, 2, 3, 5, 8, 13, 20, 40, 100 Ex 2: 0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89 Team estimates using Planning Poker Sprint Backlog: Each story gets assigned detailed tasks (in hours) at Sprint Planning Longterm planning: velocity = story points / sprint “we need about 3 sprints to finish this module at our current velocity”
- 18. Learnings from founding a Computer Vision Startup Why a process at all? Why Scrum? You need some process, otherwise you get lost. At the latest with more than 5 people more than 5 customers. Scrum doesn’t get in the way. Simple. Lightweight. “Advanced common sense” * Perfect for startups * David Allen, in order to describe “Getting Things Done”
- 19. Learnings from founding a Computer Vision Startup http://agilemanifesto.org/ Scrum is part of Agile
- 20. Learnings from founding a Computer Vision Startup More Agile Practices (or: more common sense) Testing / Test Driven Development Time To Market Quality Code refactoring Efficient Changing Teamwork Requirements User Experience Continuous Integration Pair Programming
- 21. Learnings from founding a Computer Vision Startup If you don’t like Agile ... http://www.joelonsoftware.com/articles/fog0000000043.html
- 22. Learnings from founding a Computer Vision Startup Consolidated must-haves Subversion: http://subversion.tigris.org/ Source control (no-brainer) Git: http://git-scm.com/ Google Test (http://code.google.com/p/googletest/) Automated Testing Often part of good frameworks (e.g. Rails) CDash http://www.cdash.org/ Continuous Integration Cruise Control http://cruisecontrolrb.thoughtworks.com/ Automated daily builds (Ruby only) Just give your product to a few random people (in person) Hallway Usability Testing and observe them using it (before release)
- 23. Learnings from founding a Computer Vision Startup On Deployment - We have 20 million items indexed. Need multiple machines to keep index - What if we have many requests, so we need to replicate the whole system - Buying and maintaining so much hardware is expensive Amazon Web Services (“Cloud Computing”) to the rescue
- 24. Learnings from founding a Computer Vision Startup On Outsourcing Work Do not outsource core competence Overhead from time difference, communication, project management over distance, travel Graphic design, administration etc. are things to outsource first (www.elance.com)
- 25. What is special about Vision? In Terms of Engineering
- 26. Learnings from founding a Computer Vision Startup What’s special about Vision (Consumer) Software is a lot harder to do than e.g. a web-community in PHP (means you need more money also) Lot’s of uncertainty and complexity Still early stage of technology Hard to find good specialists
- 27. How we did it
- 28. Learnings from founding a Computer Vision Startup How we did it - Two Scrum-Teams in parallel - Use of Story points - 3-week sprints - 2-day break between sprints Ec2, Trac + Agilo, C++, Rails, Googletest, SVN, CDash, Reviewboard In-house: Vision, Rails, iPhone Outsourced: Android, Symbian etc.
- 29. Learnings from founding a Computer Vision Startup Challenges we still face User stories for system related topics Uncertainty in Computer Vision related stories/tasks Testing can still be improved
- 30. Learnings from founding a Computer Vision Startup Polar Rose: How we did it - Two teams (infrastructure and vision) - Sys admin & dedicated Q/A tester - 2-3 week sprints - daily and weekly standup Skype meetings Ec2, S3, C++, Python Java, Jira, Confluence, WordPress SVN, Bamboo In-house: Vision, Infrastructure, Web, iPhone/iPad, Android outsourced: Symbian, small projects/prototypes
- 31. Q&A
- 32. Learnings from founding a Computer Vision Startup Resources II Ken Schwaber Scrum http://www.youtube.com/watch?v=IyNPeTn8fpo Agile Manifesto http://agilemanifesto.org/ http://www.joelonsoftware.com/articles/ The Joel Test fog0000000043.html Wikipedia: Scrum http://en.wikipedia.org/wiki/Scrum_(development) Wikipedia: Planning Poker http://en.wikipedia.org/wiki/Planning_poker