Building Smart Software
1 like6,822 views
The document discusses building smarter collaboration products at Atlassian using machine learning. It covers exploring what machine learning can do with data, such as classifying customer feedback and predicting issue assignees. It also discusses using machine learning to help users find information, such as routing support requests to the appropriate team or person.
Building Smart Software
- 1. SHIHAB HAMID • PRINCIPAL PRODUCT MANAGER • ATLASSIAN Building Smarts
- 4. Machine Learning Talking Point Data Science Product Detailed algorithms Tweaking parameters
- 5. Machine Learning Talking Point Data Science Product
- 6. What is a ‘smart’ feature?
- 8. Dear so and so, …. 12 Oct 2016
- 9. What’s the problem we’re trying to solve?
- 10. Smart is a solution, not a customer problem
- 11. Talking Point Talking Point What can we do with the data? What problem are we solving?
- 12. Talking Point Talking Point What can we do with the data? What problem are we solving?
- 13. Talking Point Talking Point What can we do with the data? What problem are we solving?
- 14. SIMPLE SMARTS DEEP DIVE DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT Agenda
- 15. SIMPLE SMARTS DEEP DIVE DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT Agenda
- 16. Agenda SIMPLE SMARTS DEEP DIVE DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT
- 18. 20,000+ teams
- 19. Service Team IT, Support Customers Employees, Clients
- 21. Request Types
- 24. 20% Time Accelerate U Advocates Afterhours Sales Support Analytics & Integration Atlassian Allstars Build Engineering C4L Hiring Channel Programs Chat Service Cloud Engineering (AWS) Cloud Support Community Content Marketing Corporate Events Customer Contact Research Customer Marketing Customer Science Customer Support Tech Data Engineering Data Sciences Enablement EPA Requests Executive Briefing Center Experience Feature Flag Service Field Marketing Finance & People Systems Risk and Compliance Incident Mgmt Feedback Infrastructure Services Japan Office Service Desk JIRA Support Sydney KPI Tracking Kudos Voice of the Customer Localisation Marketing Technology Mergers & Acquisitions Network Engineering Open Source Requests Operations Engineering Platform Engineering Premier Support Internal Product Advocates Public Relations Pulse Core Billing Purchasing SRE Purchasing Systems Repository Compliance Salesforce Help SalesOps Express Security Intelligence Service Desk Marketing Stock Administration Strategy & Analytics Swag Tax Desk Videos Website A/B
- 25. I need help! But I don’t know where to go :( Problem worth solving
- 29. Stop Words Stemming TF-IDF Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form.
- 30. Stop Words Stemming TF-IDF Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form.
- 31. Stop Words Stemming TF-IDF Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form.
- 32. Stop Words Stemming TF-IDF Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form.
- 35. That search is so broken! It never finds what I need :( I spilt coffee all over my keyboard, I just need ‘new keyboard’
- 36. Welcome to the Help Center new hardware Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form.
- 37. Welcome to the Help Center second monitor Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. Report Incident You can report an incident here. Note that monitoring tools automatically report outages.
- 38. Welcome to the Help Center keyboard No request forms were found for ‘keyboard’. Try something else.
- 39. hardware monitor printer keyboard mouse computer power Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. laptop desktop trackpad ipad hdmi projector scanner copier
- 40. software microsoft office adobe word excel powerpoint outlook photoshop illustrator windows sketch keynote intellij omnigraffle textmate Software Request We can provide licenses to software like Microsoft Office, Adobe Creative Suite, etc.
- 41. I need help! But I don’t know where to go :( PROBLEM
- 42. I need help! But I don’t know where to go :( I know the specific thing I want, but I don’t know the category SUB-PROBLEMPROBLEM
- 43. Learn what ML can do TIP
- 44. I know the specific thing I want, but I don’t know the category SUB-PROBLEM
- 45. Curated Labels Search Classification Issue Classification Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. KEYBOARD LAPTOP DESKTOP MOUSE COMPUTER TRACKPAD
- 46. Curated Labels Search Classification Issue Classification Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. KEYBOARD LAPTOP DESKTOP MOUSE COMPUTER TRACKPAD
- 47. Curated Labels Search Classification Issue Classification Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. new keyboard laptop upgrade broken mouse keyboard
- 48. Curated Labels Search Classification Issue Classification Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. new keyboard laptop upgrade broken mouse keyboard
- 49. Curated Labels Search Classification Issue Classification Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. new keyboard I need an ergonomic keyboard My keyboard broke
- 50. Curated Labels Search Classification Issue Classification Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. new keyboard I need an ergonomic keyboard My keyboard broke
- 51. Welcome to the Help Center work from home No request forms were found for ‘work from home’. Try something else.
- 52. Welcome to the Help Center help General IT help If you have any general IT inquiries ask us and we’ll get back to you shortly
- 53. I need to work from home General IT help General IT help
- 54. Welcome to the Help Center work from home General IT help If you have any general IT inquiries ask us and we’ll get back to you shortly
- 55. I need to work from home General IT helpVPN VPN Access
- 56. Welcome to the Help Center work from home VPN Access Request access for a virtual private network key and associated software.
- 57. Curated Labels Search Classification Issue Classification Hardware Request If you need additional hardware, like a monitor or a printer, raise a request using this form. new keyboard I need an ergonomic keyboard My keyboard broke
- 59. Search text Final request type 10 min
- 61. Time Query Request Type Service Desk 13:49 computer ? ?
- 62. Time Query Request Type Service Desk 13:49 computer ? ? 13:49 hardware Hardware Request IT
- 63. Time Query Request Type Service Desk 13:49 computer Hardware Request IT 13:49 hardware Hardware Request IT
- 64. Iterating algorithms Search Algorithm new keyboard New Account New Furniture Hardware Request Desktop Support Musical Instruments
- 67. Issue History Classification Iterating algorithms 58% Stop Words Stemming TF-IDF Keywords 31% 42%
- 68. HybridIssue History Classification Iterating algorithms 58% 67% Stop Words Stemming TF-IDF Keywords 31% 42%
- 69. mouse
- 70. new employee
- 71. pizza
- 72. In the wild 30-33% 48-52% S ≤ 200 req/month M-L > 200 req/month
- 73. In the wild 30-33% 48-52% 100%
- 74. 0
- 75. I need help! But I don’t know where to go :( Problem worth solving
- 77. Agenda SIMPLE SMARTS DEEP DIVE DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT
- 78. Agenda SIMPLE SMARTS DEEP DIVE DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT
- 79. Agenda SIMPLE SMARTS DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT DEEP DIVE
- 80. @mentions
- 81. Hey @ronald do you know if the router on level 3 is still playing up?
- 82. Section title 1
- 83. Section title 1
- 84. Hard to @mention who I want adoption users PROBLEM
- 85. @ron
- 86. @david
- 87. @dave
- 90. Problem worth solving Hard to @mention who I want
- 91. when they are… SUB-PROBLEM 1 not involved in the issue Hard to @mention who I want PROBLEM
- 92. when they are… SUB-PROBLEM 1 not involved in the issue Hard to @mention who I want already involved in the issue SUB-PROBLEM 2 PROBLEM
- 93. when they are… Hard to @mention who I want already involved in the issue SUB-PROBLEM 2 PROBLEM
- 95. already involved in the issue Hard to @mention who I want when they are
- 96. Issue Assignment
- 97. Who’s the best person to work on a ticket? tickets timeassigning PROBLEM
- 103. Project Lead Component Lead Automation Rule
- 104. Project Lead Component Lead Automation Rule
- 105. Project Lead Component Lead Automation Rule WHEN Issue created Issue commented Issue transitioned SLA about to breach IF Issue matches JQL Comment is public/private Comment contains text User is agent/customer THEN Transition issue Edit issue Assign issue Alert user
- 106. Project Lead Component Lead Automation Rule “ab-guru” ..blah A/B blah..
- 107. Project Lead Component Lead Automation Rule “team-lead”
- 108. Who’s the best person to work on a ticket? Problem worth solving
- 109. Agenda SIMPLE SMARTS DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT DEEP DIVE
- 110. Agenda SIMPLE SMARTS DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT DEEP DIVE
- 111. Experimental
- 112. Who’s the best person to work on a ticket? Problem worth solving
- 113. Section title 1
- 114. Section title 1
- 115. Section title 1
- 116. Section title 1
- 117. Section title 128%Predict Assignee
- 118. Section title 128%Predict Assignee 58% Predict 3 Potential Assignees
- 120. Talking Point Talking Point What else can we do with the data?
- 125. Machine Learning Talking Point Exploring Solving
- 127. pieces of content 24
- 128. What should I be reading? users content PROBLEM
- 131. Exploring
- 135. Section title 1
- 139. Agenda SIMPLE SMARTS DATA EXPLORATIONS SMARTS OUTSIDE PRODUCT DEEP DIVE
- 140. What do our customers think about our products? customers feedback PROBLEM
- 142. Issue Tracker Customer feedback channels
- 143. SupportIssue Tracker Customer feedback channels
- 144. SupportIssue Tracker Customer Interviews Customer feedback channels
- 145. SurveysSupportIssue Tracker Customer Interviews Customer feedback channels
- 146. NPS Question In-product feedback Overview of what they think
- 147. NPS Question In-product feedback Overview of what they think
- 148. NPS Question In-product feedback Overview of what they think Feedback Free text allows for specific feedback
- 149. In-product feedback
- 152. What do our customers think about our products? Problem worth solving
- 153. SUB-PROBLEM How can we summarise customer feedback? What do our customers think about our products? PROBLEM
- 154. FunctionalityUsabilityReliability Experience Learning Curve Visual Interaction R U F Bugs Performance Outages New Features Improvements Missing Capabilities
- 155. R U F
- 156. R U F slow broken error complex can’t find setup like to i wish customize
- 157. RUF Classifier R
- 159. What do our customers think about our products? Problem worth solving
- 160. R U F
- 161. R U F
- 162. R U F
- 163. R U F
- 164. U Ease to learn & use Creating content Finding content Aesthetics & design Integrations
- 165. What do our customers think about our products? Problem worth solving
- 166. How your insights are affecting change in 2017 Sean Cramer Head of Voice of the Customer, Atlassian Thursday 2:30pm
- 167. Who can answer my question? employees information PROBLEM
- 170. I'm locked out of my account I'm having wifi problems Can I get a new computer / mouse / keyboard? I can't sign into X system Where can I raise a ticket with X team? Where is X meeting room?
- 171. Charlie
- 172. I can’t connect to the printer Gerry Tan 1:57 PM. Yeah, thanks! Gerry Tan 2:00 PM. I have found a couple of articles that might help: 1. Setting up printers in Sydney 2. How to connect to the wifi The Charlie Bot 1:57 PM. Did Setting up printers in Sydney help solve your problem? The Charlie Bot 1:59 PM. Awesome! Glad I could help The Charlie Bot 2:00 PM.
- 173. How do I get a new laptop? Gerry Tan 2:15 PM. Yeah, thanks! Gerry Tan 2:22 PM. I have found a couple of articles that might help: 1. Policy for employee laptop updgrades 2. Choosing between Mac and Windows I have also found a few requests you can make: 1. Desktop / Laptop Support 2. Hardware Request The Charlie Bot 2:15 PM. It looks like you have created a Hardware Request. Was that what you were looking for? The Charlie Bot 2:20 PM. Awesome! Glad I could help The Charlie Bot 2:22 PM.
- 174. How do I get a new laptop? Gerry Tan 2:30 PM. You can create a request on service desk: Hardware Request The Charlie Bot 2:31 PM. It looks like you have created a Hardware Request. Was that what you were looking for? The Charlie Bot 2:20 PM.
- 175. How do I get a new swipe card for the building? Gerry Tan 2:35 PM. Nah Gerry Tan 2:38 PM. I have found a couple of articles that might help: 1. Swipe cards for new employees 2. After hours building access The Charlie Bot 2:35 PM. Did any of these help you? The Charlie Bot 2:38 PM.
- 177. ok Gerry Tan 2:55 PM. Go to reception on Level 6 - they have spare cards available Gerry Tan 2:55 PM. Sorry to hear that! If you do find a solution, let me know and I’ll remember it for next time. The Charlie Bot 2:39 PM. When someone says “How do I get a new swipe card for the building?” what should I say? The Charlie Bot 2:55 PM. Got it! Thanks Gerry The Charlie Bot 2:55 PM.
- 178. who is Vincent? Gerry Tan 4:01 PM. Vincent Wong Gerry Tan 4:01 PM. where does he sit? Gerry Tan 4:02 PM. Which Vincent did you mean? 1. Vincent Choy 2. Vincent Kok 3. Vincent Wong The Charlie Bot 4:01 PM. Vincent Wong is a Product Manager on the JIRA Service Desk team The Charlie Bot 4:01 PM. Vincent Wong is on Level 4 on 155 Clarence Street, Sydney The Charlie Bot 4:02 PM.
- 180. Request Type Search @mentions Issue Assignment Content Recommendations Feedback Summarisation Conversational Chat Bot
- 181. Talking Point What can we do with the data?
- 182. Talking Point What problem are we solving? Talking Point What can we do with the data?
- 183. Thank you! SHIHAB HAMID • PRINCIPAL PRODUCT MANAGER • ATLASSIAN
- 184. Image Credits • http://resources.workable.com/wp-content/uploads/2015/09/machine_learning.jpg • http://static1.squarespace.com/static/52810071e4b0f8aebbcac9a5/t/53e04bede4b0a545d9af6b86/1407208477715/Arrows+in+Target+Gator+Cup.JPG?format=1500w • https://www.google.com/selfdrivingcar/images/home-where.jpg • http://i.cbc.ca/1.3489012.1457821866!/fileImage/httpImage/image.jpg_gen/derivatives/original_620/south-korea-game-human-vs-computer.jpg • https://tvkinoradio.ru/upload/ckeditor/article/images/b21516d7.jpg • https://www.crowdanalytix.com/api/assets/download?path=blogs/15/file.jpg • http://littlebinsforlittlehands.com/wp-content/uploads/2015/02/Lego-Rainbow-Challenge-Building-Rainbows-Activity.jpg • http://i2.kym-cdn.com/photos/images/facebook/000/024/029/Clippy-letter20110724-22047-qcxdai.PNG • https://campbowwowscs.files.wordpress.com/2014/09/dog-food.jpg • http://ep.yimg.com/ay/yhst-14016457918231/plan-toys-50-unit-blocks-9.jpg • http://www.payreel.com/wp-content/uploads/2013/02/RockInJar.jpg • http://static1.squarespace.com/static/52810071e4b0f8aebbcac9a5/t/53e04bede4b0a545d9af6b86/1407208477715/Arrows+in+Target+Gator+Cup.JPG?format=1500w