VLDB Slides on Making Sense of Applying ML to APIs
Download as PPTX, PDF2 likes538 views
The document discusses the integration of machine learning (ML) with APIs, emphasizing the importance of data preparation and the need for human oversight in tasks suited for automation. It outlines challenges such as the dependency on good labeled data, the necessity for sufficient data for analysis, and explores potential research areas like anomaly detection and explainability in ML. Key lessons highlight that ML can enhance processes but will not eliminate the need for databases or human involvement in data interpretation.
![Proprietary and confidentialProprietary and confidential
20
Proprietary and confidential
Garbage In, Garbage Out...
http_pattern = r'(https|http)?://(w|.|/|?|=|&|%|-|#)*b'
html_pattern = '<[^<]+?>'
injection_pattern = r"">"
json_pattern = '{.*}+'
email_pattern = r'[w.-]+@[w.-]+(.[w]+)+'
quote_pattern = r'["']'
nonalpha_pattern = "[^A-Za-z0-9.? ]+"
patterns = [http_pattern, html_pattern, injection_pattern, json_pattern, email_pattern,
quote_pattern, nonalpha_pattern]
pattern = "|".join(patterns)
def substitute (text):
text = re.sub("[Cc]an't", "cannot", text)
text = re.sub("I'm", "I am", text)
text = re.sub(" +", " ", text)
return text
def cleanup(text):
text = re.sub(pattern, ' ', text)
text = substitute (text)
return text](https://image.slidesharecdn.com/vldbslides-181203221226/85/VLDB-Slides-on-Making-Sense-of-Applying-ML-to-APIs-20-320.jpg)
![Proprietary and confidentialProprietary and confidential
[glacier]
Google Photos
34](https://image.slidesharecdn.com/vldbslides-181203221226/85/VLDB-Slides-on-Making-Sense-of-Applying-ML-to-APIs-34-320.jpg)
VLDB Slides on Making Sense of Applying ML to APIs
- 1. Proprietary and confidential Making Sense of Applying ML to APIs Anant Jhingran, CTO, Apigee (Google) Sridhar Rajagopalan, Chief Architect, Data, Apigee (Google)
- 2. Proprietary and confidentialProprietary and confidential 2 Proprietary and confidential APIs are everywhere
- 3. Proprietary and confidentialProprietary and confidential Check social media = API
- 4. Proprietary and confidentialProprietary and confidential Flight check in = API
- 5. Proprietary and confidentialProprietary and confidential Online purchase = API
- 6. Proprietary and confidentialProprietary and confidential
- 7. Proprietary and confidentialProprietary and confidential 7 Proprietary and confidential Apigee Full Lifecycle API Management Platform Deliver connected experiences for customers, employees and partners /idm/account /erp/pricing/id /cms/images/item /inventory/itemIDAPIs /catalog /payment /customer Internet of Things Legacy Apps Cloud Apps Employee Apps Consumer Apps Partner Apps API Management Platform ExposureConsumption
- 8. Proprietary and confidential RETAIL .34 GOVERNMENT TRAVEL & HOSPITALITY OTHER FINANCIAL SERVICES MEDIA & ENTERTAINMENT TECHNOLOGY MANUFACTURING HEALTHCARE TELCO EDUCATION 700+ Customers, > 1T calls/year
- 9. Proprietary and confidentialProprietary and confidential 9 Proprietary and confidential A Sample Response from Google Places API
- 10. Proprietary and confidentialProprietary and confidential 10 X 1,000,000,000,000
- 11. Proprietary and confidential With so much data, what can we do?
- 12. Proprietary and confidentialProprietary and confidential 12 Visibility for API Teams Custom Reports Business and Operational Analytics Out of the Box Dashboards
- 13. Proprietary and confidentialProprietary and confidential
- 14. Proprietary and confidentialProprietary and confidential Predict if an API call is from a “bot”
- 15. Proprietary and confidentialProprietary and confidential Predict time to resolution of a support ticket We are seeing 500n server side errors impacting customers for second time today {"organization":"promisepay","environment":"prod","apiProduct":null,"pro xyName":"prelive_rev9_2016_10_04","appName":null,"verb":"GET","url": "https://secure.api.promisepay.com/status","responseCode":500,"respon seReason":"Internal Server Error","clientLatency":9,"targetLatency":0,"totalLatency":9,"remote_ip":"1 92.168.61.56","correlation_id":"rrt-089f6aa9f9c90adaa-b-wo-30508- 63404252- 1","request_host":"secure.api.promisepay.com","platform_name":"unkno wn","platform_quota_volume":"null/null","endpoint_quota_volume":"null/n ull","platform_quota_period":"nullmin","platform_spike_volume":"null/null" ,"endpoint_spike_volume":"null/null","asm_quota_req_quota_platform_k ey":"unknown-status","asm_quota_all_quota_default_key":"default- all","asm_quota_all_quota_platform_key":"unknown-all"} Start at 15:36 ESDT
- 16. Proprietary and confidentialProprietary and confidential Predict correctly that this rollout is troublesome
- 17. Proprietary and confidentialProprietary and confidential 17 In other words, we will only focus on “predictions”
- 18. Proprietary and confidentialProprietary and confidential 18 Proprietary and confidential “Wow! ML” “BI level Analysis” “Data Prep” Don’t Predictions need ML? “Sometimes needed” “Always Needed, and often enough for predictions” “Always Needed” ML at the top might or might not be needed, but ML helping at the bottom seems really promising
- 19. Proprietary and confidential Lesson #1: Dataprep is always needed
- 20. Proprietary and confidentialProprietary and confidential 20 Proprietary and confidential Garbage In, Garbage Out... http_pattern = r'(https|http)?://(w|.|/|?|=|&|%|-|#)*b' html_pattern = '<[^<]+?>' injection_pattern = r"">" json_pattern = '{.*}+' email_pattern = r'[w.-]+@[w.-]+(.[w]+)+' quote_pattern = r'["']' nonalpha_pattern = "[^A-Za-z0-9.? ]+" patterns = [http_pattern, html_pattern, injection_pattern, json_pattern, email_pattern, quote_pattern, nonalpha_pattern] pattern = "|".join(patterns) def substitute (text): text = re.sub("[Cc]an't", "cannot", text) text = re.sub("I'm", "I am", text) text = re.sub(" +", " ", text) return text def cleanup(text): text = re.sub(pattern, ' ', text) text = substitute (text) return text
- 21. Proprietary and confidentialProprietary and confidential 21 A good area of research is applying ML to the data prep problem
- 22. Proprietary and confidential Lesson #2: Databases are not likely to be eliminated any time soon. Aka: select count(*) goes a loooonng way
- 23. Proprietary and confidentialProprietary and confidential Bot Detection Rules
- 24. Proprietary and confidentialProprietary and confidential 24 Many of these are expressible as SQL – deep learning might give marginal better results, BUT…
- 25. Proprietary and confidentialProprietary and confidential 25 Customers always want explainability
- 26. Proprietary and confidentialProprietary and confidential 26 A good area of research is applying ML to Explainability←→ Predictability Tradeoff in SQLs
- 27. Proprietary and confidential Lesson #3: Humans are not likely to be eliminated any time soon
- 28. Proprietary and confidentialProprietary and confidential 28 Proprietary and confidential Public API calls Distinguishing QA from bot traffic... API QA
- 29. Proprietary and confidentialProprietary and confidential 29 A good area of research is transfer learning from a huge number of QA tasks
- 30. Proprietary and confidential Lesson #4: ML replaces some human tasks, not select count(*)
- 31. Proprietary and confidentialProprietary and confidential 31 Proprietary and confidential Visually detecting Anomalies does not work at scale
- 32. Proprietary and confidentialProprietary and confidential 32 A good area of research is ML for anomaly detection
- 33. Proprietary and confidential Lesson #5: Without good labeled data, you get nowhere
- 34. Proprietary and confidentialProprietary and confidential [glacier] Google Photos 34
- 35. Proprietary and confidentialProprietary and confidential
- 36. Proprietary and confidentialProprietary and confidential 36 But labeling improves precision, not recall
- 37. Proprietary and confidentialProprietary and confidential 37 A good area of research is ML for giving labeling tasks to humans
- 38. Proprietary and confidential Lesson #6: Without enough data, you get nowhere
- 39. Proprietary and confidentialProprietary and confidential 39 Proprietary and confidential APIs Traffic Have enough data for ML Not enough data to do anything, but we could pool similar APIs together to get enough data.. Is the new deployment of API or platform bad?
- 40. Proprietary and confidentialProprietary and confidential 40 A good area of research is learning about anomalies by grouping a number of similar small data sets.
- 41. Proprietary and confidential Lesson #7: Data litter will always get you.
- 42. Proprietary and confidentialProprietary and confidential 42 1. Latency (minutes) at which 99% of data is ingested into the warehouse. Let’s say we have an internal goal of X minutes. 2. Regardless of when you run an analysis, you will not get 100% of the data. So you have to accept that any timely analysis will have incomplete data. 3. Tradeoff between timeliness of analysis, completeness of data, and cost of collection is fundamental.
- 43. Proprietary and confidentialProprietary and confidential 43 A good area of research is to be able to distinguish whether missing data is random noise or systematic drops.
- 44. Proprietary and confidential Of course, “Wow! ML” is also needed for end user problems
- 45. Proprietary and confidentialProprietary and confidential 45 Proprietary and confidential Rapidly accelerating use of deep learning at Google 2012 2018 and more …
- 46. Proprietary and confidentialProprietary and confidential 46 Proprietary and confidential But it is as much needed in the boiler room... “Wow! ML” “BI level Analysis” “Data Prep” ML at the top might or might not be needed, but ML helping at the bottom seems really promising
- 47. Proprietary and confidentialProprietary and confidential 47 Proprietary and confidential Lesson #1: Dataprep is always needed. Lesson #2: Databases are not likely to be eliminated any time soon. Lesson #3: Humans are not likely to be eliminated any time soon. Lesson #4: ML replaces some human tasks, not select count(*). Lesson #5: Without good labeled data, you get nowhere. Lesson #6: Without enough data, you get nowhere. Lesson #7: Data litter will always get you.
- 48. Proprietary and confidentialProprietary and confidential 48 Don’t go chasing the next publication on reinforced, one-shot, transfer learning using gensim word2vec model for ImageNet, chase it for index selections, query optimizations, data prep, labeling… :)
- 49. Proprietary and confidentialProprietary and confidential https://cyberomin.github.io/startup/2018/07/01/sql-ml-ai.html
- 50. Proprietary and confidentialProprietary and confidential https://cyberomin.github.io/startup/2018/07/01/sql-ml-ai.html ML/AI Applied to often
- 51. Proprietary and confidential Thank You!