Protecting Artificial Intelligence/Machine Learning Inventions in the United States - Knobbe Practice Webinar Series
0 likes412 views
The document discusses protecting artificial intelligence and machine learning inventions through various intellectual property approaches. It provides an overview of different machine learning models including supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. It then discusses ways of protecting different aspects of machine learning technologies including data set generation, machine learning processing, and results/post-processing through contracts/copyrights, data privacy laws, and patents. Specific examples of patentable machine learning inventions are also provided. The document concludes with best practices for patenting artificial intelligence technologies in both the US and Europe.
Protecting Artificial Intelligence/Machine Learning Inventions in the United States - Knobbe Practice Webinar Series
- 1. Protecting Artificial Intelligence/Machine Learning Inventions in the United States Knobbe Europe Practice Series December 3, 2021 Mauricio Uribe mauricio.uribe@knobbe.com Vlad Teplitskiy vlad.teplitskiy@knobbe.com Harnik Shukla harnik.Shukla@knobbe.com
- 2. Machine Learning/Artificial Intelligence • Minimal Requirements for an Algorithm to be ML ⎼ Representation– Classifiers or basic language that a computer can understand ⎼ Evaluation - Inputting data and generating output (score) ⎼ Optimization - Developing a strategy to get from inputs to outputs 1 Supervised Learning 2Unsupervised Learning 3Semi-Supervised Learning 4Reinforcement Learning Learning Models
- 3. Introduction to Machine Learning – Different Machine Learning Models General Characteristics • Basic Concept: Machine learning is programmed with expected outputs (e.g., labeled training set) to generated learned algorithm • Quality of performance of the learned algorithm is dependent on the training set 3 SUPERVISED LEARNING UNSUPERVISED LEARNING SEMI- SUPERVISED REINFORCEMENT LEARNING
- 4. Introduction to Machine Learning – Different Machine Learning Models General Characteristics • Basic Concept: Machine learning is programmed without labeled data (e.g., unlabeled data without human influence) to generate output • Real-time analysis without pre-existing data using only logic operations • No training provided to the machine learning algorithm 4 SUPERVISED LEARNING UNSUPERVISED LEARNING SEMI- SUPERVISED REINFORCEMENT LEARNING
- 5. Machine Learning Outputs – Regression vs. Classification（回帰と分類） • Classification: A model (function) which helps in separating the data into multiple categorical classes. ⎼ Data is categorized under different labels according to parameters ⎼ Labels are predicted for the data. • Regression/Continuous: A model (function) distinguishing the data into continuous real values instead of categorical classes. ⎼ Function attempts to approximate value with the minimum error deviation. ⎼ No labels 5 0 100 40% 0 100 Continuous/Regression Classification No Rain Rain Output = “40%” Output = “No Rain”
- 6. Exemplary Algorithm Types 6 • Association Rule Analysis • Apriori • Equivalence Class Transformation • FP-Growth • Hidden Markov Model • Classification • K-Nearest Neighbors • Decision/Boosted Trees • Logic Regression/Naive-Bayes • Neural Networks • Support Vector Machine (SVM) • Clustering and Dimensionality • K-Means • Singular Value Decomposition • Principle Component Analysis • Regression • Linear Regression • Polynomial Regression • Decision Trees • Random Forests Unsupervised Learning Algorithms Supervised Learning Algorithms Classification Output Continuous Output
- 7. Introduction to Machine Learning – Different Machine Learning Models General Characteristics • Combination of labeled and unlabeled data sets • Mitigates cost of labeling data for larger data sets • Mitigates some human bias for the unlabeled data 7 SUPERVISED LEARNING UNSUPERVISED LEARNING SEMI- SUPERVISED REINFORCEMENT LEARNING
- 8. Introduction to Machine Learning – Different Machine Learning Models General Characteristics • Introduction of reward function to allow algorithm to adapt • Includes the utilization of randomization of values based on reward function 8 SUPERVISED LEARNING UNSUPERVISED LEARNING SEMI- SUPERVISED REINFORCEMENT LEARNING
- 9. Comparison of Supervised Learning to Reinforcement Learning Supervised Learning Algorithms Reinforcement Learning Algorithms Supervised Learning Training Data Inputs Outputs Supervised Learning Training Data Inputs Outputs Reward/Penalty
- 10. Protecting ML Technologies 10 • Contract/Copyright • Data Privacy • Potential Patentable Subject Matter • Contract/Copyright • Data Privacy • Potential Patentable Subject Matter • Contract/Copyright • Data Privacy • Potential Patentable Subject Matter Data Set Generation and Inputs ML Processing ML Results and Post Processing
- 11. Protecting ML Technologies - Data Set Generation and Inputs • Contract/Copyright ⎼ Securing data rights from users or third-parties • Data Privacy ⎼ Providing necessary information ⎼ Maintaining data appropriately • Potential Patentable Subject Matter ⎼ Collecting or Forming Data Set ⎼ Supplementing Data Set 11 CONTRACT/ COPYRIGHT DATA PRIVACY PATENTABLE SUBJECT MATTER
- 12. Protecting ML Technologies - ML Processing • Contract/Copyright ⎼ Third-party ML processing services • Data Privacy ⎼ Providing data to third-party services ⎼ Maintaining data appropriately • Potential Patentable Subject Matter ⎼ Modifications/Improvements to Al algorithms 12 CONTRACT/ COPYRIGHT DATA PRIVACY PATENTABLE SUBJECT MATTER
- 13. Protecting ML Technologies - ML Results and Post Processing • Contract/Copyright ⎼ Limitations/restrictions of the generated result • Data Privacy ⎼ Maintaining processed data appropriately • Potential Patentable Subject Matter ⎼ Post-processing feedback ⎼ Use of ML processed data 13 CONTRACT/ COPYRIGHT DATA PRIVACY PATENTABLE SUBJECT MATTER
- 14. Protecting ML Technologies 14 • Contract/Copyright ⎼ Securing data rights from users or third- parties • Data Privacy ⎼ Providing necessary information ⎼ Maintaining data appropriately • Potential Patentable Subject Matter ⎼ Collecting or Forming Data Set ⎼ Supplementing Data Set • Contract/Copyright ⎼ Third-party ML processing services • Data Privacy ⎼ Providing data to third- party services ⎼ Maintaining data appropriately • Potential Patentable Subject Matter ⎼ Modifications/Improve ments to Al algorithms • Contract/Copyright ⎼ Limitations/restrictions of the generated result • Data Privacy ⎼ Maintaining processed data appropriately • Potential Patentable Subject Matter ⎼ Post-processing feedback ⎼ Use of ML processed data Data Set Generation and Inputs ML Processing ML Results and Post Processing
- 15. ML Patent Classification – Class 706 - DATA PROCESSING - ARTIFICIAL INTELLIGENCE Issued U.S. Patents (Class 706): 13,537 15 Comparison of Section 101 Rejections “Business Method” vs. ML Allowance Percentage “Business Method” vs. ML Source: Artificial Intelligence Technologies Facing Heavy Scrutiny at the USPTO, IP Watchdog, November 28, 2018.
- 16. A computer-implemented method of training a neural network for facial detection comprising: collecting a set of digital facial images from a database; applying one or more transformations to each digital facial image including mirroring, rotating, smoothing, or contrast reduction to create a modified set of digital facial images; creating a first training set comprising the collected set of digital facial images, the modified set of digital facial images, and a set of digital non-facial images; training the neural network in a first stage using the first training set; creating a second training set and digital non-facial images that are incorrectly detected as facial images after the first stage of training; and training the neural network in a second stage using the second training set. USPTO’s Neural Network Example
- 17. Deep Learning 17 Neural Network Trained Network Trained Network Input Data Output Training Data Preprocessing
- 19. Facial Recognition 19 Neural Network Trained Network Test Image Face Detected? Transformed Facial Images Preprocessing Facial Images Non-Facial Images
- 20. A computer-implemented method for transcribing speech comprising: receiving an input audio from a user; normalizing the input audio to make a total power of the input audio consistent with a set of training samples used to train a trained neural network model; generating a jitter set of audio files from the normalized input audio by translating the normalized input audio by one or more time values; for each audio file from the jitter set of audio files, which includes the normalized input audio: generating a set of spectrogram frames for each audio file; inputting the audio file along with a context of spectrogram frames into a trained neural network; obtaining predicted character probabilities outputs from the trained neural network; and decoding a transcription of the input audio using the predicted character probabilities outputs from the trained neural network constrained by a language model that interprets a string of characters from the predicted character probabilities outputs as a word or words. Ex Parte Hannun (PTAB, Dec. 2019) - US10540957
- 21. Enumerated Groupings METHODS OF ORGANIZING HUMAN ACTIVITY Fundamental economic principles or practices (including hedging, insurance, mitigating risk); Commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); Managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) MATH CONCEPTS relationships, formulas, equations, calculations MENTAL PROCESSES observation, evaluation, judgment, opinion
- 22. “These are not steps that can practically be performed mentally.” “The claims do recite using predicted character probabilities to decide a transcription of the input audio, which the Examiner, relying on the Specification, determines is using a mathematical formula. Namely, the Examiner identifies that the Specification discloses an algorithm to obtain the predicted character probabilities. The mathematical algorithm or formula, however, is not recited in the claims. As such, under the recent Memorandum, the claims do not recite a mathematical concept.” Moreover, even if the claims were considered to recite a mathematical concept, under prong two of step 2A the claims are not directed to an abstract idea because the alleged judicial exception is integrated into a practical application. How to show integration? Specification support: For example, the Specification describes that using DeepSpeech learning, i.e. a trained neural network, along with a language model “achieves higher performance than traditional methods on hard speech recognition tasks while also being much simpler.” Spec. ¶ 29. US 1,054,0957 – Good Specification Saves the Day!
- 23. Best Practices 1. Include description of the technical substance underlying the AI technology. Simply relying on black box description of “artificial intelligence” or “machine learning” will likely not be sufficient. 2. Avoid using “modules” or “unit” or generic terms. 3. Include detailed step-by-step algorithms and concrete examples of how the AI/machine learning can be applied. 4. Discuss Improvements in the Specification. - Performance improvements - “A commonly employed technique in computer vision during network evaluation is to randomly jitter inputs by translations or reflections, feed each jittered version through the network, and vote or average the results. This is not common in speech recognition, however; it was found that translating the raw audio files by 5 milliseconds (ms) (which represented half the filter bank step size used in embodiments herein) to the left and right, forward propagating the recomputed features, and averaging the results beneficial.” 23
- 24. Best Practices Overlapping Best Practices Between the U.S. and Europe 1. Much of the above advice for U.S. patent applications also applies in Europe. 2. Identifying technical problems in the specification coupled with the specific, technical solutions—and claiming those solutions—remain viable approaches for AI inventions in both the U.S. and Europe. 3. Describing improvements to how a computer performs machine learning or executes AI (e.g., by running faster, using less memory, etc.) helps both in the U.S. and Europe. 4. Reciting specific use cases may be specifically helpful in Europe 24
- 25. Thank you!