Testing for the deeplearning folks
- 1. Testing for the Deeplearning Folks By : Vishwas Narayan
- 2. Who Am I? β I am a Compute Nerd who loves more of the hardware β Worked on different datasets in the Computer Vision β I just some good practises /Habits: β Read a lot of paper β Just loves to understand the Neural Network(Even though I haven't learnt anything)
- 6. AI is an attempt to build humans intuitive intel Machine do have different perception
- 8. Agenda? Neural Network Testing in the Deep Learning
- 9. Neural Nets (A brief biological history)
- 16. Ofcourse I feel that you all know/if you don't know we will summarise it. β Neural network β Activation function β What are hyperparameters and also many more concepts in the Neural Networks
- 17. Nature of Dataset is Simple Basically cross validation
- 18. Model Evaluation
- 19. Bias in ML System β Any ML model is Biased toward the model that it is trained on β Often Data is biased β It is important to know what you model is trained on and how they affect the model.
- 20. Testing for the neural network can be β A unit test β A test for the presence of the adversarial examples β A test for the models performance β A test for the Fakes and also many more
- 22. What is An Adversarial Attack? β A one pixel attack and a many pixel attack β A picture with noisy information β A picture with the noisy decoder β A very faulty βCODECβ - βCompressor and Decompressorβ or a βCompressor and Decoderβ
- 23. What is An Adversarial Attack? A one pixel attack and a many pixel attack A picture with noisy information A picture with the noisy decoder A very faulty βCODECβ - βCompressor and Decompressorβ or a βCompressor and Decoderβ Basically one designed by a adversary or an attacker to confuse the machine and its sometimes/most of the time developed by the developer itself So why do they do it
- 24. Can a human get confused?
- 25. Thus there is a a lot of new things to think about? If we try to attack by the optimization then we have to know the parameters and also the Architecture. Thus Companies have to be very strong to get the right architecture in the right time.
- 26. But Skewness in the model parameters are the key How think about your brain.
- 27. Yes they can be transferred to humans Just identify the context for inducing the Adversarial examples. But also a time limited human can make an error.
- 28. Yes they can be transferred to humans Just identify the context for inducing the Adversarial examples. But also a time limited human can make an error. Hypothesis is that:they can be transfer to models and can target features that are relevant to human visual system or inference but we are smart
- 29. Yes they can be transferred to humans Just identify the context for inducing the Adversarial examples. But also a time limited human can make an error. Hypothesis is that:they can be transfer to models and can target features that are relevant to human visual system or inference but we are smart Thus we can never be fooled.
- 30. More of these kinds of the problem statements are: Context where we can really detect the hypothesis example.
- 31. Yes they can be transfer to a model If you have an access to the optimization process, parameters in the neural network that you are building lossfunction and other parameters. You can be made to have no similarity in the access to the brain function.
- 32. Black box attack on CV models β Can be transferred through: β Different architecture β When trained on different data β When Trained with different parameters for the neural network β Loss function,hyper parameters etc β Data Bias and Model Bias
- 33. But how can we prevent it We move with fetching clues,petreubition seen more dominantly.
- 34. Thus we also aim at Reducing the gap between the model and the Brain.
- 35. But models have evolved over a period of time They can generate an image they can fake the presence of the pixel Thus scientist have also leverage this potential for both good and some have used for the βbadβ Model quantization can also induce the noise thus you have to be very careful..
- 36. So i talked a lot on images what about the Other formats fo the sequential datasets JSON,csv,XML,YAML, And you models have different file formats: CKPT,h5,PMML and so on and sometimes even in json
- 37. Outline of the project β Concept β Project structure β ML Test Score:A test for the production readiness β Infrastructure/tooling β CI/Testing β Docker β Deployment through web/Device β Monitoring
- 38. So we all know that deep learning system β Makes the prediction β Brings the accuracy β Then we work on the deployment
- 39. Deep Learning system that we want to consider
- 41. Trust me let's just move on to the testing β Unit Testing: test of single unit of code in isolation(βquarantineβ) β Integrated Testing: TEst how all the different components work together β Black Box testing: Functionality β White box/Glass Box testing : Process β Regression Test: We create a problem and test it β Smoke Test: A subset of the test case verifying the core of the system. β Alpha test : In house final function testing β Beta Test: initial release of subset to real users.
- 42. Well Tested code helps you to β Find Bugs early β Iterate faster β Debug more Easily β Design better code
- 43. Learning to write the good test code is a Really good investment
- 44. Learning to write the good test code is a Really good investment As the way in which software engineering works is a whole level of the different story.
- 45. Where to start? β Pick a single,specific functionality to verify β Use available tool to get everything else out of the way β Don't try writing all the codebase as one. β Writing test as early as they can be valuable.
- 46. So let's get digging for the handson β YOLO model testing. β Deployments being tested. β Methods on the deployment.
Editor's Notes
- #22: https://www.youtube.com/watch?v=UgsmV2cCO44