AIM410R1 Deep learning applications with TensorFlow, featuring Fannie Mae (December 2019)
Download as PPTX, PDF3 likes636 views
Fannie Mae leverages Amazon SageMaker for machine learning applications to more accurately value properties and reduce mortgage risk. Amazon SageMaker provides a fully managed service that enables Fannie Mae to focus on modeling while ensuring data security, self-service access, and end-to-end governance through techniques like private subnets, encryption, IAM policies, and operating zones. The presentation demonstrates how to get started with TensorFlow on Amazon SageMaker.
![Interface endpoint enforcement: Example
{
"Effect": "Allow",
"Action": "sagemaker:CreatePresignedNotebookInstanceUrl",
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:SourceVpce": "vpce-x"
}
}
}
{
"Effect": ”Allow",
"Action": "sagemaker:CreatePresignedNotebookInstanceUrl",
"Resource": "*"
"Principal": "*",
"Condition": {
"IpAddress": {
"aws:VpcSourceIp": [
"x.x.x.x/a",
"y.y.y.y/b"
]
}
}
}
Interface endpoint policy
Identity policy](https://image.slidesharecdn.com/aim410r1sagemakerfanniemaefinal-191208160905/85/AIM410R1-Deep-learning-applications-with-TensorFlow-featuring-Fannie-Mae-December-2019-22-320.jpg)
![Access controls enforcement: Examples
{
"Effect": "Allow",
"Action": "sagemaker:CreateTrainingJob" ,
"Resource": "arn:aws:sagemaker:x:x:*/app1-*",
"Condition": {
"StringEquals": {
"aws:RequestTag/CostCenter": "x",
"sagemaker:VolumeKmsKey": "arn:aws:kms:x:x:key/x"
},
"Bool": {
"sagemaker:InterContainerTrafficEncryption": "true",
"sagemaker:NetworkIsolation": "true"
},
"ForAllValues:StringEquals": {
"sagemaker:VpcSubnets": [
"subnet-a",
"subnet-b"
],
"sagemaker:VpcSecurityGroupIds": [
"sg-x",
"sg-y"
]
},
"Null": {
"sagemaker:VpcSubnets": "false"
}
}
}
{
"Effect": "Deny",
"Action": "sagemaker:CreatePresignedNotebookInstanceUrl",
"Resource": "*",
"Condition": {
"ForAllValues:StringNotEquals": {
"sagemaker:ResourceTag/creatorUserId": "${aws:userId}"
}
}
}
Network and encryption enforcement
Notebook access control](https://image.slidesharecdn.com/aim410r1sagemakerfanniemaefinal-191208160905/85/AIM410R1-Deep-learning-applications-with-TensorFlow-featuring-Fannie-Mae-December-2019-25-320.jpg)
AIM410R1 Deep learning applications with TensorFlow, featuring Fannie Mae (December 2019)
- 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deep learning applications using TensorFlow, featuring Fannie Mae A I M 4 1 0 - R 1 Julien Simon Global Evangelist AI/ML Amazon Web Services Bin Lu Senior Director of Risk Modeling & Analytics Fannie Mae Vindhan Sahayam Lead Architect Fannie Mae
- 3. Agenda TensorFlow on AWS Customer case study: Fannie Mae Demo: TensorFlow on Amazon SageMaker Getting started
- 4. TensorFlow https://www.tensorflow.org • Main API in Python, with support for Javascript, Java, C++ • TensorFlow 1.x: symbolic execution • ‘Define then run’: build a graph, optimize it, feed data, and compute • Low-level API: variables, placeholders, tensor operations • High-level API: tf.estimator.* • Keras library: Sequential and Functional API, predefined layers • TensorFlow 2.0: imperative execution (aka eager execution) • ‘Define by run’: normal Python code, similar to numpy • Run it, inspect it, debug it • Keras is the preferred API
- 5. AWS: The platform of choice for TensorFlow https://aws.amazon.com/tensorflow/ 85% of all TensorFlow workloads in the cloud run on AWS 89% of all deep learning workloads in the cloud run on AWS
- 6. TensorFlow: a first-class citizen on Amazon SageMaker • Built-in TensorFlow containers for training and prediction • Code available on Github: https://github.com/aws/sagemaker-tensorflow-containers • Build it, run it on your own machine, customize it, etc. • Versions : 1.4.1 1.15 (2.0 coming soon) • Not just TensorFlow • Standard tools: TensorBoard, TensorFlow Serving • SageMaker features: Local Mode, Script Mode, Model Tuning, Spot Training, Pipe Mode, Amazon EFS & Amazon FSx for Lustre, Amazon Elastic Inference, etc. • Performance optimizations: GPUs and CPUs (AWS, Intel MKL-DNN library) • Distributed training: Parameter Server and Horovod
- 7. Amazon SageMaker re:Invent 2019 announcements SageMaker Studio SageMaker Notebooks (preview) SageMaker Debugger SageMaker Experiments SageMaker Model Monitor SageMaker Autopilot
- 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Bin Lu Senior Director of Risk Modeling and Analytics Fannie Mae Vindhan Sahayam Lead Architect Fannie Mae
- 9. Fannie Mae is a leading source of financing for mortgage lenders • Provide access to affordable mortgage financing in all markets at all times • Effectively manage and reduce risk to our business, taxpayers, and the housing finance system
- 10. Accurate property valuation reduces mortgage risk It is used in all stages of the loan lifecycle: • Origination and underwriting, where a lender determines whether a borrower's loan application is an acceptable risk • Post-purchase quality control • Portfolio risk management, financial reporting, and regulatory reporting • Loss mitigation Fannie Mae credit portfolio is ~$3 trillion Mortgage lifecycle Origination Servicing Securitization Foreclosure
- 11. Machine learning example: Property valuation Property appraisal by certified/licensed appraiser • Quantitative valuation based on comparable property sale prices and market trends • Adjustments for unobservable inputs Fannie Mae is leveraging machine learning • Automated home price valuation model based on observables (XGBoost, KNN) • Automated review of the adjustment based on visual inspection (TensorFlow – CNN) Fannie Mae receives ~40,000 appraisal reports, with 500,000+ property images every day
- 12. Technology challenges in machine learning Limited CPU/GPU resources to train and run models No streamlined approach for model development Process of packaging and hosting models is complex and time consuming Difficult to connect machine learning and analytics tools to data
- 13. Amazon SageMaker fits our needs • Flexible and self- service machine learning platform • Easy access to compute resources and data • Streamlined model training and deployment • Built-in governance procedure and audit trail
- 14. Automated property image classification Three multi-layer convolutional neural network models with transferred learning 1st layer fixes image orientation 2nd layer identifies room type 3rd layer predicts marketability score
- 15. Benefits of Amazon SageMaker Effective cost management • Never pay for idle; the cost is based on actual vCPU/GPU usage, not the maximum processing capacity of the infrastructure • Designed to enable performance improvement at zero cost Rapid time to market • Instant access to dedicated computing resources • Ability to focus on business needs; no server to manage and no complex code to write for distributed model training, hyperparameter tuning, or model deployment AWS breadth and depth • Streamlined integration with big data analytics platform • Automated version controls, governance, audit trails, and secured workload • Business resiliency
- 16. Consideration for provisioning Amazon SageMaker Implementation of governance is as important as developing business capabilities • InfoSec risk management • Data governance • Model governance • Technology risk management Establish guiding principles at the start • Technology and software • Models and analytics Consider data gravity • Co-locate machine learning platform with data sources We engaged with the Amazon SageMaker team early
- 17. A special shout-out to the Fannie Mae Digital Incubator team for developing the property image classification machine learning model: Hamid Reza Khakpour, Timur Fatykhov, and Felix Meale
- 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 19. Three very important goals Realistic to achieve all the above with a fully-managed service such as Amazon SageMaker? + Non-negotiable data security + Self-service access + End-to-end governance with traceability
- 20. Given these conditions … • Amazon SageMaker infrastructure is deployed in AWS-managed, multi- tenant VPCs and subnets • Data scientists work with highly sensitive data using powerful dev tools How do we keep data absolutely secure?
- 21. Private subnet Keeping data secure: Harden network security Fannie Mae VPC Amazon VPC AWS Cloud Amazon SageMaker notebook Amazon S3 Security group ENI AWS KMS AWS CodeCommit Amazon ECR Amazon SageMaker Amazon SageMaker training Amazon SageMaker hosting ENI ENI S3 gateway endpoint Endpoint subnet Interface VPC endpoints Security group + How do we prevent data exfiltration? + How do we avoid exposure to internet?
- 22. Interface endpoint enforcement: Example { "Effect": "Allow", "Action": "sagemaker:CreatePresignedNotebookInstanceUrl", "Resource": "*", "Condition": { "StringEquals": { "aws:SourceVpce": "vpce-x" } } } { "Effect": ”Allow", "Action": "sagemaker:CreatePresignedNotebookInstanceUrl", "Resource": "*" "Principal": "*", "Condition": { "IpAddress": { "aws:VpcSourceIp": [ "x.x.x.x/a", "y.y.y.y/b" ] } } } Interface endpoint policy Identity policy
- 23. Keeping data secure: Encrypt everywhere Use customer managed CMK for volumes and S3 encryption Enable Amazon S3 default encryption. Additionally use deny policies to prevent unencrypted uploads
- 24. With the greater flexibility of self-service access … • How do we ensure users comply with security controls? • How do we ensure users do not step into each other?
- 25. Access controls enforcement: Examples { "Effect": "Allow", "Action": "sagemaker:CreateTrainingJob" , "Resource": "arn:aws:sagemaker:x:x:*/app1-*", "Condition": { "StringEquals": { "aws:RequestTag/CostCenter": "x", "sagemaker:VolumeKmsKey": "arn:aws:kms:x:x:key/x" }, "Bool": { "sagemaker:InterContainerTrafficEncryption": "true", "sagemaker:NetworkIsolation": "true" }, "ForAllValues:StringEquals": { "sagemaker:VpcSubnets": [ "subnet-a", "subnet-b" ], "sagemaker:VpcSecurityGroupIds": [ "sg-x", "sg-y" ] }, "Null": { "sagemaker:VpcSubnets": "false" } } } { "Effect": "Deny", "Action": "sagemaker:CreatePresignedNotebookInstanceUrl", "Resource": "*", "Condition": { "ForAllValues:StringNotEquals": { "sagemaker:ResourceTag/creatorUserId": "${aws:userId}" } } } Network and encryption enforcement Notebook access control
- 26. Enabling governance: Operating zones Data scientists Train & test models NPI Amazon SageMaker notebooks Non-NPI Research zone Controlled code/model migration Production Retrain Deploy Automated process Approve models Lambda API Gateway Application CI/CD DevUATProd Test Deploy API Gateway Application developers AWS CodeCommit Lambda Application zone Development Create guardrails early: Establish zones to manage ML lifecycle
- 27. Machine learning orchestration with auditing: Example Source code AWS CodeCommit Amazon S3 Dataset Model artifacts AWS CodeBuild Lambda LambdaApproval gate Batch transformAWS Lambda AWS CodePipeline Deploy endpoint AWS Cloud Train + Reproducible and reusable pipeline + Built-in audit tracking capability + Other options: AWS Step Functions, Apache Airflow Amazon DynamoDB Model metadata for audit tracking Amazon ECR Create model
- 28. Fannie Mae’s Enterprise Data Lake (EDL) at a glance Build machine learning capability with a fully functional data lake as a foundation ... and growing 100+ applications3,000+ datasets 500+ AWS Glue Data Catalog databases 1,000+ users
- 29. Amazon SageMaker in EDL: Reference architecture Amazon QuickSight Amazon ES Amazon Athena Amazon S3 Encrypted objects Amazon EMR Amazon SageMaker AWS Glue metastore Amazon ECS AWS Batch AWS Lambda Corporate data center Data warehouses RDBMS File systems Third-party data ADFS AWS Direct Connect Rest/CLI Zscaler Ingestion AWS Step Functions Amazon Redshift Workflow ETL/Analytics/ML Data visualization Enterprise data lake platform Amazon Kinesis Auditing Security and governance Logging and monitoring Platform built with 100% native AWS services => less integration challenges
- 30. Key takeaways + New IAM context keys are valuable + Restrict access to buckets, utilize S3 endpoint policy + Amazon SageMaker has full support for PrivateLink endpoints; Enabling and enforcing those is crucial + Data is a first-class primitive in ML workflows; keep track of data collection and preparation + Make predictions traceable to original training record + Introduce segregation of duties; establish operating zones + Leverage data lake pattern Build a highly-secure, self-service & end-to-end traceable ML capability with Amazon SageMaker
- 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Bin Lu Senior Director of Risk Modeling and Analytics Fannie Mae Vindhan Sahayam Lead Architect Fannie Mae
- 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. + Script Mode + Managed Spot Training + Elastic Inference
- 33. Build, train, deploy machine learning models quickly at scale SageMaker Studio IDE Amazon SageMaker Ground Truth Algorithms & Frameworks Quick-start notebooks Experiments Training & Tuning Deployment & Hosting Reinforcement Learning ML Marketplace Debugger Autopilot Monitoring NEW! NEW! NEW! NEW! NEW! NEW! Neo Amazon SageMaker
- 35. Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Julien Simon Global Evangelist AI/ML Amazon Web Services Bin Lu Senior Director of Risk Modeling & Analytics Fannie Mae Vindhan Sahayam Lead Architect Fannie Mae
- 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.