CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimization Techniques

Editor's Notes

• #64: Hive – thin metadata layer on top of S3Used for ad-hoc analytics (Ursula for merge ETL)HiveQL gets compiled into set of MR jobs (1 -> many)Is a CLI – runs on the gateways, not like a relational DB server, or a service that the query gets shipped toPig – used for ETL (can create DAGs, workflows for Hadoop processes)Pig scripts also get compiled into MR jobsJava – straight up Hadoop, not for the faint of heart. Some recommendation algorithms are in Hadoop.Python/Java – UDFsApplications such as Sting use the tools on some gateway to access all the various componentsNext – focus on two key components: Data & Clusters
• #87: We have to be wrong a lot in order to right a lotCloud really helps you to reduce the cost of failure.
• #89: Since we’ve invested in facilities around the world, we can offer you global reach at a moment’s notice. It’s cost prohibitive to put your own data center where all your customers are, but with AWS, you get the benefit without having to make the huge investment.
• #96: Only happens in the cloud
• #100: Our strategy of pricing each service independently gives you tremendous flexibility to choose the services you need for each project and to pay only for what you use
• #101: Personal Optimization Assistant
• #102: Netflix now serves 2x the customer traffic with the same amount of AWS resources as deployed 10 months ago
• #105: Reduced TCO remains one of the core reasons why customers choose the AWS cloud. However, there are a number of other benefits when you choose AWS, such as reduced time to market and increased business agility, which cannot be overlooked.
• #111: No Enterprise has only Steady State Workloads.In fact, no system is entirely steady state.
• #112: You should use Consolidated Billing for any of the following scenarios:You have multiple accounts today and want to get a single bill and track each account's charges (e.g., you might have multiple projects, each with its own AWS account).You have multiple cost centers to track.You've acquired a project or company that has its own existing AWS account and you want to consolidate it on the same bill with your other AWS accounts.
• #113: You should use Consolidated Billing for any of the following scenarios:You have multiple accounts today and want to get a single bill and track each account's charges (e.g., you might have multiple projects, each with its own AWS account).You have multiple cost centers to track.You've acquired a project or company that has its own existing AWS account and you want to consolidate it on the same bill with your other AWS accounts.
• #114: You should use Consolidated Billing for any of the following scenarios:You have multiple accounts today and want to get a single bill and track each account's charges (e.g., you might have multiple projects, each with its own AWS account).You have multiple cost centers to track.You've acquired a project or company that has its own existing AWS account and you want to consolidate it on the same bill with your other AWS accounts.
• #117: Cloud is highly cost-effective because you can turn off and stop paying for it when you don’t need it or your users are not accessing. Build websites that sleep at night
• #118: In addition, Only Use What You Need to Use.