Journey Towards Scaling Your Application to Million Users

Editor's Notes

• #5: We need some basics to lay the foundations we’ll need to build our knowledge of AWS on top of.
• #8: Latency regulation
• #9: Invest time to save time
• #12: Simplicity.  Durability.  Scalability.  Security.  Broad integration with other AWS services Cloud Data Migration options. Tiered Storage Classes
• #16: Cloudfront allows you to cache static content at the CF edge for faster delivery from a local pop to the end user; in other words, your static content gets cached locally to a user and then delivered locally reducing download times for the website overall there are over 60 CF cache pops around the world as we mentioned earlier. CloudFront helps lower load on your origin infrastructure You can front end static content as discussed and dynamic content as well For dynamic content, CF proxies and accelerates your connection back to your dynamic origin and you would set a 0 ttl on your dynamic content so CloudFront always goes back to origin to fetch this content.
• #19: If you look at the far right end of the spectrum, you can see the G2, wich provide a cost-effective, high-performance platform for graphics applications. The P2, designed for general purpose GPU computing and the F1, our FPGA instance designed to accelerate computationally intensive algorithms. One instance in particular has become very popular amoung our customers: the P2 instance
• #20: This here is the most basic set up you would need to serve up a web application. Any user would first hit Route53 for DNS resolution. Behind the DNS service is an EC2 instance running our webapp and database on a single server, We will need to attach an Elastic IP so Route53 can direct traffic to our webstack at that IP Address with an A record. To scale this infrastructure, the only real option we have is to get a bigger EC2 instance…
• #21: Same with Docker ( application container ..)
• #24: At AWS there are a lot of different options to running databases. One is to just install pretty much any database you can think of on an EC2 instance, and manage all of it yourself. If you are really comfortable doing DBA like activities, like backups, patching, security, tuning, this could be an option for you. Also, if you need something highly specialized or customized and need to manage the hardware to achieve this, again this might be for you. If not, then we have a few options that we think are a better idea: First is Amazon RDS, or Relational Database Service. With RDS you get a managed database instance of either MySQL, Oracle, Postgres or SQL Server, with features such as automated daily backups, simple scaling, patch management, snapshots and restores, High availability, and read replicas - depending on the engine you go with. We also have Aurora in Preview today. Amazon Aurora is a MySQL-compatible relational database that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. Aurora provides up to five times better performance than MySQL at a price point one tenth that of a commercial relational databases while delivering similar performance and availability. Next up we have DynamoDB, a NoSQL database, built on top of SSDs. DynamoDB is based on the Dynamo whitepaper published by Amazon.com back in 2003. This whitepaper was considered the grandfather of most modern NoSQL databases like Cassandra. DynamoDB is kind of like a cousin of the original paper or an evolution of that whitepaper. One of the key concepts to DynamoDB is what we call “Zero Administration”. With DynamoDB the only knobs to tweak are the reads and writes per second you want the DB to be able to perform at. You set it, and it will give you that capacity with query responses averaging in single digit millisecond. We’ve had customers with loads such as half a million reads and writes per second without DynamoDB even blinking.
• #26: So Why start with SQL databases? Generally speaking SQL based databases are established and well worn technology. There’s a good chance SQL is older than most people in this room. It has however continued to power most of the largest web applications we deal with on a daily basis. There are a lot of existing code, books, tools, communities, and people who know and understand SQL. Some of these newer nosql databases might have a handful of companies using them at scale. People are key here in addition to all of the other points as you may need to hire people to manage your database. You also aren’t going to break SQL databases in your first 10 million users. And yes there is an asterisk here, and we’ll get to that in a second. Lastly, there are a lot of clear patterns for scalability that we’ll discuss a bit through out this talk. So as for my point here at the bottom, I again strongly recommend SQL based technology, unless your application is doing something SUPER weird with the data, or you’ll have MASSIVE amounts of it, even then, SQL will be in your stack.
• #27: So why else might you need NoSQL? There are definitely usecases where it makes sense to go NoSQL right off the bat. Some examples: Super low latency applications. Metadata driven datasets High-unrelational data Kind of going along with the previous is where you really need schema-less data constructs. And lets highlight the word NEED here. This isn’t just developers saying its easy to make apps without schemas. That’s just laziness Massive amounts of data, again from the previous slide, in the several TB range. Rapid ingest of data. Where you need to ingest potentially thousands of records per second into a single dataset
• #28: So for this scenario today and based upon our discussion, we’re going to go with RDS and MYSQL as our database engine.
• #30: Whats the biggest problem with this one?
• #33: provide additional visibility into the health of the target instances and containers perform and report on health checks on a per-port basis
• #34: Next up we need to address the lack of failover and redundancy in our infrastructure. We’re going to do this by adding in another webapp instance, and enabling the Multi-AZ feature of RDS, which will give us a standby instance in a different AZ from the Primary. We’re also going to replace our EIP with an Elastic Load Balancer to share the load between our two web instances Now we have an app that is a bit more scalable and has some fault tolerance built in as well.
• #36: We could use Elasticache as a place to store common database query information for content that doesn’t change often, like information on our user, or what is in their cart. We should try and do this as often as possible; so what is Elasticache? Elasticache is hosted Memcache or Redis It does speak the same API as the traditional open source products so think of this as Memcache or Redis as a Service where we manage the clusters for you You can scale from one to many nodes This provides very fast single digit ms latencies as well Managed Simplifies and offloads the management, monitoring, and operation of in-memory cache environments. Compatible Most client libraries will work with the respective engines they were built for - no additional changes or tweaking required. Monitored Detailed monitoring statistics for the engine nodes at no extra cost via Amazon CloudWatch No persistence or replication with Memcache With Redis, you can put a replica in a different AZ with persistence
• #37: We can also move things like session information to Elasticache. We can also use Elasticache to store some of our common database query results which will prevent us from hitting the database too much. This should take load off of our DB tier. Removing session state from our web / app tier is also very key as it allows us to scale up and down without losing session information when this horizontal scaling happens. This is called making our tier “stateless”
• #38: At the stage, you typically start to have a significant amount of traffic and you can already detect patterns (usage patterns) This is amazon.com usage patterns.. East and weast coast ..
• #43: CAP theorem
• #45: Write and updates Counters!!!! Not on the DB – redis!!
• #46: Database Federation is where we break up the database by function. In our example, we have broken out the Forums DB from the User DB from the Products DB Of course, cross functional queries are harder to do and you may need to do your joins at the application layer for these types of queries This will reduce our database footprint for a while and the great thing is, this does prevent you from having to shard until much further down the line. This isn’t going to help for single large tables; for this we will need to shard.
• #47: Sharding is where we break up that single large database into multiple DBs. We might need to do this because of database or table size or potentially for high write IOPs as well. Here is an example of us breaking up a database with a large table into 3 databases. Above we show where each userID is located, but the easiest way to describe how this would work would be to use the example of all users with A-H go into one DB, and I – M go in another, and N – Z go into the third DB. Typically this is done by key space and your application has to be aware of where to read from, update and write to for a particular record. ORM support can help here. This does create operation complexity so if you can federate first, do that. This can be done with SQL or NoSQL, and DynamoDB does this for you under the covers on the backend as your data size increases and the reads / writes per second scale.
• #61: When we start getting into the 5M user plus range, we may start seeing database contention issues on writes to the Master We are going to drill into a couple of techniques to solve these types of issues, and those include Federation and Sharding