Deep dive de Amazon DynamoDB

Notas do Editor

• #4: consistent, single-digit millisecond latency at any scale Fast, Consistent Performance [automatic partitioning, SSD technology]Highly Scalable [store as much data, limits are there for safety] Fully Managed [choose key schema, tablename, provisioned capacity] Event Driven Programming [Streams, Lambda/KCL] Fine-grained Access Control [item-level, IAM credentials] Flexible [doc, kv]
• #9: Hash Key only
• #11: Hash Key & Sort Key
• #13: Online Indexing
• #14: Think of this as a parallel table asynchronously populated by DynamoDB Eventually consistent 1 Table update = 0, 1 or 2 GSI updates
• #17: There is a limit in place to avoid run-away apps. But you can request a limit increase. http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html For every distinct hash key value, the total sizes of all table and index items cannot exceed 10 GB This request would have caused the ReadCapacityUnits limit to be exceeded for the account in us-west-2. Current ReadCapacityUnits reserved by the account: 237. Limit: 2000. Requested: 5000. Refer to the Amazon DynamoDB Developer Guide for current limits and how to request higher limits. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: LimitExceededException; Request ID: 0U3G4FLA19BQMELKS4G7RSTH83VV4KQNSO5AEMVJF66Q9ASUAAJG)
• #18: Picture from: http://www.amazon.com/Black-Aluminum-Control-Amplifier-Wheel/dp/B005HU1ZHA Scan and Query Cumulative size of processed items – ceiling (Ʃ(item sizes)/4KB) Batch GetItem ceiling [(Ʃ(item1 size)/4KB) + …ceiling (Ʃ(itemN size)/4KB)] Consumed throughput is measured per operation Provisioned throughput is divided between all partitions
• #20: RCUs and WCUs are uniformly spread across partitions on table creation
• #24: Uneven access across key space
• #27: Key/values access patters (Map, Dictionary…)
• #31: Touch on elasticity
• #33: Add a single table chart, describe bottlenecks
• #34: Warn: You still consume IO even if the item doesn’t exist
• #37: Often some items in your table are accessed more frequently than others. For example, this graph illustrates how many requests per second were made for each item in your table. For collections like a product catalog, some items are substantially more popular than others. The same probably goes for tweets sent from a celebrity, and that sort of thing. Displaying those “hot items” on everyone’s twitter feed is problematic since they cause an uneven request distribution. (ask audience) Can anyone think of some things that we can do to “cool off” those hot items?
• #40: One thing we can do is cache those reads in the application. Since a tweet doesn’t change once you post it, you can cache it in memory or in something like Amazon ElastiCache. This is a technique used in high throughput applications with a traditional database as well.
• #43: 50 messages to read from Large items not necessarily expensive on their own, but the cost adds up Query comes from same partition
• #44: Put the metadata into a GSI. 128 bytes versus 256KB
• #45: Last section was 5:25
• #47: This is your base table
• #50: filter will drop items, but you still pay for reads
• #53: BEGINS_WITH
• #55: Previous section is 5:05
• #56: Touch on elasticity
• #66: { "Records":[ { "EventName":"INSERT", "EventVersion":"1.0", "EventSource":"aws:dynamodb", "Dynamodb":{ "NewImage":{ "Message":{ "S":"New item!" }, "Id":{ "N":"101" } }, "SizeBytes":26, "StreamViewType":"NEW_AND_OLD_IMAGES", "SequenceNumber":"111", "Keys":{ "Id":{ "N":"101" } } }, "EventID":"1", "eventSourceARN":"arn:aws:dynamodb:us-east-1:acct-id:table/ExampleTableWithStream/stream/stream-id/", "AwsRegion":"us-east-1" }, { "EventName":"MODIFY", "EventVersion":"1.0", "EventSource":"aws:dynamodb", "Dynamodb":{ "NewImage":{ "Message":{ "S":"This item has changed" }, "Id":{ "N":"101" } }, "SizeBytes":59, "StreamViewType":"NEW_AND_OLD_IMAGES", "SequenceNumber":"222", "OldImage":{ "Message":{ "S":"New item!" }, "Id":{ "N":"101" } }, "Keys":{ "Id":{ "N":"101" } } }, "EventID":"2", "eventSourceARN":"arn:aws:dynamodb:us-east-1:acct-id:table/ExampleTableWithStream/stream/stream-id/", "AwsRegion":"us-east-1" }, { "EventName":"REMOVE", "EventVersion":"1.0", "EventSource":"aws:dynamodb", "Dynamodb":{ "SizeBytes":38, "StreamViewType":"NEW_AND_OLD_IMAGES", "SequenceNumber":"333", "OldImage":{ "Message":{ "S":"This item has changed" }, "Id":{ "N":"101" } }, "Keys":{ "Id":{ "N":"101" } } }, "EventID":"3", "eventSourceARN":"arn:aws:dynamodb:eu-west-1:acct-id:table/ExampleTableWithStream/stream/stream-id/", "AwsRegion":"eu-west-1" } ] }
• #72: Kinesis Firehose: Elasticsearch, S3, Redshift FIXME
• #74: O Titans Group é lider em serviços de telefonia e provedores de internet, com operação em 40 operadores em 17 países… um principais produtos chamado “X” é um aplicativo de sincronização de arquivos para mobile e desktop
• #75: Titans Group precisava garantir que a solução de storage do produto atendesse os requisitos mais exigentes do ponto de vista de durabilidade, disponibilidade e confidencialidade das informações dos usuários. Também era preciso garantir a escalabilidade tanto do ponto de vista técnico quanto do ponto de vista de custo.
• #76: NoSQL you have to ask – how will the application access the data? And store your data in such a way to retrieve the data with just a Select, and No joins. Designed by keeping in mind Access patterns. Via duplication of data (storage) and using Hierarchical structures, you can now optimize for compute, and therefore it is very fast. == Non-relational (NoSQL) databases typically do not enforce a schema. A “partition key” is generally used to retrieve values, column sets, or semi-structured JSON, XML, or other documents containing related item attributes.