Amazon SimpleDB
1 like3,639 views
The document discusses the relational model of data for large shared data banks. It introduces the concept of n-ary relations and the universal data sublanguage as an alternative to tree-structured files or network models. The relational model provides independence between data representation and programs/queries by removing ordering, indexing, and access path dependencies from the user's data model. This allows changes to the internal data representation without affecting user activities.
![Relational Model
Information Retrieval P. BAXENDALE, Editor
A Relational Model of Data for The relational view (or model) of data described in
Section 1 appears to be superior in several respects to the
Large Shared Data Banks graph or network model [3,4] presently in vogue for non-
inferential systems. It provides a means of describing data
with its natural structure only-that is, without superim-
E. F. CODD posing any additional structure for machine representation
IBM Research Laboratory, San Jose, California purposes. Accordingly, it provides a basis for a high level
data language which will yield maximal independence be-
tween programs on the one hand and machine representa-
Future users of large data banks must be protected from tion and organization of data on the other.
having to know how the data is organized in the machine (the A further advantage of the relational view is that it
internal representation). A prompting service which supplies forms a sound basis for treating derivability, redundancy,
such information is not a satisfactory solution. Activities of users and consistency of relations-these are discussedin Section
at terminals and most application programs should remain 2. The network model, on the other hand, has spawned a
unaffected when the internal representation of data is changed number of confusions, not the least of which is mistaking
and even when some aspects of the external representation the derivation of connections for the derivation of rela-
are changed. Changes in data representation will often be tions (seeremarks in Section 2 on the “connection trap”).
needed as a result of changes in query, update, and report Finally, the relational view permits a clearer evaluation
traffic and natural growth in the types of stored information. of the scope and logical limitations of present formatted
Existing noninferential, formatted data systems provide users data systems, and also the relative merits (from a logical
with tree-structured files or slightly more general network standpoint) of competing representations of data within a
models of the data. In Section 1, inadequacies of these models single system. Examples of this clearer perspective are
are discussed. A model based on n-ary relations, a normal cited in various parts of this paper. Implementations of
form for data base relations, and the concept of a universal systems to support the relational model are not discussed.
data sublanguage are introduced. In Section 2, certain opera-
1.2. DATA DEPENDENCIES PRESENTSYSTEMS
IN
tions on relations (other than logical inference) are discussed
The provision of data description tables in recently de-
and applied to the problems of redundancy and consistency
veloped information systems represents a major advance
in the user’s model.
toward the goal of data independence [5,6,7]. Such tables
KEY WORDS AND PHRASES: data bank, data base, data structure, data facilitate changing certain characteristics of the data repre-
organization, hierarchies of data, networks of data, relations, derivability, sentation stored in a data bank. However, the variety of
redundancy, consistency, composition, join, retrieval language, predicate
calculus, security, data integrity
data representation characteristics which can be changed
CR CATEGORIES: 3.70, 3.73, 3.75, 4.20, 4.22, 4.29 without logically impairing some application programs is
still quite limited. Further, the model of data with which
users interact is still cluttered with representational prop-
erties, particularly in regard to the representation of col-
lections of data (as opposed to individual items). Three of
the principal kinds of data dependencies which still need
1. Relational Model and Normal Form to be removed are: ordering dependence, indexing depend-
ence, and accesspath dependence. In some systems these
1.I. INTR~xJ~TI~N dependencies are not clearly separable from one another.
This paper is concerned with the application of ele- 1.2.1. Ordering Dependence. Elements of data in a
mentary relation theory to systems which provide shared data bank may be stored in a variety of ways, someinvolv-
access large banks of formatted data. Except for a paper
to ing no concern for ordering, some permitting each element
by Childs [l], the principal application of relations to data to participate in one ordering only, others permitting each
systems has been to deductive question-answering systems. element to participate in several orderings. Let us consider
Levein and Maron [2] provide numerous referencesto work those existing systems which either require or permit data
in this area. elements to be stored in at least one total ordering which is
In contrast, the problems treated here are those of data closely associated with the hardware-determined ordering
independence-the independence of application programs of addresses.For example, the records of a file concerning
and terminal activities from growth in data types and parts might be stored in ascending order by part serial
changesin data representation-and certain kinds of data number. Such systems normally permit application pro-
inconsistency which are expected to become troublesome grams to assumethat the order of presentation of records
even in nondeductive systems. from such a file is identical to (or is a subordering of) the
Volume 13 / Number 6 / June, 1970 Communications of the ACM 377](https://image.slidesharecdn.com/nosql-120221210246-phpapp01/85/Amazon-SimpleDB-5-320.jpg)
![https://sdb.amazonaws.com/
?Action=PutAttributes
&Attribute.1.Name=Color
&Attribute.1.Value=Blue
&Attribute.2.Name=Size
&Attribute.2.Value=Med
&Attribute.3.Name=Price
&Attribute.3.Value=0014.99
&Attribute.3.Replace=true
&AWSAccessKeyId=[valid access key id]
&DomainName=MyDomain
&ItemName=Item123
&SignatureVersion=2
&SignatureMethod=HmacSHA256
&Timestamp=2010-01-25T15%3A03%3A05-
07%3A00
&Version=2009-04-15
&Signature=[valid signature]](https://image.slidesharecdn.com/nosql-120221210246-phpapp01/85/Amazon-SimpleDB-37-320.jpg)
Amazon SimpleDB
- 1. Amazon SimpleDB Sean Collins
- 3. Tale of Two Cities • Relational • “Non-Relational”
- 4. Tale of Two Cities • Relational
- 5. Relational Model Information Retrieval P. BAXENDALE, Editor A Relational Model of Data for The relational view (or model) of data described in Section 1 appears to be superior in several respects to the Large Shared Data Banks graph or network model [3,4] presently in vogue for non- inferential systems. It provides a means of describing data with its natural structure only-that is, without superim- E. F. CODD posing any additional structure for machine representation IBM Research Laboratory, San Jose, California purposes. Accordingly, it provides a basis for a high level data language which will yield maximal independence be- tween programs on the one hand and machine representa- Future users of large data banks must be protected from tion and organization of data on the other. having to know how the data is organized in the machine (the A further advantage of the relational view is that it internal representation). A prompting service which supplies forms a sound basis for treating derivability, redundancy, such information is not a satisfactory solution. Activities of users and consistency of relations-these are discussedin Section at terminals and most application programs should remain 2. The network model, on the other hand, has spawned a unaffected when the internal representation of data is changed number of confusions, not the least of which is mistaking and even when some aspects of the external representation the derivation of connections for the derivation of rela- are changed. Changes in data representation will often be tions (seeremarks in Section 2 on the “connection trap”). needed as a result of changes in query, update, and report Finally, the relational view permits a clearer evaluation traffic and natural growth in the types of stored information. of the scope and logical limitations of present formatted Existing noninferential, formatted data systems provide users data systems, and also the relative merits (from a logical with tree-structured files or slightly more general network standpoint) of competing representations of data within a models of the data. In Section 1, inadequacies of these models single system. Examples of this clearer perspective are are discussed. A model based on n-ary relations, a normal cited in various parts of this paper. Implementations of form for data base relations, and the concept of a universal systems to support the relational model are not discussed. data sublanguage are introduced. In Section 2, certain opera- 1.2. DATA DEPENDENCIES PRESENTSYSTEMS IN tions on relations (other than logical inference) are discussed The provision of data description tables in recently de- and applied to the problems of redundancy and consistency veloped information systems represents a major advance in the user’s model. toward the goal of data independence [5,6,7]. Such tables KEY WORDS AND PHRASES: data bank, data base, data structure, data facilitate changing certain characteristics of the data repre- organization, hierarchies of data, networks of data, relations, derivability, sentation stored in a data bank. However, the variety of redundancy, consistency, composition, join, retrieval language, predicate calculus, security, data integrity data representation characteristics which can be changed CR CATEGORIES: 3.70, 3.73, 3.75, 4.20, 4.22, 4.29 without logically impairing some application programs is still quite limited. Further, the model of data with which users interact is still cluttered with representational prop- erties, particularly in regard to the representation of col- lections of data (as opposed to individual items). Three of the principal kinds of data dependencies which still need 1. Relational Model and Normal Form to be removed are: ordering dependence, indexing depend- ence, and accesspath dependence. In some systems these 1.I. INTR~xJ~TI~N dependencies are not clearly separable from one another. This paper is concerned with the application of ele- 1.2.1. Ordering Dependence. Elements of data in a mentary relation theory to systems which provide shared data bank may be stored in a variety of ways, someinvolv- access large banks of formatted data. Except for a paper to ing no concern for ordering, some permitting each element by Childs [l], the principal application of relations to data to participate in one ordering only, others permitting each systems has been to deductive question-answering systems. element to participate in several orderings. Let us consider Levein and Maron [2] provide numerous referencesto work those existing systems which either require or permit data in this area. elements to be stored in at least one total ordering which is In contrast, the problems treated here are those of data closely associated with the hardware-determined ordering independence-the independence of application programs of addresses.For example, the records of a file concerning and terminal activities from growth in data types and parts might be stored in ascending order by part serial changesin data representation-and certain kinds of data number. Such systems normally permit application pro- inconsistency which are expected to become troublesome grams to assumethat the order of presentation of records even in nondeductive systems. from such a file is identical to (or is a subordering of) the Volume 13 / Number 6 / June, 1970 Communications of the ACM 377
- 6. • A Relational Model of Data for Large Shared Data Banks • E. F. Codd • IBM Research Laboratory, San Jose, California • CACM June 1970
- 7. • Data as Relations • “In many commercial, governmental, and scientific data banks ... some of the relations are of quite high degree... Accordingly, we propose that users deal, not with relations which are domain-ordered, but with relationships”
- 8. Relationships • Customer To Order • Order to Items • And So Forth
- 9. Relational • Provides SQL interface to developers • ACID • Atomicity • Consistency • Isolation • Durability
- 10. Tale of Two Cities • “Non-Relational”
- 11. CAP Theorem
- 12. CAP Theorem • Consistency • Availability • Partition-tolerance
- 13. Non-Relational • Less structured • “Schema-less” • Key-value storage • Implement parts of ACID
- 15. WHY?
- 16. WHY? • Speed
- 18. WHY? • Speed • Flexibility • Scale
- 20. Speed
- 21. Speed • No JOINS
- 22. Speed • No JOINS • No special column types
- 23. Speed • No JOINS • No special column types • Concurrent operations
- 25. Flexibility
- 26. Flexibility • No table definition • Store whatever you want • Wherever you want • Adjust on the fly
- 28. Scalability
- 29. Scalability • Eventual consistency • Writes propagate across nodes • Propagation time is not constant
- 30. Amazon SimpleDB • Amazon AWS • “Structured Data” Storage • Notable users include Netflix
- 31. SimpleDB Data Model • Domain • Item • Name • Attributes
- 32. SimpleDB Data Model • All data stored as Strings
- 33. SimpleDB Features Eventually Consistent Consistent Read Read Stale Reads Possible No Stale Reads Lowest read latency Potential higher read latency Potential lower read Highest read throughput throughput
- 34. SimpleDB Features • Conditional Transactions • PUT/DELETE • At the Item Level • Based on Item Attributes
- 35. Using SimpleDB • Operations are issued as HTTP GET requests (REST) • Responses are XML • Supports an SQL-like syntax for fetching items from the domain
- 36. Using SimpleDB • Supports an SQL-like syntax for fetching items from the domain • SELECT <specification> FROM <domain> WHERE <condition> • Specifications • * (all attributes) • itemName() • count(*) • Specific attributes
- 37. https://sdb.amazonaws.com/ ?Action=PutAttributes &Attribute.1.Name=Color &Attribute.1.Value=Blue &Attribute.2.Name=Size &Attribute.2.Value=Med &Attribute.3.Name=Price &Attribute.3.Value=0014.99 &Attribute.3.Replace=true &AWSAccessKeyId=[valid access key id] &DomainName=MyDomain &ItemName=Item123 &SignatureVersion=2 &SignatureMethod=HmacSHA256 &Timestamp=2010-01-25T15%3A03%3A05- 07%3A00 &Version=2009-04-15 &Signature=[valid signature]
- 38. <PutAttributesResponse> <ResponseMetadata> <RequestId>490206ce-8292-456c- a00f-61b335eb202b</RequestId> <BoxUsage>0.0000219907</BoxUsage> </ResponseMetadata> </PutAttributesResponse>
- 39. Case Study • ZINC Database • Commercially available compounds • Virtual Screening • Clean “Drug Like” (#13) • Approx. 3,751,744 compounds
- 40. Data Model • Item • Name = ZINC_ID • Attributes • Molecular Weight • Charge • SMILES • “Simplified molecular input line entry specification”
- 41. Boto • Provides a library for accessing Amazon AWS services • Encapsulates SimpleDB data in Python objects • Dictionaries • Iterators • etc..
- 42. for item in domain.select("SELECT * FROM zinc_13"): print item.name print item.keys() print item.values()
- 44. Some Tips • Aggregate your operations • <= 25 rows per request • Shard your data across Domains • Handling Numerical Data • Zero Padding • Negative Numbers Offsets • Dates
- 45. Advantages • Faster development times • (No) Administration • No Hardware! • Scale-as-you-go • Pay-as-you-go
- 46. Pricing • 1GB Free Storage • $0.25/GB/mo Thereafter • $0.10/GB Transfer In • $.15/GB Out • 25 Machine Hours Free/month • $0.14/hr Thereafter
- 47. Limitations • Less Features = More Work for the Developer • Dates • Numerical Data • Data Consistency
- 48. Limits Limitations Following is a table that describes current limits within Amazon SimpleDB. Parameter Restriction Domain size 10 GB per domain Domain size 1 billion attributes per domain Domain name 3-255 characters (a-z, A-Z, 0-9, '_', '-', and '.') Domains per account 100 Attribute name-value pairs per item 256 Attribute name length 1024 bytes Attribute value length 1024 bytes Item name length 1024 bytes Attribute name, attribute value, and item All UTF-8 characters that are valid in XML documents. name allowed characters Control characters and any sequences that are not valid in XML are returned Base64-encoded. For more information, see Working with XML-Restricted Characters . Attributes per PutAttributes operation 256 Attributes requested per Select 256 operation Items per BatchPutAttributesoperation 25 Maximum items in Selectresponse 2500 Maximum query execution time 5 seconds Maximum number of unique attributes 20 per Selectexpression Maximum number of comparisons per 20 Selectexpression Maximum response size for Select 1MB Copyright Information
- 49. Editorial • NoSQL vs. SQL • Coder vs. Architect • Business Requirements • Time vs. Features • “The Nightmare Scenario” • “Race to the Bottom” • “Me Too Syndrome”
- 50. Editorial • Relational Databases Need to Catch Up • Meet/Exceed developer expectations • Netflix wouldn’t have fork-lifted ~1 Billion Rows out of Oracle “just for fun”
- 51. Q&A
Editor's Notes
- #2: \n
- #3: \n
- #4: \n
- #5: \n
- #6: \n
- #7: \n
- #8: \n
- #9: \n
- #10: Atomic - \nEither a operation completes successfully, or fails. \nNo partial writes or updates.\n\nConsistent - \nDatabase provides mechanisms to ensure consistent data. \nIf a transaction fails, the database reverts to the previous consistent state. \nIf columns can refer to other tables, references \nto non-existent rows are not allowed.\n\nIsolation - Concurrent operations operate \non written data, not data that is in the process of being modified.\n\nDurability - One a transaction has completed, \nthe transaction&#x2019;s changes will survive hardware failure.\n
- #11: \n
- #12: \n
- #13: \n
- #14: \n
- #15: \n
- #16: \n
- #17: \n
- #18: \n
- #19: Concurrent operations - \nMost of the database operations are just a \nfunction that is operated on each row.\n
- #20: Concurrent operations - \nMost of the database operations are just a \nfunction that is operated on each row.\n
- #21: Concurrent operations - \nMost of the database operations are just a \nfunction that is operated on each row.\n
- #22: Concurrent operations - \nMost of the database operations are just a \nfunction that is operated on each row.\n
- #23: \n
- #24: \n
- #25: \n
- #26: \n
- #27: \n
- #28: \n
- #29: \n
- #30: \n
- #31: Conditional update a ticket item stored in \nSimpleDB where Attribute named &#x201C;Reserved&#x201D; = &#x201C;False&#x201D;, set it to &#x201C;True&#x201D;\n
- #32: \n
- #33: \n
- #34: \n
- #35: \n
- #36: \n
- #37: \n
- #38: \n
- #39: \n
- #40: \n
- #41: Since all data is stored as strings, compare on lexicographical ordering.\nZero pad to ensure &#x201C;10&#x201D; is greater than &#x201C;2&#x201D;.\n\nNegative numbers need to be set with an offset. -5 becomes 5 if the offset is 10.\n\nDates in ISO 8601 for lexicographical comparison.\n
- #42: Scale as you go. AmazonDB has geographic aware nodes. Integrates well \nwith other Amazon AWS products.\n
- #43: \n
- #44: \n
- #45: \n
- #46: I&#x2019;m going to throw out some red meat to the group tonight.\n\n&#x201C;Nightmare Scenario&#x201D;\n\nYour database is so screwed up, \nyou might as well just use a Non-Relational Database.\n\nRelational databases don&#x2019;t do \nsquat if you never bother to use referential integrity.\n\n&#x201C;Race To the Bottom&#x201D;\n\nDesigning database systems is &#x201C;Hard&#x201D; - \nmuch easier to throw out features that were relied on \n- pretty much everything ACID encompasses.\n\n&#x201C;Me Too Syndrome&#x201D; - Can&#x2019;t swing a dead cat without hitting\na new NonRel system. Possibly indicates that it&#x2019;s just a fad?\n\nTwitter - Ruby as a Fad. Gave up and went to the Java platform with Scala.\n
- #47: Schema Versioning:\nWe wouldn&#x2019;t have a conflict between Rel and NonRel \nif it wasn&#x2019;t like pulling teeth trying to update a schema\n\nClustering:\nPaying huge bucks for clustering is gone the way of the dodo.\nLook at Google: Commodity hardware and systems designed to share\nnothing.\n\nEnterprise OSes vs. Linux - Came up from behind, took over the \ndatacenter. Commercial UNIX vendors woke up when the dirt got shoveled\nover them.\n
- #48: \n