Are Transactions Right For You? Using and Not Abusing Transactions
Download as PPTX, PDF0 likes240 views
Transactions in MongoDB provide atomic, consistent, isolated, and durable (ACID) behavior for multi-document operations. The document discusses how MongoDB supports transactional behavior both with and without explicit transactions through features like write concerns, read concerns, and embedding related data within documents. It also compares MongoDB's transaction model to relational databases and outlines some limitations of transactions in MongoDB, such as increased resource usage for sharded collections.
![{
"_id" :
ObjectId("5ea7137e2295dc3b6afb6057"),
"name" : "Sheeri Cabral",
"accounts" : [
{ "id" : 111,
"type" : "savings",
"balance" :
NumberDecimal("1000") },
{ "id" : 222,
"type" : "checking",
"balance" :
NumberDecimal("500") },
],
"phone" : "+1 123-456-7890",
"email" : "sheeri.cabral@mongodb.com"
}
Document
Model
Atomicity](https://image.slidesharecdn.com/mdbccon-2169-cabral-200609134854/85/Are-Transactions-Right-For-You-Using-and-Not-Abusing-Transactions-26-320.jpg)
![Document
Model
(technically)
People
{
"_id" : ObjectId("5ea7137e2295dc3b6afb6057"),
"name" : "Sheeri Cabral",
"phone" : "+1 123-456-7890",
"email" : "sheeri.cabral@mongodb.com"
"account_ids " : [ 111, 222 ]
}
Accounts
{
"id" : 111,
"type" : "savings",
"balance" : NumberDecimal("1000")
}
{
"id" : 222,
"type" : "checking",
"balance" : NumberDecimal("500")
}](https://image.slidesharecdn.com/mdbccon-2169-cabral-200609134854/85/Are-Transactions-Right-For-You-Using-and-Not-Abusing-Transactions-27-320.jpg)
![{
"_id" : ObjectId("5ea7137e2295dc3b6afb6057"),
"name" : "Sheeri Cabral",
"accounts" : [
{ "id" : 111,
"type" : "savings",
"balance" : NumberDecimal("1000") },
{ "id" : 222,
"type" : "checking",
"balance" : NumberDecimal("500") },
],
"phone" : "+1 123-456-7890",
"email" : "sheeri.cabral@mongodb.com"
}
db.getCollection("accounts").updateOne(
{ "_id": ObjectId("5ea7137e2295dc3b6afb6057") },
{ $inc: {
"accounts.$[sv].balance": -100,
"accounts.$[ck].balance": 100
} },
{ arrayFilters: [
{ "sv.id": 111 },
{ "ck.id": 222 }
]
});
Document
Model
Atomicity
{
"_id" : ObjectId("5ea7137e2295dc3b6afb6057"),
"name" : "Sheeri Cabral",
"accounts" : [
{ "id" : 111,
"type" : "savings",
"balance" : NumberDecimal("900") },
{ "id" : 222,
"type" : "checking",
"balance" : NumberDecimal("600") },
],
"phone" : "+1 123-456-7890",
"email" : "sheeri.cabral@mongodb.com"
}
db.getCollection("accounts").updateOne(
{ "_id": ObjectId("5ea7137e2295dc3b6afb6057") },
{ $inc: {
"accounts.$[sv].balance": -100,
"accounts.$[ck].balance": 100
} },
{ arrayFilters: [
{ "sv.id": 111 },
{ "ck.id": 222 }
]
});
db.getCollection("accounts").updateOne(
{ "_id": ObjectId("5ea7137e2295dc3b6afb6057") },
{ $inc: {
"accounts.$[sv].balance": -100,
"accounts.$[ck].balance": 100
} },
{ arrayFilters: [
{ "sv.id": 111 },
{ "ck.id": 222 }
]
});
db.getCollection("accounts").updateOne(
{ "_id": ObjectId("5ea7137e2295dc3b6afb6057") },
{ $inc: {
"accounts.$[sv].balance": -100,
"accounts.$[ck].balance": 100
} },
{ arrayFilters: [
{ "sv.id": 111 },
{ "ck.id": 222 }
]
});](https://image.slidesharecdn.com/mdbccon-2169-cabral-200609134854/85/Are-Transactions-Right-For-You-Using-and-Not-Abusing-Transactions-28-320.jpg)
![{
"_id" : ObjectId("5ea7137e2295dc3b6afb6057"),
"name" : "Sheeri Cabral",
"accounts" : [
{ "id" : 111,
"type" : "savings",
"balance" : NumberDecimal("1000") },
{ "id" : 222,
"type" : "checking",
"balance" : NumberDecimal("500") },
],
"phone" : "+1 123-456-7890",
"email" : "sheeri.cabral@mongodb.com"
}
Document
Model
Atomicity
{
"_id" : ObjectId("7137e2295dc3b6afb60575ea"),
"name" : "Antonio Cabral",
"accounts" : [
{ "id" : 333,
"type" : "checking",
"balance" : NumberDecimal(”700") }
],
}](https://image.slidesharecdn.com/mdbccon-2169-cabral-200609134854/85/Are-Transactions-Right-For-You-Using-and-Not-Abusing-Transactions-31-320.jpg)
![Using MongoDB Transactions
function transferMoney( dbName, session, src, trg, amount ) {
session.start_transaction()
const db = session.get_database(dbName)
db.accounts.updateOne(
{ $and: [{"_id": src.uid)}, { "accounts.id": src.acctId } ] },
{ $inc: { "accounts.$.balance" : -amount } } )
db.accounts.updateOne(
{ $and: [{"_id": trg.uid)}, { "accounts.id": trg.acctId } ] },
{ $inc: { "accounts.$.balance" : amount } } )
session.commit_transaction()
}
accounts
{
_id : …
name : …
accounts" : [
{ id: …,
type : …,
balance : … },
{…},
],
…
}
src = { uid: …, acctId: …}
trg = { uid: …, acctId: …}
amount](https://image.slidesharecdn.com/mdbccon-2169-cabral-200609134854/85/Are-Transactions-Right-For-You-Using-and-Not-Abusing-Transactions-49-320.jpg)
Are Transactions Right For You? Using and Not Abusing Transactions
- 1. Are Transactions Right For You? Sheeri Cabral Product Manager, MongoDB Using and Not Abusing Transactions
- 3. We can’t cross the tracks. The train comes! That would be a big problem waste big problem Because….’cause we waste yucky gas.
- 4. waste big problem We use transactions to avoid having a big problem Using unnecessary transactions is a waste of resources
- 5. MongoDB Transactions MongoDB has transactions More than one document More than one shard
- 6. What is a transaction? “A transaction is a data state transition” - Vlad Mihalcea A Beginner’s Guide to ACID and Database Transactions https://vladmihalcea.com/a-beginners-guide-to-acid-and-database-transactions/
- 7. Data State Transition Before a transaction Data is in one state After a transaction Data is in another state Canonical example Bank transfer
- 8. ACID Transactions …and the problems they solve
- 9. Atomicity All actions either succeed or fail There is no partial success A C I D
- 10. Atomicity All actions either succeed or fail There is no partial success Data state before $1000 in savings, $500 in checking Data state after $900 in savings, $600 in checking $$$ $$$ $$$ $$ $$$ $ Total: $1500
- 11. Total: $1600 Total: $1400 Atomicity $$$ $$$ $$$ $$ $$$ $$$ $$$$ $$$ $$$ $$$ $$$ $$$ $$$ $$$ $$$ Total: $1500
- 12. Atomicity All actions either succeed or fail There is no partial success Data state before $1000 in savings, $500 in checking Data state after $900 in savings, $600 in checking A C I D
- 13. Consistency Follow the Rules “any given database transaction must change affected data only in allowed ways” Data Correctness Data types Key constraints Data validation Triggers Different from CAP Theorem ACID consistency is not related to data freshness C A I D
- 14. Isolation Levels Can a transaction see writes inside another concurrent transaction? Tradeoff Performance vs. accuracy Issues Dirty reads Non-repeatable reads Phantom reads I A CD
- 15. Durability No losing data New state is permanent Hard to achieve Log, then write Looks at log D A C I Recovery
- 16. Why We Use ACID Transactions Atomicity Everything happens or nothing happens Consistency Data follows the rules Isolation Controls what is seen mid-transaction by other transactions A C I D Durability Changes are saved
- 17. No Transactions < 4.0? Summer 2018 Took 9 years (1st release 2009) Document model Transactions are hard! MySQL MySQL had none from 1995 – 2001. Innobase Fully merged in 2013
- 18. Relational DBs and MongoDB: Transactions are slower
- 19. Social media: dirty/phantom reads are OK. Airplane ticketing and seating US restaurants doesn’t use transactions. https://martinfowler.com/bliki/Transactionless.html - 2007 “The rationale for not using transactions was that they harm performance at the sort of scale that eBay deals with.” Do you really need a transaction?
- 20. Relational DBs and MongoDB: Transactions are slower
- 21. Code-your-own transactions in the app layer Optimistic concurrency control – online shopping cart Pessimistic concurrency control – ticket sales and timers Controllable performance hit Going Transactionless in the Database
- 23. Which steps need transactions? Ordering a coffee: - order/pay - making/receiving coffee Get transactional behavior without transactions Use Fewer Transactions Strictly ordered queue Not-so-strictly ordered queue
- 24. Relational Model Atomicity savings account_id balance 111 1000 UPDATE savings SET balance=balance-100 WHERE account_id=111 UPDATE checking SET balance=balance+100 WHERE account_id=222 savings account_id balance 111 900 checking account_id balance 222 500 checking account_id balance 222 600
- 25. Accounts account_id balance 111 1000 222 500 UPDATE savings SET balance=balance-100 WHERE account_id=111 UPDATE checking SET balance=balance+100 WHERE account_id=222 Accounts account_id balance 111 900 222 500 Accounts account_id balance 111 900 222 600 Relational Model Atomicity
- 26. { "_id" : ObjectId("5ea7137e2295dc3b6afb6057"), "name" : "Sheeri Cabral", "accounts" : [ { "id" : 111, "type" : "savings", "balance" : NumberDecimal("1000") }, { "id" : 222, "type" : "checking", "balance" : NumberDecimal("500") }, ], "phone" : "+1 123-456-7890", "email" : "sheeri.cabral@mongodb.com" } Document Model Atomicity
- 27. Document Model (technically) People { "_id" : ObjectId("5ea7137e2295dc3b6afb6057"), "name" : "Sheeri Cabral", "phone" : "+1 123-456-7890", "email" : "sheeri.cabral@mongodb.com" "account_ids " : [ 111, 222 ] } Accounts { "id" : 111, "type" : "savings", "balance" : NumberDecimal("1000") } { "id" : 222, "type" : "checking", "balance" : NumberDecimal("500") }
- 28. { "_id" : ObjectId("5ea7137e2295dc3b6afb6057"), "name" : "Sheeri Cabral", "accounts" : [ { "id" : 111, "type" : "savings", "balance" : NumberDecimal("1000") }, { "id" : 222, "type" : "checking", "balance" : NumberDecimal("500") }, ], "phone" : "+1 123-456-7890", "email" : "sheeri.cabral@mongodb.com" } db.getCollection("accounts").updateOne( { "_id": ObjectId("5ea7137e2295dc3b6afb6057") }, { $inc: { "accounts.$[sv].balance": -100, "accounts.$[ck].balance": 100 } }, { arrayFilters: [ { "sv.id": 111 }, { "ck.id": 222 } ] }); Document Model Atomicity { "_id" : ObjectId("5ea7137e2295dc3b6afb6057"), "name" : "Sheeri Cabral", "accounts" : [ { "id" : 111, "type" : "savings", "balance" : NumberDecimal("900") }, { "id" : 222, "type" : "checking", "balance" : NumberDecimal("600") }, ], "phone" : "+1 123-456-7890", "email" : "sheeri.cabral@mongodb.com" } db.getCollection("accounts").updateOne( { "_id": ObjectId("5ea7137e2295dc3b6afb6057") }, { $inc: { "accounts.$[sv].balance": -100, "accounts.$[ck].balance": 100 } }, { arrayFilters: [ { "sv.id": 111 }, { "ck.id": 222 } ] }); db.getCollection("accounts").updateOne( { "_id": ObjectId("5ea7137e2295dc3b6afb6057") }, { $inc: { "accounts.$[sv].balance": -100, "accounts.$[ck].balance": 100 } }, { arrayFilters: [ { "sv.id": 111 }, { "ck.id": 222 } ] }); db.getCollection("accounts").updateOne( { "_id": ObjectId("5ea7137e2295dc3b6afb6057") }, { $inc: { "accounts.$[sv].balance": -100, "accounts.$[ck].balance": 100 } }, { arrayFilters: [ { "sv.id": 111 }, { "ck.id": 222 } ] });
- 29. Relational DBs offer transactional behavior without transactions in ONE case: changing one row
- 30. MongoDB has more options for transactional behavior without transactions
- 31. { "_id" : ObjectId("5ea7137e2295dc3b6afb6057"), "name" : "Sheeri Cabral", "accounts" : [ { "id" : 111, "type" : "savings", "balance" : NumberDecimal("1000") }, { "id" : 222, "type" : "checking", "balance" : NumberDecimal("500") }, ], "phone" : "+1 123-456-7890", "email" : "sheeri.cabral@mongodb.com" } Document Model Atomicity { "_id" : ObjectId("7137e2295dc3b6afb60575ea"), "name" : "Antonio Cabral", "accounts" : [ { "id" : 333, "type" : "checking", "balance" : NumberDecimal(”700") } ], }
- 32. MongoDB has transactional behavior without transactions if you model your data that way
- 33. MongoDB has transactional behavior with fewer transactions if you model your data that way
- 34. Transactions in MongoDB are multi-document
- 35. MongoDB Consistency / Data Rules Flexible schema != lack of schema Similar to relational world: • Unique indexes, _id • Required fields, data type • Enum
- 36. MongoDB Consistency / Data Rules • More flexibility with MongoDB • Range • Most query selectors, including $regex and complex logic • validationLevel • validationAction • strict (all) • moderate (grandfathered in) • error (reject) • warn (accept)
- 37. Isolation dirty reads non-repeatable reads phantom reads none of the above Tradeoff Issues What should happen?
- 39. readConcern OUTSIDE of a transaction MongoDB Isolation
- 41. readConcern majority Read Committed OUTSIDE of a transaction P SS MongoDB Isolation
- 42. P SS MongoDB Isolation readConcern majority linearizable local Read Committed Read Committed OUTSIDE of a transaction
- 43. readConcern majority linearizable local Read Committed Read Committed Read Uncommitted OUTSIDE of a transaction P SS MongoDB Isolation
- 44. readConcern majority linearizable local available Read Committed Read Committed Read Uncommitted Read Uncommitted OUTSIDE of a transaction P SS MongoDB Isolation
- 45. readConcern INSIDE a transaction - ONE readConcern majority linearizable local available snapshot multi-doc txn Read Committed (not available) Read Uncommitted (not available) Repeatable Read MongoDB Isolation
- 46. MongoDB Durability writeConcern Within transactions or not Outside of transactions, per-operation level Per-transaction level for multi-document transactions <number> • 0 • 1 “majority” <---- This makes MongoDB durable, without transactions! All transaction in MongoDB should be multi-document!!
- 47. MongoDB Outside of Transactions ACID Transactions in the Relational World Atomicity Multiple writes in one document are atomic Consistency Schema validation – extremely flexible Isolation readConcern - Durability writeConcern Atomicity Consistency Schema validation – extremely flexible Isolation Durability Changes are saved READ COMMITTED READ UNCOMMITTED Everything happens or nothing happens Controls what is seen mid-transaction by other transactions
- 48. MongoDB Outside of Transactions MongoDB Transactions Atomicity Multiple writes in one document are atomic Consistency Schema validation – extremely flexible Isolation readConcern - Durability writeConcern Atomicity Consistency Isolation Durability writeConcern READ COMMITTED READ UNCOMMITTED READ COMMITTED READ UNCOMMITTED Multi-document, multi-shard atomic writes Schema validation – extremely flexible readConcern REPEATABLE READ
- 49. Using MongoDB Transactions function transferMoney( dbName, session, src, trg, amount ) { session.start_transaction() const db = session.get_database(dbName) db.accounts.updateOne( { $and: [{"_id": src.uid)}, { "accounts.id": src.acctId } ] }, { $inc: { "accounts.$.balance" : -amount } } ) db.accounts.updateOne( { $and: [{"_id": trg.uid)}, { "accounts.id": trg.acctId } ] }, { $inc: { "accounts.$.balance" : amount } } ) session.commit_transaction() } accounts { _id : … name : … accounts" : [ { id: …, type : …, balance : … }, {…}, ], … } src = { uid: …, acctId: …} trg = { uid: …, acctId: …} amount
- 50. Time: 60s Number of documents: no limit Transient failures catch and re-try transaction use retryable writes Existing transactions block DDL Existing DDL blocks transactions MongoDB Transaction Limitations
- 51. More shards = more resources used Shards with arbiters will abort transactions Transactions have an impact on collections that do chunk migrations MongoDB Transaction Limitations - sharding
- 52. Thank you Sheeri Cabral | Product Manager | MongoDB | @sheeri A Beginner’s Guide to ACID and Database Transactions by Vlad Mihalcea Wikipedia entry on consistency Martin Fowler on ebay not using transactions Further topic: SELECT…for UPDATE inside MongoDB Transactions