Qardio experience with Core Data
- 1. Qardio experience with Core Data. Can a fancy stack solve all your problems? Dmitrii Ivanov iOS Team Lead at Qardio or
- 3. In this talk 1. Qardio iOS app 2. Common Core Data issues we faced 3. Our struggle (part I) 4. Questions to ask yourself before diving into Core Data 5. Core Data stacks 6. Our struggle (part II) 7. Results and conclusion
- 4. part 1 of 7
- 5. part 1 of 7
- 6. Data synchronisation in Qardio • Off-line work with locally stored data • Data from our bluetooth devices • Bidirectional exchange with the cloud • Bidirectional exchange with HealthKit Currently: 40 Core Data entities, 50 data model versions part 1 of 7
- 7. Common Core Data issues • Multithreading violation • Context merge issues • Deadlocks and UI freezes • Versioning and migration • Crashes... and more crashes part 2 of 7
- 8. Multithreading violation part 2 of 7
- 9. Collection was mutated while being enumerated Multithreading violation part 2 of 7
- 10. CoreData couldn’t fulfil a fault Multithreading violation part 2 of 7
- 11. • Multithreading violation • Context merge issues • Deadlocks and UI freezes • Versioning and migration • Crashes... and more crashes Common Core Data issues part 2 of 7
- 12. version 3 version 3version 3a version 3b new version: 4 Versioning and migration part 2 of 7
- 13. • Multithreading violation • Context merge issues • Deadlocks and UI freezes • Versioning and migration • Crashes... and more crashes Common Core Data issues part 2 of 7
- 14. Fatal Exception: NSInternalInconsistencyException when saving the context Crashes part 2 of 7
- 15. Our struggle (part I)
- 16. Versioning and migrations • Start with versioning from the beginning • New version per production release if there are any changes • Remember about migrations from the previous versions part 3 of 7
- 17. Synchronisation through the Main context
- 18. Main context abuse part 3 of 7
- 19. Issues with UI performance
- 20. Wrong Stack is the root of all evil
- 21. Questions to ask before integrating Core Data • Do I need Core Data? • Which Core Data benefits can we utilise? • How deeply incapsulated should it be? • Which stack to use? • How to properly use the stack? • Why do I need versioning? • Which rules to follow when writing new code? part 4 of 7
- 22. What is Core Data? Not ORM, not SQLite wrapper but an object graph MVC, MVVM, MVP (VIPER) part 4 of 7
- 23. Do I need Core Data in my project? Not necessarily part 4 of 7
- 24. Core Data essence •Object relations •Object lifecycle (insertion, mutation, deletion) •Data access optimisations (faulting, row cache) part 4 of 7
- 25. FMDB - https://github.com/ccgus/fmdb > 12k stars SQLite.swift - https://github.com/stephencelis/SQLite.swift > 5k stars SQLiteManager4iOS - https://github.com/misato/SQLiteManager4iOS - 125 stars SQift - https://github.com/Nike-Inc/SQift - 39 stars SQLite wrappers YapDatabase - https://github.com/yapstudios/YapDatabase > 3k stars Disk - https://github.com/saoudrizwan/Disk > 2k stars Couchbase - https://github.com/couchbase/couchbase-lite-ios > 1,3k stars LevelDB (C++) - https://github.com/google/leveldb > 14k stars Other options part 4 of 7
- 26. How deeply should it be integrated?
- 29. Abstraction level Deep Integration Relative Abstraction •Easy to use, no extra code •All the benefits: relations, lifecycle, optimisations •Inspired by Apple Complete Abstraction part 4 of 7
- 31. Abstraction level Deep Integration Relative Abstraction •Architectural cleanness an lose coupling •Less overhead with multithreading •Less direct access to the coordinator and the store Complete Abstraction •Easy to use, no extra code •All the benefits: relations, lifecycle, optimisations •Inspired by Apple part 4 of 7
- 32. What Core Data stack to choose?
- 33. What is Core Data Stack? NSManagedObjectContext (main, private) NSPersistentStoreCoordinator Store (SQLite, XML, Binary, In-memory) part 5 of 7
- 34. Simple Pros: • Simplicity • Data base interaction is fast Cons: • No background operations part 5 of 7
- 35. Nested 1 (PSC - Main - Worker) Pros: • Automatic syncing the Worker context • Disposable Workers Cons: • I/O to the store in the main thread (for all the contexts) part 5 of 7
- 36. Nested 2 (PSC - Worker - Main) Pros: • I/O in the background Cons: • Manual syncing the Main context • Busy Worker blocks the store from the Main context part 5 of 7
- 37. Nested Context Pros: • Disposable Workers • Automatic merging Workers to the Main context • I/O on a Private context Cons: • Pushing the changes down to the store • Passing all the Workers’ changes through the Main context • Significant UI impact on high-load operations • No merge policy between the contexts part 5 of 7
- 38. Shared Coordinator Pros: • Independent communication to the coordinator • Common row cache • Adjustable merge policy • Flexible context merge Cons: • Manual setup of the merge policy • Merge is not instant • The policy to solve deleted objects conflicts part 5 of 7
- 39. Shared Store Pros: • Minimal interaction between the contexts • Ability to work with the store from completely different locations Cons: • No common row cache • Accessing the disc when merging changes • No ObjectID (just URIRepresentation) part 5 of 7
- 40. Mixed Pros: • Background I/O to the store • Disposable Workers • Maximum synchronisation between the Workers and the Main context • Separate stack for high-load operations Cons: • Complex set up and tuning • Additional complexity in using part 5 of 7
- 41. Functional comparison part 5 of 7
- 43. How to choose the stack? • Pick the least complex one • Consider your data flow and use cases a. which data is being changed from UI, which - from the background b. is there any high-load operations? c. is there a lot of simultaneous high-load operations and UI data access? d. how possible the conflicts in data changes are? e. how important the entry threshold to the stack is • It’s all about your priorities part 5 of 7
- 44. Functional comparison part 5 of 7
- 45. Our struggle (part II)
- 46. Qardio: migration to Mixed Stack BNRCoreDataStack • Easy async initialising with just one function • Binding context with notifications observing • Pushing the changes down to the store • Handy extensions for Core Data classes part 6 of 7
- 47. Improvements caused by the migration • Code revision • Proper contexts for different tasks (Main, Worker, Batch Worker) • Merge policy set up part 6 of 7
- 48. Some issues were solved (no merge conflicts, less crashes and UI problems) New issues started to appear
- 49. Developers are the root of all evil
- 50. Further improvements • Knowledge sharing inside the team • Abstracting some models from Core Data • Multithreading sanitation part 6 of 7
- 51. Abstracting some models from Core Data part 6 of 7
- 52. Abstracting some models from Core Data part 6 of 7
- 53. Further improvements • Knowledge sharing inside the team • Abstracting some models from Core Data • Multithreading sanitation part 6 of 7
- 54. Multithreading sanitation part 6 of 7
- 55. “All that is left to us is honor!” part 6 of 7
- 56. General Rules • Understand your edge of framework abstraction • Don’t access NSManagedObjects out of their context • Remember that NSManagedObject is not NSObject • Keep in mind versioning and migrations part 7 of 7
- 57. •Easy to use, no extra code •All the benefits: relations, lifecycle, optimisations •Inspired by Apple Abstraction level Deep Integration Relative Abstraction •Architectural cleanness an lose coupling •Less overhead with multithreading •Less direct access to the coordinator and the store Complete Abstraction We are somewhere here part 7 of 7
- 58. General Rules • Understand your edge of framework abstraction • Don’t access NSManagedObjects out of their context • Remember that NSManagedObject is not NSObject • Keep in mind versioning and migrations part 7 of 7
- 59. Mixed Stack Usage Rules •Only UI related operations in the Main MOC •Limited operations in the Regular Workers •All the rest should go to the Batch Worker part 7 of 7
- 60. Results • UI issues disappeared • Context merge issues disappeared • Migration and versioning problems disappeared • Multithreading violations (almost) disappeared • Amount of crashes dramatically decreased (less than 1 crash/day instead of dozens) part 7 of 7
- 61. Conclusions • Core Data may bring you much pain • Deep understanding of the framework before writing production code is essential • Share knowledge inside the team • Stack is not a silver bullet (although quite important) • Create clear rules and principles for writing code • Enforce the rules with any possible tools • Core Data may bring you much pain • Deep understanding of the framework before writing production code is essential • Share knowledge inside the team • Stack is not a silver bullet (although quite important) • Create clear rules and principles for writing code • Enforce the rules with any possible tools part 7 of 7 telegram: @topolog e-mail: dmtopolog@gmail.com