Deterministic Lazy Mutable OCL Collections
Download as ODP, PDF1 like138 views
The document discusses the implementation and performance of deterministic lazy OCL (Object Constraint Language) collections under the Eclipse Foundation's EPL 1.0 license. It outlines various collection types, their performance characteristics, issues related to determinism, and methods for lazily evaluated collections. The author emphasizes the necessity of determinism in OCL applications and the trade-offs between eager and lazy evaluations, highlighting potential memory savings and speed improvements from mutable collection evaluations.
Deterministic Lazy Mutable OCL Collections
- 1. Made available under EPL 1.0 Deterministic Lazy Mutable OCL Collections Edward Willink Willink Transformations Ltd Eclipse Foundation MMT Component co-Lead OCL Project Lead QVTd Project Lead QVTo Committer OMG (Model Driven Solutions) OCL 2.3, 2.4, 2.5 RTF Chair QVT 1.2, 1.3, 1.4 RTF Chair OCL 2017 @ STAF 2017 20th July 2017
- 2. 20-July-2017 Deterministic Lazy OCL Collections 2Made available under EPL 1.0 Overview Collection performance New Collection implementation bypass/reconsider specification obstacles unify Bag, OrderedSet, Sequence, Set fewer conversions deterministic - predictable, debuggable Sequence - same size/speed Set - 13% larger, faster/slower lazy - usually smaller, slower lazy+partial - usually smaller, faster mutable/cached - quadratic => linear, smaller, faster
- 3. 20-July-2017 Deterministic Lazy OCL Collections 3Made available under EPL 1.0 Specification obstacles is OCL deterministic? no partial Collection evaluation no Collection mutation many Collection kinds
- 4. 20-July-2017 Deterministic Lazy OCL Collections 4Made available under EPL 1.0 Traditional Collection Value Classes Distinct Bag/OrderedSet/Sequence/Set classes
- 5. 20-July-2017 Deterministic Lazy OCL Collections 5Made available under EPL 1.0 Traditional Collection Implementation Delegate from OCL to Java semantics
- 6. 20-July-2017 Deterministic Lazy OCL Collections 6Made available under EPL 1.0 New Collection Implementation Single OCL class Bag/OrderedSet /Sequence/Set 'Two' Java collections ArrayList<T> deterministic iteration HashMap<T,Integer> unique-element => repeat count One lazy loading Iterator Many accessing Iterators
- 7. 20-July-2017 Deterministic Lazy OCL Collections 7Made available under EPL 1.0 Size list map new vs old Sequence ordered elements null no change Set ordered unique elements unique element => 1 13% larger OrderedSet ordered unique elements unique element => 1 13% larger Bag ordered unique elements unique element => count 13% larger 64 bit machine list element pointer - 8 bytes set/map node + pointers - 60 bytes map + list - 68 bytes => +13% ?? custom combined implementation ??
- 8. 20-July-2017 Deterministic Lazy OCL Collections 8Made available under EPL 1.0 Bag-aware iteration OrderedSet, Sequence, Set list iterator returns each element once Bag default iterator returns each collection element once multiple returns for repeated content bag iterator returns each distinct element once single return for each repeated content repeat count accessible from iterator faster bag-aware impementation
- 9. 20-July-2017 Deterministic Lazy OCL Collections 9Made available under EPL 1.0 Creation Speed 2 collections to create rather than 1 'new' about 2 times slower
- 10. 20-July-2017 Deterministic Lazy OCL Collections 10Made available under EPL 1.0 Iteration Speed simple list rather than sparse tree iteration 'new' 3 times faster for large sets iteration faster than creation more than 3 iterations needed to outweigh creation
- 11. 20-July-2017 Deterministic Lazy OCL Collections 11Made available under EPL 1.0 Determinism as specified Non-determinism is bad Is OCL deterministic? 11.7.1: Collection::asSequence() : Sequence(T) A Sequence that contains all the elements from self, in an order dependent on the particular concrete collection type. 11.7.2: Set::asSequence() : Sequence(T) A Sequence that contains all the elements from self, in undefined order. Ditto asOrderedSet(). 11.9.1 Collection::any ... an indeterminate choice of one of them is returned ... (non-normative) A.2.5.5 Note that the semantics of the operation asSequence is nondeterministic. normative: asSequence unknown, determinate non-normative: asSequence non-determinate overall: unclear
- 12. 20-July-2017 Deterministic Lazy OCL Collections 12Made available under EPL 1.0 Implemented Determinism OCL Set implemented by Java HashSet asSequence computed by Java iterator iteration order determined by hash codes hash code determined by object location location dependent on garbage collection timing Convenient implementation is non-determinate OCL is therefore non-determinate OCL-based applications non-determinate M2M transformations non-determinate hard to justify, hard to debug
- 13. 20-July-2017 Deterministic Lazy OCL Collections 13Made available under EPL 1.0 NewCollection Determinism Every Collection has a deterministic list Collection::any can be respecified as the first choice 13% non-Sequence Collection size penalty swings/roundabouts speed gain/penalty acceptable cost for a very fundamental fix new mitigating opportunities...
- 14. 20-July-2017 Deterministic Lazy OCL Collections 14Made available under EPL 1.0 Specified Eagerness, Invalidity specification exposition implies full evaluation one operation/iteration after another specification mandates full evaluation any invalid collection element => invalid result requires full evaluation to check for not-invalid a->including(b)->excludes(c) get: a compute: a+b check: a+b excludes c invalidity => full eager evaluation
- 15. 20-July-2017 Deterministic Lazy OCL Collections 15Made available under EPL 1.0 What is invalid? null - a well-defined undefined value invalid - every other bad null navigation divide by zero Sequence/OrderedSet index out of bounds 4-level logic to accommodate invalid mayBeNull = null or mayBeNull.doSomething() null = null or null .doSomething() true or invalid true
- 16. 20-July-2017 Deterministic Lazy OCL Collections 16Made available under EPL 1.0 Are Program Failures invalid? OCL Specification is silent about Stack Overflow Network Connection Failure etc. presumably program failure is invalid Normal OCL usage program failures are fatal, result cannot be useful program failure as invalid is perverse, inefficient failure may be 'caught' and so succeed OCL-specified Debugger / Job control job failure must be useable in a computation
- 17. 20-July-2017 Deterministic Lazy OCL Collections 17Made available under EPL 1.0 Program Failure Clarification clarify: Let program failures always be 'strict' cannot be 'caught' by and/or pedantically now 5-level logic can only be caught by oclIsFailure() supports the obscure Job control use case invalid sources are enumerable many expressions are provably not-invalid redundant computations can be skipped lazy/partial evaluation can be valid and profitable
- 18. 20-July-2017 Deterministic Lazy OCL Collections 18Made available under EPL 1.0 Eager Collection Evaluation a->including(b)->including(c) 1: load and cache a 2: load and cache b 3: compute ab, 4: cache ab 5: load and cache c 6: compute abc, 7: cache abc 1 2 3 4 5 6 7 8
- 19. 20-July-2017 Deterministic Lazy OCL Collections 19Made available under EPL 1.0 Lazy Collection Evaluation a->including(b)->including(c) 1: load and cache b 2: load and cache c 3.1: request an abc element 4.1: request an ab+c element 5.1: request an ab element 6:.1 request an a+b element 7.1: request an a element 8.1: request an a element 1 2 345678
- 20. 20-July-2017 Deterministic Lazy OCL Collections 20Made available under EPL 1.0 Multi-access Lazy Evaluation Cache to avoid repeated lazy first lazy computation populates the cache further computations re-use the cache
- 21. 20-July-2017 Deterministic Lazy OCL Collections 21Made available under EPL 1.0 NewCollection modes lazy Bag, OrderedSet, Sequence, Set iterator, list, map eager/lazily cached Sequence iterator, list, map eager/lazily cached Bag, OrderedSet, Set iterator, list, map lazy/eager/lazily cached Collection Operation e.g. IncludingIteration extends NewCollectionValue overridden loading iterator performs the algorithm Gotcha - lazy/cached must be determined early awkward as part of a casual API 'easy' using automated compile-time analysis
- 22. 20-July-2017 Deterministic Lazy OCL Collections 22Made available under EPL 1.0 Full Eager/Lazy Performance a->including(b)->including(c) lazy has fewer caches to create eager has fewer function calls
- 23. 20-July-2017 Deterministic Lazy OCL Collections 23Made available under EPL 1.0 Immutable repeated computation aCollection->iterate(e; acc : Set(String) | acc->including(e.name)) eager: each iteration executes including() once new collection created for each iteration 0.5 * N * N element assigns - bad O(N*N) speed O(N*N) intermediate elements - bad O(N*N) size lazy: each iteration daisy chains 2 iterators final lazy result 2 * N * N iterator next() calls - bad O(N*N) speed 2 * N small iterators - good O(N) size
- 24. 20-July-2017 Deterministic Lazy OCL Collections 24Made available under EPL 1.0 Mutable repeated computation OCL side-effect free - no Collection mutation Set::including(element) rather than Set::add(element) but specification is on observable behavior unobserved behavior can perform mutation Compile-time expression analysis, rewrite acc is only accessed by including() aCollection->iterate(e; acc : Set(String) | acc->mutableIncluding(e.name)) mutable: each iteration updates collection N element assigns - good O(N) speed N element result - good O(N) size
- 25. 20-July-2017 Deterministic Lazy OCL Collections 25Made available under EPL 1.0 Mutable Accumulation Performance aCollection->iterate(e; acc : Coll(String) | acc->including(e.name)) clear benefit for modest/large collection possible degration for small collections 'in the noise' exploits a compile time analysis
- 26. 20-July-2017 Deterministic Lazy OCL Collections 26Made available under EPL 1.0 select() idioms sourceCollection->select(booleanPredicate) type conformance OCL 2.0: S->select(oclIsKindOf(MyType)).oclAsType(MyType) OCL 2.4: S->selectByKind(MyType) amenable to global compile-time analysis appropriate caching, similar to allInstances lookup S->select(element | element.name = wantedName) full search for each lookup - O(N) applications often have many lookups - O(N*N) introduce a value2key cache - O(N)
- 27. 20-July-2017 Deterministic Lazy OCL Collections 27Made available under EPL 1.0 select() lookup performance let S = Sequence{1..N} in let t = S->collect(i|Tuple{x=i}) in S->collect(i | t->select(x = i)) Compile-time analysis to balance trade-offs possible for QVT, less obvious for OCL huge speed benefit iff there are many lookups significant cache cost, perhaps 8 times
- 28. 20-July-2017 Deterministic Lazy OCL Collections 28Made available under EPL 1.0 Status Oct 2016: ... Martin's lightning presentation Nov 2016: Initial Eclipse OCL experiments major API compatibility difficulty: next major version Apr 2017: Results, workshop submission May 2017: Eclipse OCL integration API compatibility resolved by a level of wrappers Gotcha, lazy operation calls get wrong parameters need lazy operation call / let variable June 2017: Context 'iterator' Today: only available on ewillink/509670 branch
- 29. 20-July-2017 Deterministic Lazy OCL Collections 29Made available under EPL 1.0 Summary Deterministic OCL is feasible (and ? required) 13% non-Sequence Collection memory cost Partial collection evaluation is feasible but only if program failures are not invalid Lazy collection evaluation is challenging huge memory saving, marginal slow down lazy variable access must use correct state Mutable collection evaluation very beneficial linear rather than quadratic speed/size cost Partial+Lazy+Mutable+Deterministic mutual speed-ups shared compile-time analysis