![Scale OUT Using Cloud Technology…?
TL
Very Large Models (VLMs):
Model Transformation (MT)
a key bottleneck
Challenge to application
in industry
Explicitly
Parallelization
[Tisi et al. MoDELS’13]
Distribution
[Benelallam et al. SLE’15]
Implicitly (NoSQL)
Models only
[Gwendal et al. Sci.
Comput. Program 2017]
MT too
[Gwendal et al. ASE’17]](https://image.slidesharecdn.com/models18-181017202121/85/Expressive-and-Efficient-Model-Transformation-with-an-Internal-DSL-of-Xtend-2-320.jpg)
![new Rule('ClassToTable')
.in('c', CD.class_).build()
.out('t', DB.table, [
// environment vbles
val c = 'c'.fetch as Class
val t = 't'.fetch as Table
val pk_col = 'pk_col'.fetch as Column
// bindings
t.name = c.name
t.col += c.attr.fetch as List<Column>
t.col += pk_col
val list = c.incomingReferences
.fetch('ReferenceToFkColumn')
as List<Column>
if (list !== null)
t.col += list
]).build()
.out('pk_col', DB.column, [
// environment vbles
val c = 'c'.fetch as Class
val pk_col = 'pk_col'.fetch as Column
// bindings
pk_col.name = 'pk_' + c.name
]).build()
.build()
YAMTL Rule
rule
• matched | lazy | uniqueLazy
• transient
• filter(EXPR): global constraints
• using(VBLE, EXPR): local variables
• endWith(EXPR): imperative expression
name = "Item"
1:Class
name = "Item"
1:Table
name = "pk_Item"
4:Column
col
TL](https://image.slidesharecdn.com/models18-181017202121/85/Expressive-and-Efficient-Model-Transformation-with-an-Internal-DSL-of-Xtend-8-320.jpg)
![new Rule('ClassToTable')
.in('c', CD.class_).build()
.out('t', DB.table, [
// environment vbles
val c = 'c'.fetch as Class
val t = 't'.fetch as Table
val pk_col = 'pk_col'.fetch as Column
// bindings
t.name = c.name
t.col += c.attr.fetch as List<Column>
t.col += pk_col
val list = c.incomingReferences
.fetch('ReferenceToFkColumn')
as List<Column>
if (list !== null)
t.col += list
]).build()
.out('pk_col', DB.column, [
// environment vbles
val c = 'c'.fetch as Class
val pk_col = 'pk_col'.fetch as Column
// bindings
pk_col.name = 'pk_' + c.name
]).build()
.build()
YAMTL Rule
input pattern elements
• with(VBLE) dependencies
• filter(EXPR): local constraint (optional)
• derivedWith(EXPR)
name = "Item"
1:Class
name = "Item"
1:Table
name = "pk_Item"
4:Column
col
TL](https://image.slidesharecdn.com/models18-181017202121/85/Expressive-and-Efficient-Model-Transformation-with-an-Internal-DSL-of-Xtend-9-320.jpg)
![new Rule('ClassToTable')
.in('c', CD.class_).build()
.out('t', DB.table, [
// environment vbles
val c = 'c'.fetch as Class
val t = 't'.fetch as Table
val pk_col = 'pk_col'.fetch as Column
// bindings
t.name = c.name
t.col += c.attr.fetch as List<Column>
t.col += pk_col
val list = c.incomingReferences
.fetch('ReferenceToFkColumn')
as List<Column>
if (list !== null)
t.col += list
]).build()
.out('pk_col', DB.column, [
// environment vbles
val c = 'c'.fetch as Class
val pk_col = 'pk_col'.fetch as Column
// bindings
pk_col.name = 'pk_' + c.name
]).build()
.build()
YAMTL Rule
output pattern elements
• action(EXPR) dependencies
• drop()
name = "Item"
1:Class
name = "Item"
1:Table
name = "pk_Item"
4:Column
col
TL](https://image.slidesharecdn.com/models18-181017202121/85/Expressive-and-Efficient-Model-Transformation-with-an-Internal-DSL-of-Xtend-10-320.jpg)
![new Rule('ClassToTable')
.in('c', CD.class_).build()
.out('t', DB.table, [
// environment vbles
val c = 'c'.fetch as Class
val t = 't'.fetch as Table
val pk_col = 'pk_col'.fetch as Column
// bindings
t.name = c.name
t.col += c.attr.fetch as List<Column>
t.col += pk_col
val list = c.incomingReferences
.fetch('ReferenceToFkColumn')
as List<Column>
if (list !== null)
t.col += list
]).build()
.out('pk_col', DB.column, [
// environment vbles
val c = 'c'.fetch as Class
val pk_col = 'pk_col'.fetch as Column
// bindings
pk_col.name = 'pk_' + c.name
]).build()
.build()
YAMTL Output Action
fetch
to access execution environment
name = "Item"
1:Class
name = "Item"
1:Table
name = "pk_Item"
4:Column
col
TL](https://image.slidesharecdn.com/models18-181017202121/85/Expressive-and-Efficient-Model-Transformation-with-an-Internal-DSL-of-Xtend-11-320.jpg)
![new Rule('ClassToTable')
.in('c', CD.class_).build()
.out('t', DB.table, [
// environment vbles
val c = 'c'.fetch as Class
val t = 't'.fetch as Table
val pk_col = 'pk_col'.fetch as Column
// bindings
t.name = c.name
t.col += c.attr.fetch as List<Column>
t.col += pk_col
val list = c.incomingReferences
.fetch('ReferenceToFkColumn')
as List<Column>
if (list !== null)
t.col += list
]).build()
.out('pk_col', DB.column, [
// environment vbles
val c = 'c'.fetch as Class
val pk_col = 'pk_col'.fetch as Column
// bindings
pk_col.name = 'pk_' + c.name
]).build()
.build()
YAMTL Output Action
fetch
to resolve references in…
name = "Item"
1:Class
name = "Item"
1:Table
name = "pk_Item"
4:Column
col
TL](https://image.slidesharecdn.com/models18-181017202121/85/Expressive-and-Efficient-Model-Transformation-with-an-Internal-DSL-of-Xtend-12-320.jpg)
![new Rule('ClassToTable')
.in('c', CD.class_).build()
.out('t', DB.table, [
// environment vbles
val c = 'c'.fetch as Class
val t = 't'.fetch as Table
val pk_col = 'pk_col'.fetch as Column
// bindings
t.name = c.name
t.col += c.attr.fetch as List<Column>
t.col += pk_col
val list = c.incomingReferences
.fetch('ReferenceToFkColumn')
as List<Column>
if (list !== null)
t.col += list
]).build()
.out('pk_col', DB.column, [
// environment vbles
val c = 'c'.fetch as Class
val pk_col = 'pk_col'.fetch as Column
// bindings
pk_col.name = 'pk_' + c.name
]).build()
.build()
YAMTL Output Action
a) declative rules
new Rule('AttributeToColumn')
.in('att', CD.attribute).filter([ ... ]).build()
.out('col', DB.column, [ ... ]).build()
.build(),
fetch
to resolve references in…
name = "Item"
1:Class
name = "Item"
1:Table
name = "pk_Item"
4:Column
col
TL](https://image.slidesharecdn.com/models18-181017202121/85/Expressive-and-Efficient-Model-Transformation-with-an-Internal-DSL-of-Xtend-13-320.jpg)
![new Rule('ClassToTable')
.in('c', CD.class_).build()
.out('t', DB.table, [
// environment vbles
val c = 'c'.fetch as Class
val t = 't'.fetch as Table
val pk_col = 'pk_col'.fetch as Column
// bindings
t.name = c.name
t.col += c.attr.fetch as List<Column>
t.col += pk_col
val list = c.incomingReferences
.fetch('ReferenceToFkColumn')
as List<Column>
if (list !== null)
t.col += list
]).build()
.out('pk_col', DB.column, [
// environment vbles
val c = 'c'.fetch as Class
val pk_col = 'pk_col'.fetch as Column
// bindings
pk_col.name = 'pk_' + c.name
]).build()
.build()
YAMTL Output Action
b) lazy rules
new Rule('ReferenceToFkColumn')
.uniqueLazy
.in('ref', CD.attribute).build()
.out('fk_col', DB.column, [ ... ]).build()
.build()
a) declative rules
new Rule('AttributeToColumn')
.in('att', CD.attribute).filter([ ... ]).build()
.out('col', DB.column, [ ... ]).build()
.build(),
fetch
to resolve references in…
name = "Item"
1:Class
name = "Item"
1:Table
name = "pk_Item"
4:Column
col
TL](https://image.slidesharecdn.com/models18-181017202121/85/Expressive-and-Efficient-Model-Transformation-with-an-Internal-DSL-of-Xtend-14-320.jpg)
Expressive and Efficient Model Transformation with an Internal DSL of Xtend
- 2. Scale OUT Using Cloud Technology…? TL Very Large Models (VLMs): Model Transformation (MT) a key bottleneck Challenge to application in industry Explicitly Parallelization [Tisi et al. MoDELS’13] Distribution [Benelallam et al. SLE’15] Implicitly (NoSQL) Models only [Gwendal et al. Sci. Comput. Program 2017] MT too [Gwendal et al. ASE’17]
- 4. Scale UP with… Yet Another Model Transformation Language ! • Internal DSL of Xtend • Compiled to Java and excellent interoperability with JVM programs • High-level API to work with collections • Succinct syntax to work with lambda expressions • IDE support: type checker and debugger… • Focus on • expressiveness: advanced contructs to model MT using ATL as reference • efficiency: MT performance using VIATRA3 as reference • productivity as a consequence of being an internal DSL TL Krikava et al. On the use of an internal DSL for enriching EMF models. OCL and Textual Modelling. 2012. Hinkel et al. Using internal domain-specific languages to inherit tool support and modularity for model transformations. SoSyM. 2017.
- 6. MTs in YAMTL: ORM Example name: String «stereotype» NamedElt «stereotype» Classifier DataType Package isAbstract: Boolean Class * owner attr * type *super * classifiers name: String «stereotype» Named Type Database Column Table * owner col * type * types * tables keyOf key * 0..1 name = "Item" 1:Class multiValued: Boolean Attribute name = "product" multiValued=false 2:Attribute name = "Order" 4:Class name = "date" multiValued=false 3:Attribute name = "items" multiValued=true 5:Attribute attr attr attr 1:Class 4:Class name = "Item" 1:Table name = "product" 2:Column name = "Order" 6:Table name = "date" 3:Column name = "pk_Order" 7:Column col col col name = "pk_Item" 4:Column col name = "fk_Item--items-->Order" 5:Column col name = "Item" 1:Table name = "Invoice" 6:Table typename = "String" 0:DataType typetype 0:DataType Source metamodel (class diagrams) Target metamodel (database schemas) Initial source model Propagation of delta a (case 4) Ms MtInitial target model TL
- 7. YAMTL Module Declaration class cd2db extends YAMTLModule { new () { header().in('cd', CD).out('db', DB) ruleStore( // rule declarations ) helperStore( // helper declarations ) } val CD = CDPackage.eINSTANCE val DB = RelationalPackage.eINSTANCE // other helpers as Xtend methods } Façade interface: loading models, execution, statistics MT name MT signature for traceable models inOut() for in-place MT TL
- 8. new Rule('ClassToTable') .in('c', CD.class_).build() .out('t', DB.table, [ // environment vbles val c = 'c'.fetch as Class val t = 't'.fetch as Table val pk_col = 'pk_col'.fetch as Column // bindings t.name = c.name t.col += c.attr.fetch as List<Column> t.col += pk_col val list = c.incomingReferences .fetch('ReferenceToFkColumn') as List<Column> if (list !== null) t.col += list ]).build() .out('pk_col', DB.column, [ // environment vbles val c = 'c'.fetch as Class val pk_col = 'pk_col'.fetch as Column // bindings pk_col.name = 'pk_' + c.name ]).build() .build() YAMTL Rule rule • matched | lazy | uniqueLazy • transient • filter(EXPR): global constraints • using(VBLE, EXPR): local variables • endWith(EXPR): imperative expression name = "Item" 1:Class name = "Item" 1:Table name = "pk_Item" 4:Column col TL
- 9. new Rule('ClassToTable') .in('c', CD.class_).build() .out('t', DB.table, [ // environment vbles val c = 'c'.fetch as Class val t = 't'.fetch as Table val pk_col = 'pk_col'.fetch as Column // bindings t.name = c.name t.col += c.attr.fetch as List<Column> t.col += pk_col val list = c.incomingReferences .fetch('ReferenceToFkColumn') as List<Column> if (list !== null) t.col += list ]).build() .out('pk_col', DB.column, [ // environment vbles val c = 'c'.fetch as Class val pk_col = 'pk_col'.fetch as Column // bindings pk_col.name = 'pk_' + c.name ]).build() .build() YAMTL Rule input pattern elements • with(VBLE) dependencies • filter(EXPR): local constraint (optional) • derivedWith(EXPR) name = "Item" 1:Class name = "Item" 1:Table name = "pk_Item" 4:Column col TL
- 10. new Rule('ClassToTable') .in('c', CD.class_).build() .out('t', DB.table, [ // environment vbles val c = 'c'.fetch as Class val t = 't'.fetch as Table val pk_col = 'pk_col'.fetch as Column // bindings t.name = c.name t.col += c.attr.fetch as List<Column> t.col += pk_col val list = c.incomingReferences .fetch('ReferenceToFkColumn') as List<Column> if (list !== null) t.col += list ]).build() .out('pk_col', DB.column, [ // environment vbles val c = 'c'.fetch as Class val pk_col = 'pk_col'.fetch as Column // bindings pk_col.name = 'pk_' + c.name ]).build() .build() YAMTL Rule output pattern elements • action(EXPR) dependencies • drop() name = "Item" 1:Class name = "Item" 1:Table name = "pk_Item" 4:Column col TL
- 11. new Rule('ClassToTable') .in('c', CD.class_).build() .out('t', DB.table, [ // environment vbles val c = 'c'.fetch as Class val t = 't'.fetch as Table val pk_col = 'pk_col'.fetch as Column // bindings t.name = c.name t.col += c.attr.fetch as List<Column> t.col += pk_col val list = c.incomingReferences .fetch('ReferenceToFkColumn') as List<Column> if (list !== null) t.col += list ]).build() .out('pk_col', DB.column, [ // environment vbles val c = 'c'.fetch as Class val pk_col = 'pk_col'.fetch as Column // bindings pk_col.name = 'pk_' + c.name ]).build() .build() YAMTL Output Action fetch to access execution environment name = "Item" 1:Class name = "Item" 1:Table name = "pk_Item" 4:Column col TL
- 12. new Rule('ClassToTable') .in('c', CD.class_).build() .out('t', DB.table, [ // environment vbles val c = 'c'.fetch as Class val t = 't'.fetch as Table val pk_col = 'pk_col'.fetch as Column // bindings t.name = c.name t.col += c.attr.fetch as List<Column> t.col += pk_col val list = c.incomingReferences .fetch('ReferenceToFkColumn') as List<Column> if (list !== null) t.col += list ]).build() .out('pk_col', DB.column, [ // environment vbles val c = 'c'.fetch as Class val pk_col = 'pk_col'.fetch as Column // bindings pk_col.name = 'pk_' + c.name ]).build() .build() YAMTL Output Action fetch to resolve references in… name = "Item" 1:Class name = "Item" 1:Table name = "pk_Item" 4:Column col TL
- 13. new Rule('ClassToTable') .in('c', CD.class_).build() .out('t', DB.table, [ // environment vbles val c = 'c'.fetch as Class val t = 't'.fetch as Table val pk_col = 'pk_col'.fetch as Column // bindings t.name = c.name t.col += c.attr.fetch as List<Column> t.col += pk_col val list = c.incomingReferences .fetch('ReferenceToFkColumn') as List<Column> if (list !== null) t.col += list ]).build() .out('pk_col', DB.column, [ // environment vbles val c = 'c'.fetch as Class val pk_col = 'pk_col'.fetch as Column // bindings pk_col.name = 'pk_' + c.name ]).build() .build() YAMTL Output Action a) declative rules new Rule('AttributeToColumn') .in('att', CD.attribute).filter([ ... ]).build() .out('col', DB.column, [ ... ]).build() .build(), fetch to resolve references in… name = "Item" 1:Class name = "Item" 1:Table name = "pk_Item" 4:Column col TL
- 14. new Rule('ClassToTable') .in('c', CD.class_).build() .out('t', DB.table, [ // environment vbles val c = 'c'.fetch as Class val t = 't'.fetch as Table val pk_col = 'pk_col'.fetch as Column // bindings t.name = c.name t.col += c.attr.fetch as List<Column> t.col += pk_col val list = c.incomingReferences .fetch('ReferenceToFkColumn') as List<Column> if (list !== null) t.col += list ]).build() .out('pk_col', DB.column, [ // environment vbles val c = 'c'.fetch as Class val pk_col = 'pk_col'.fetch as Column // bindings pk_col.name = 'pk_' + c.name ]).build() .build() YAMTL Output Action b) lazy rules new Rule('ReferenceToFkColumn') .uniqueLazy .in('ref', CD.attribute).build() .out('fk_col', DB.column, [ ... ]).build() .build() a) declative rules new Rule('AttributeToColumn') .in('att', CD.attribute).filter([ ... ]).build() .out('col', DB.column, [ ... ]).build() .build(), fetch to resolve references in… name = "Item" 1:Class name = "Item" 1:Table name = "pk_Item" 4:Column col TL
- 15. Reuse in YAMTL • Multiple Rule Inheritance and abstract rules • Dispatch semantics: leftmost top-down search for most concrete applicable rule for each object • Execution semantics of output element actions (inherits/overriding) • Module composition by class extension • Inheritance: by default • Redefinition: when a rule (or a helper) in the extending module have a compatible interface with one rule (or helper) in the extended module • Module reuse in other JVM programs using imports TL
- 17. VIATRA CPS Benchmark YAMTL solution: 1. Helper to determine receiving transitions (waitFor) 2. Build the structure of deployment model using rules with high priority 3. Initialize triggers with a transient rule (only updates) 4. Extract CPS traceability model from YAMTL’s internal traceability model action triggers to be set after behaviours are instantiated behaviours are instantiated (copy) traceability model TL
- 18. VIATRA CPS Benchmark • Adaptation • Model generation with VIATRA CPS model generator • Each experiment (size): 13 iterations • Median of inner results (excluding 1st – warm up) 1 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768 65536 131072 262144 1 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768 time (ms) scale Client-server VIATRA EMFTVM Xtend ATL YAMTL EMFTVM (lazy) ATL (lazy) scale factor time (ms) YAMTL EMFTVM (imp.) EIQEMFTVM (decl.) ATL2010 (imp.) ATL2010 (decl.) Xtend (optimized) 8 3k 7k 7 45 36 35 120 124 258 16 5k 15k 14 74 56 65 195 201 979 32 10k 29k 26 113 94 102 445 438 3579 64 20k 59k 49 226 190 212 1400 1346 26865 128 40k 123k 102 480 408 473 5316 4916 54594 256 80k 229k 216 768 786 827 20806 20595 223874 512 160k 453k 450 1442 1607 1887 107526 120443 1024 319k 940k 927 2935 3563 5288 2048 636k 1,940k 1889 6019 7800 15942 4096 1.27M 3.69M 3724 10141 15735 50886 8192 2.54M 7.25M 7610 19145 32866 184803 16384 5.08M 14.26M 15838 36490 72377 32768 10.16M 27.53M 32680 67969 oom • Findings • YAMTL at least one orders of magnitud faster than EIQ • YAMTL also uses resources efficiently • ATL/EMFTVM with lazy rules scales well but trafo not complete TL
- 19. Final Remarks https://yamtl.github.io/ • YAMTL brings declarative M2M transformations to the Xtend world • Advanced model transformation constructs • Support for VLMs/good performance • Productivity and analysis tools available via Xtend/Java • Next: Incremental MT - most scalable solution of TTC’18 • Interested in exploring applications • Thanks to Ábel Hegedüs for integrating YAMTL solution in official VIATRA CPS Benchmark Thank you for your attention Artur Boronat