Can we run the Whole Web on Apache Sling?
2 likes779 views
The document discusses the capabilities of Apache Sling and its potential to run large-scale web applications. It emphasizes dynamic routing, instance registration, and the benefits of using customized Sling Docker images for efficient scaling. Key points include resource-driven routing, proper repository management for performance, and a focus on optimizing storage through bundled JCR nodes.
![Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Query Precisely
• Use specific nodetype
• Relativize property names
wrt root of micro tree
• Include path restrictions
• Use union if nodetypes differ
SELECT *
FROM [nt:base] AS a
WHERE
a.[type] = 'image'
SELECT *
FROM [dam:Asset] AS a
WHERE ISDESCENDANTNODE([/content/dam])
AND a.[jcr:content/metadata/type] = 'image'](https://image.slidesharecdn.com/adaptto2016-whole-web-on-apache-sling-bertrand-delacretaz-chetan-mehrotra-161004120409/85/Can-we-run-the-Whole-Web-on-Apache-Sling-18-320.jpg)
![Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Observe Precisely
▪ Observation costs resources
▪ Use JackrabbitEventFilter to filter on
▪ Multiple paths
▪ Nodetypes
▪ In works* filtering based on
▪ Property names
▪ Node names
*OAK-4796
JackrabbitEventFilter eventFilter
= new JackrabbitEventFilter()
.setAbsPath(paths[0])
.setNodeTypes(new String[]{"dam:Asset"})
.setEventTypes(Event.NODE_ADDED)
.setIsDeep(true);
eventFilter.setAdditionalPaths("/content/en");
JackrabbitObservationManager om =
(JackrabbitObservationManager)session.getWorkspace()
.getObservationManager();
om.addEventListener(this, eventFilter);](https://image.slidesharecdn.com/adaptto2016-whole-web-on-apache-sling-bertrand-delacretaz-chetan-mehrotra-161004120409/85/Can-we-run-the-Whole-Web-on-Apache-Sling-19-320.jpg)
![Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Bundle Multiple JCR Node in one Document
*OAK-1312
+ book.jpg (nt:file)
+ jcr:content
- jcr:data
Bundling Pattern
+ jcr:system/documentstore/bundlor
+ nt:file
- pattern - [jcr:content]
{
"_id": "2:/test/book.jpg",
"_commitRoot": {"r1560bfe1650-0-1": "0"},
":pattern": {"r1560bfe1650-0-1": "["str:jcr:content"]"},
"jcr:primaryType": {"r1560bfe1650-0-1": ""nt:file""}
"jcr:content/jcr:data": {"r1560bfe1650-0-1": ""bar""},
}
▪ Store subtree aggregates of specific
nodes in same NodeDocument
▪ JCR Node to Mongo mapping
▪ 1 JCR Node = 1 Mongo Doc
▪ 1 dam:Asset ~ 20 JCR Node
▪ 1M Assets = 20M Mongo Doc
▪ Nodetype as content annotation hint
to optimize storage
▪ Benefits
▪ Lower size of _id index
▪ Less number of queries to read the micro
tree
Bundled NodeDocument](https://image.slidesharecdn.com/adaptto2016-whole-web-on-apache-sling-bertrand-delacretaz-chetan-mehrotra-161004120409/85/Can-we-run-the-Whole-Web-on-Apache-Sling-25-320.jpg)
Can we run the Whole Web on Apache Sling?
- 1. APACHE SLING & FRIENDS TECH MEETUP BERLIN, 26-28 SEPTEMBER 2016 Can we run the whole Web on Apache Sling? Bertrand Delacretaz & Chetan Mehrotra @bdelacretaz - @chetanmeh Sling committers and PMC members CQ/AEM core team members, Adobe slides revision 2016-09-26
- 2. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 The Whole Web ??2 Are you guys crazy? (yes, but not that much)
- 3. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 3 Where’s Sling going? from SERVERSto SYSTEMS
- 4. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 4 httpd mod_lua script mod_proxy haproxy etcd -> config MongoDB processor selector more pools… more pools… fileserver pool slingshot pool default pool more pools… etcd monitoring / metrics REDDR - Resource Driven Dynamic Routing Sling resource/script resolution drives routing 1 2 3 4 1 2 3 4 Request to Sling Processor Selector Response with a single word: fileserver +Header: Sling-Processor-Role:fileserver haproxy routes to fileserver pool Rebuild haproxy config on changes Sling instances announce themselves to etcd: -IP address -Port -Role Pools are actually haproxy “backends” fun! client request port 80 port 81 Sling-Processor-Role
- 5. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 5 REDDRREsource-Driven Dynamic Routing
- 6. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 6 REDDR routing to fileserver instances pool httpd mod_lua script mod_proxy haproxy etcd -> config processor selector fileserver pool1 2 3 4 0 1 2 3 4 Request to Sling Processor Selector Response with a single word: fileserver +Header: Sling-Processor-Role:fileserver haproxy routes to fileserver pool 0 Client request .routing script engine + select based on resource type and parent types. Default servlets and scripts disabled. Select backend pool based on Sling-Processor-Role header, using haproxy “acl” generated from etcd
- 7. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 7 Dynamic Registration of Sling Instances regenerate haproxy config based on etc data
- 8. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 8 haproxy etcd -> config more pools… more pools… fileserver pool slingshot pool default pool more pools… etcd haproxy config is rebuilt via event-driven confd + reload.sh script Sling instances announce themselves to etcd: -IP address -Port -Role Dynamic wiring of Sling instances in the haproxy pools 1 2 1 2
- 9. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 9 Building customised Sling Docker images with just a provisioning model + trivial Dockerfile
- 10. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 10 base/pom.xml base/src/main/docker/Dockerfile base/src/main/docker/slingroot/announce.sh base/src/main/docker/slingroot/start.sh base/src/main/docker/slingroot/wait-for-it.sh default-processor/pom.xml default-processor/src/main/docker/Dockerfile default-processor/src/main/provisioning/composum-browser.txt default-processor/src/main/provisioning/default-processor.txt default-processor/src/main/provisioning/launchpad.txt default-processor/src/main/provisioning/… Base and default-processor images FROM ch.x42.at16.base COPY ${project.build.finalName}.jar /opt/sling/launchpad.jar Dockerfile for default-processor Custom Sling Docker im ages
- 11. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 11 DYNAMIC CLUSTER DEMO A Big Sling Cluster…on my laptop
- 12. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 12 httpd mod_lua script mod_proxy haproxy etcd -> config MongoDB processor selector more pools… more pools… fileserver pool slingshot pool default pool more pools… etcd monitoring / metrics REDDR - Resource Driven Dynamic Routing Sling resource/script resolution drives routing 1 2 3 4 1 2 3 4 Request to Sling Processor Selector Response with a single word: fileserver +Header: Sling-Processor-Role:fileserver haproxy routes to fileserver pool Rebuild haproxy config on changes Sling instances announce themselves to etcd: -IP address -Port -Role Pools are actually haproxy “backends”
- 13. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 13 “Background Worker” Sling instance minimal, disposable, focused
- 14. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Asset worker pool default pool MongoDB ActiveMQ 5 2 1 3 4 1 2 3 4 Request to Sling Frontend server Write to NodeStore Add job to message queue Pick up of job by one of the worker 5 Write back of Job result Worker Profile • Tailor made Sling instance • Min required bundles • Low footprint • JCR API used mostly for CRUD • No observation • Disposable • Take post processing load off frontend servers
- 15. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 15 Repository Etiquette be precise while talking to repository and thou shall not be troubled!
- 16. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Choose the right nodetype Avoid nt:unstructured • Orderable children • Limits writes Prefer oak:Unstructured OAK-657 + dashboard (nt:unstructured) - role_observer - projects-outdoors - role_editor - projects-editor + gadgets (nt:unstructured) + team + asset + dashboard (oak:Unstructured) - role_observer - projects-outdoors - role_editor - projects-editor + gadgets (oak:Unstructured) + team + asset
- 17. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Choose the right nodetype Avoid nt:resource with nt:file • Referenceable • 1 nt:resource = 1 entry in uuid index • 1M Files = 1M entry in uuid index Prefer oak:Resource + book.jpg (nt:file) - jcr:createdBy - admin + jcr:content (nt:resource) - jcr:lastModifiedBy - admin - jcr:mimeType - image/jpeg - jcr:uuid - "dafe0c9c-1872-4397" OAK-4567 + book.jpg (nt:file) - jcr:createdBy - admin + jcr:content (oak:Resource) - jcr:lastModifiedBy - admin - jcr:mimeType - image/jpeg
- 18. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Query Precisely • Use specific nodetype • Relativize property names wrt root of micro tree • Include path restrictions • Use union if nodetypes differ SELECT * FROM [nt:base] AS a WHERE a.[type] = 'image' SELECT * FROM [dam:Asset] AS a WHERE ISDESCENDANTNODE([/content/dam]) AND a.[jcr:content/metadata/type] = 'image'
- 19. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Observe Precisely ▪ Observation costs resources ▪ Use JackrabbitEventFilter to filter on ▪ Multiple paths ▪ Nodetypes ▪ In works* filtering based on ▪ Property names ▪ Node names *OAK-4796 JackrabbitEventFilter eventFilter = new JackrabbitEventFilter() .setAbsPath(paths[0]) .setNodeTypes(new String[]{"dam:Asset"}) .setEventTypes(Event.NODE_ADDED) .setIsDeep(true); eventFilter.setAdditionalPaths("/content/en"); JackrabbitObservationManager om = (JackrabbitObservationManager)session.getWorkspace() .getObservationManager(); om.addEventListener(this, eventFilter);
- 20. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Define your own types ▪ Nodetype/mixin are content annotation ▪ Repository uses them to ▪ Enforce structure ▪ Determine index rules ▪ Filter observation events ▪ Bundle Nodes* (new stuff!) *OAK-1312
- 21. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Prefer Lucene Property Indexes ▪ Property Indexes ▪ Cause conflict in index data ▪ Causes commit root to be ‘/’ ▪ Stored as nodes ▪ Good for sparse and full sync case foo=bar bar=/a/b Commit Root
- 22. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Prefer Lucene Property Indexes … ▪ Lucene Property Indexes ▪ Async indexing ▪ No impact on commit root ▪ Compact storage ▪ Multi restriction evaluation ▪ Are not they async? ▪ In works near real time indexing* *OAK-4412 foo=bar Commit Root
- 23. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 FUTURE PERFORMANCE-RELATED Oak FEATURES work in progress!
- 24. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Mongo Segment NSDocument NS Sling Segment NSDocument NS Sling Segment NS as Local Secondary Store ▪ Segment NS act a local copy of remote repository like a local git repo ▪ Updated via Observation ▪ Handles read call for “not so recently modified” Nodes ▪ Reduces read latency ▪ Configured to store certain path. Defaults to ‘/’ *OAK-4180
- 25. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 Bundle Multiple JCR Node in one Document *OAK-1312 + book.jpg (nt:file) + jcr:content - jcr:data Bundling Pattern + jcr:system/documentstore/bundlor + nt:file - pattern - [jcr:content] { "_id": "2:/test/book.jpg", "_commitRoot": {"r1560bfe1650-0-1": "0"}, ":pattern": {"r1560bfe1650-0-1": "["str:jcr:content"]"}, "jcr:primaryType": {"r1560bfe1650-0-1": ""nt:file""} "jcr:content/jcr:data": {"r1560bfe1650-0-1": ""bar""}, } ▪ Store subtree aggregates of specific nodes in same NodeDocument ▪ JCR Node to Mongo mapping ▪ 1 JCR Node = 1 Mongo Doc ▪ 1 dam:Asset ~ 20 JCR Node ▪ 1M Assets = 20M Mongo Doc ▪ Nodetype as content annotation hint to optimize storage ▪ Benefits ▪ Lower size of _id index ▪ Less number of queries to read the micro tree Bundled NodeDocument
- 26. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 26 CODA The Whole Web, really?
- 27. Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 27 CODA Thank you for attending! Chetan Mehrotra (@chetanmeh) Bertrand Delacretaz (@bdelacretaz) REDDR enables (extreme) scaling driven by Sling resource/script resolution. Building customized Sling instances is easy: provisioning model + trivial Dockerfile. From servers to systems! Precise and focused repository types and operations improve performance.