Dynamic components using SPA concepts in AEM

Editor's Notes

• #3: Dynamic is a ambiguous term and can mean a lot of things, so lets clear up the context in which we are talking here. Dynamic Components are CQ components that: Depend on the request/context (logged in user data, personalized content, session data, etc.) Can not be cached. Just think of the HTML of the component. Can that HTML be same for all requests? If not, we are probably talking about a dynamic component.
• #4: Now, why are dynamic components so important? Well, first, CQ doesn’t come with out-of-the box solution for dynamic component development. All pages are by default cached, so you need to come up with a custom solution for this problem. Second, we as a CQ developers (or integrators), are usually tasked with developing components that are much more complex than the most common, simple static components (such as text, image, table, etc). We took a quick look at our development over the past few years, and we realized that over 50% of our components are dynamic. I actually expected this number to be much higher, because, most of the time we actually spend on the dynamic components. The static components are present in such high number because a lot of them were created when the project was started. At that point, the most widely used, generic components were implemented, such as: text, image, list, paragraph, key visuals, navigation menus. Of course, these numbers are subjective and are based only on our experience. But I doubt that you can find a person or a company with more than 30 CQ projects, so that you can make a representative statistical sample  Still, because of the high number of dynamic components, and the time we have spent developing them, and also the fact that you most probably have already implemented or integrated some kind of solution for handling them, - we are going to describe their development and share some useful insights.
• #5: So, how are dynamic components usually developed? First step is, to create some kind of placeholder. This placeholder will be put in the dispatcher, that will subsequently perform a request to get the HTML of that component, populated with dynamic data. There are different approaches of what do we put here, we will get back to this in a moment. Now, CQ, actually Sling, has this nice feature, that you can use the component path to request the HTML of the component directly via URL. For example if we have a /page, under /content, and some component entered via the paragraph system, then you can request only the component HTML with the URL as /content/page/_jcr_content/par/component.html. As this will also be cached, we can add a selector, lets say “dynamic”, to this kind of requests. And we will, exclude all the requests that contain the selector from being cached on the dispatcher. Now, we have some basic building blocks for our abstract dynamic component. There are two approaches on how we can implement the “placeholder” part, so we will go through both of them.
• #6: The first approach is to use Server-Side-Includes. SSIs are apache directives that are placed in HTML, and evaluated on the dispatcher while the page is being served. They let you add dynamically generated content to an existing HTML page, without having to serve the entire page from CQ. This means that, in your dynamic SSI component, in the JSP you put the SSI directive, and then when the page will be served from the cache, a request will be made to CQ. Lets go through the diagram: The browser asks for the page.html The dispatcher has cached the page, and for all the dynamic SSI components it has a separate directive, that will make a separate request to CQ. This directive is the placeholder in this approach. Because the request contains the dynamic selector, it is: First: not cached Second: we can use separate rendering script, that will render the content with dynamic data, and not the apache directive After all SSI-directives are evaluated, the dispatcher assembles the complete HTML and returns it to the browser
• #7: The second approach is to use AJAX. This time, when the page is requested, it is served from the dispatcher. For the dynamic components, on the placeholder, the component has rendered JS code. This code will be executed by the browser, that will do additional call (URL to the component path with dynamic selector), and will add the component HTML to some part of the page. Since now the ping-ponging is done between the browser and the CQ instance, and not between the dispatcher and the CQ instance like in the SSI approach, this essentially makes the AJAX approach slower than SSI. Mostly due to the network latency. Why would you use AJAX instead of SSI? Well, in most cases SSI is more suitable, and we have used it more frequently. But there are cases where AJAX approach can find its place: If you have a component that needs more time to get the dynamic data, having an AJAX might be better, because the page will be shown and you can implement a spinner, or something similar, to improve the user-experience Cache Control is a challenge with both approaches, but you can have more success with AJAX
• #8: Let’s do a quick summary about the dynamic component development, since we are not in fact presenting a complete solution here, just a general overview. Sling has various concepts that help you develop a solution for dynamic components: Rendering only a component based on the component-path Sling selectors to select special rendering scripts and to be distinguished on the publish server Sling resource resolution, and especially sling:resourceSuperType that can help abstract the infrastructure code from the business logic All of these will help you develop your solution. It will probably be some-kind of aspect-oriented solution where, the developers will get used to it, and after a while will not even notice much difference between static and dynamic component development. You can also integrate a framework that does this for you. The Sling-Dynamic-Include from Cognifide does this. It was open-sourced and presented 2 years ago on this meet-up. So, go check it out. The question that arises now, is why did we need something else for dynamic components?
• #9: Well, there were multiple reasons, but it all boils down to the type of components and templates we foresaw coming, that we would need to develop. We realized that much of the new pages will be comprised of dynamic data, maybe over 90% of the page. Also we had components that needed to communicate between themselves, update each others state, communicate with the back-end, and so on. It didn’t seem like page refresh would be natural. And we also had some new requirements concerning how we should notify the user if something goes wrong. Besides the normal redirection to error pages, we wanted sometimes to display errors in a notification-like manner. In the end, we realized that what we would need much more front-end logic. We needed a more responsive web application. We needed something that is nowadays done using Single Page Web Application Concepts.
• #10: Why the current approaches were not suited for this? Although our usual approaches were not exactly show stoppers, we preferred to try out a different solution that would feel more natural given the problem at hand. The main issue with the SSI approach was that it already uses the server-side-include mechanism to achieve the dynamic behavior. Using an additional markup refreshing mechanism such as ajax for the updates on top of that would really increase the complexity and lead to bad and inconsistent component design. And although the AJAX components seem to have what we want at first glance, since the refreshing mechanism in that case would be the same for both the initial rendering as well as the updates, we were not quite happy with it for several reasons: the result given from the server was a ready to use markup, which is something that doesn’t necessary need to be done in the backend. In fact there is no good reason for it. Secondly, having the markup creation happen on the server limits the possibility to split up the work between ‘front end’ and AEM developers, which started to look like a blocker. And last but not least, we anticipated that waiting for the server to return the markup may lead to a user interface that is lagging and not responsive enough especially when we do relay on results from web services and other time costly operations. In other words, it was time for a change. Image reference: http://c2.staticflickr.com/4/3047/2754478731_6cac6d30a8.jpg
• #11: We decided to go with Single Page Application concepts. The term SPA is often used to describe web applications that have a desktop look and feel, in a sense that once user accesses the application, he/she is never redirected to a different address, nor the page is refreshed. Of course, we didn’t take this strict SPA approach completely. We still wanted to access pages the normal CQ way. But when a page is loaded, we wanted the components to form kind of a mini application, in a sense, that they will depend on each other where needed, and communicate appropriately. Something like a SPA on a page, or even component level. It is also becoming clearer, that a lot of the front-end logic is going to be shifted from the server to the client. Which is a good thing, since, we were already noticing that the server was generating way too much stuff, that can be easily transferred to the browser.
• #12: How to make a smooth transition to this? We create a CQ component, structured to have SPA behavior. The rendering script, or the JSP, contains only the static component markup - the static HTML, the template, and the initial call to the module, so that it can initialize itself. The complete script is cacheable, and will be cached on the dispatcher. Also, you can still leverage sling:resourceSuperType to do inheritance, and include other components (though usually scripts) to do composition. Each component, besides the rendering script, contains a module. It’s a Java Script file, that follows the Module Pattern, so that we can compose various modules, and can do all the dynamic requirements. Each module, can make multiple calls, to various handlers on the backend. This is a bit different than the usual SSI/AJAX approaches, where we usually have the one handler that populates the module with the dynamic data. We used inheritance quite a lot, so that we can share logic in the Controllers.
• #13: So lets go quickly to what is happening when a page is loaded. First the browser asks for the Page. The call is intercepted by the dispatcher, and at this point, if the page is already cached, it is immediately served. If not, the call is let through to the AEM server, which returns the naked markup. (for simplicity - this sub step is not shown on the diagram). In most of the cases, this call is never made, and the dispatcher serves the markup to the browser. Once the markup is rendered by the browser, it contains JS code that makes one or more AJAX calls to retrieve the actual dynamic data. This calls can either be POSTs (never cached by the dispatcher), or GETs (these are cached by default, so some exceptions need to be configured in the dispatcher). Either way, the requests always reaches the AEM server, where the dynamic data is serialized into a JSON response and sent back to the browser. The browser now has both the naked markup and the JSON, so the third and final step is to ‘inject’ the values into the markup via DOM manipulation. The process is repeated at least once for each component. Wherever it makes sense to break up the dynamic content needed for a single component into more than one logical unit, this is done, and each unit is served by a separate AJAX call. After all the component are initialized, the user starts using the application, and it changes its state, based on the actions done via the modules.
• #14: Lets say a few words about some of the technologies that we used and the reasoning behind them. jQuery was used for creation of the modules. We didn’t use any modern module system that usually comes with libraries like RequireJS or AngularJS. This was mostly due to the team’s jQuery know-how, and a simpler transition to the new type of components. jQuery Template was used for loading HTML templates and populating it with the dynamic data. It’s a jQuery Plugin, that we used quite extensively and submitted patches. GSON was used to create the JSON responses. This turned out to be tremendously useful. Its much more robust and packed with features then the rudimentary sling-json support. It has great conventions, so if you want to serialize a model object, it works pretty much as you like it. Most of the times though, we used GSON Serializes where we converted the model to the actual reasonable response for the corresponding request. It was a nice separation layer, that is very configurable and gives great flexibility to separate the front-end engineering and the back-end business logic. We have always used Spring in the projects, and this was no exception. Most importantly we used MVC to have the nice handlers for the various dynamic calls. We also used some interceptors to improve error handling, now re-written in a new way, that works with JSON responses.
• #15: Lastly, lets talk briefly about the two main benefits that came from implementing this new approach: First and foremost is the improvement in the perceived user experience. SPA components lead to a sleeker, more responsive and user friendly pages. And the second important benefit was the separation of the front end from the backend engineering – where experts in each field could preaty much work on their own part without causing any blockers for each other and work in a more independent manner. This also meant that we were able to create a FE prototype, that works with mocked data in form of JSON objects, which was 100% percent independent from the AEM server, but at the same time still benefit the AEM developers greatly.
• #16: Next, I am briefly going to go through some of the technical details of component implementation. Basically, I will talk you through a simple showcase – that is a simple table component to display the adaptTo() talk schedule, with the options of paging and sorting the results. The full component can be found in the demo project we have provided, and what I will show is a simplified version, with focus on the key elements.
• #17: Let’s take a closer look at the way the mark up is rendered. First of all we have the static markup, which in this case is the table itself, with the header already in place, and an empty body. This HTML snippet is cached in the dispatcher. Next dynamic HTML is generated using a template, which is placed in a script tag and also cached in the dispatcher. All the placeholders, where the dynamic data will be injected are marked with data attributes. In the example here, we can see that the template contains the markup for a singe table row, with the data cells for the “talkname“ and the ‘talktime” clearly marked. To bring all the static markup and the template together , ‘loadTemplate’ is called, on the container which is to include the dynamic markup (in this case it is, the body of the table). The method is called with two arguments, the script element containing the template and the JSON containing the dynamic data. The end result is that each element in the JSON is injected into the template and the result is appended into the allocated container. Lastly, in the component itself, the self initializing script is placed, and it doesn’t contain any business logic, it only initializes the java script module.
• #18: Next interesting point is the JS module itself (technically it is a self invoking anonymous JS function), and this is where most of the magic happens. In this example, you can see that the persondetailstable module of the namespace aemdemo is dependent on the jquery module, as well as the paging and the utils modules (also from the aemdemo namespace) and it also has a single public method, that is the self initialization invoked from the component. What are the functionalities of this module? For simplicity I am going to show only the basic ones: get page by number, sort the entries by a given criteria, and change the page size. Each of these contains an ajax call to the backend, retrieves the data it needs, and renders. Also, each one of the ajax calls has a specific handler in the controller and receives a unique JSON, based on the information it needs to refresh. For example, the sorting needs not update the pager, so the information from the server would not include the pager numbers, and so on… Lastly, I would like to again point out that all the code shown on these slides is cached, and all the resulting dynamic behavior comes from the script execution in conjunction with the dynamic JSON’s coming from the server.
• #20: As we already mentioned, all custom POSTs were handled in a special way by the use of spring controllers, so we eventualy came to a situation where for some reason (broken content or broken implementation) this was not done correctly. To rectify this, we implemented a custom filter as a check handler, that intercepts each call and in the case where the request came from an ajax call from a SPA component, and is to be handled by the SlingPostServlet, we basically know something went wrong