MongoDB Days Silicon Valley: Using MongoDB with Adobe AEM Communities

Editor's Notes

• #2: Hello everyone and welcome to this presentation on using Mongo with AEM Communities My name is Kevin Nennig and I’m a Corporate Technical Instructor for Adobe. I’ve been working with Adobe Experience Manager for almost 2 years My job gives me a lot of opportunities to work with new companies looking to implement our AEM. One of the topics a lot of clients want to hear about is the Communities feature of AEM The Current release of AEM is 6.1 and with the release comes an updated Communities Module. In this communities module, 2 out of the 3 implementations use Mongo as the data storage mechanism. In this presentation we will discuss each of these options and understand why Mongo is a good solution for AEM.
• #3: The Adobe Marketing Cloud is a service for digital marketers, advertisers, and publishers that works in conjunction with the Adobe Creative Cloud. The Adobe Marketing Cloud consists of these capabilities : 1. A scalable, secure and open platform to host data & content 2. A multi-channel analytics and decision capability 3. Optimization to increase visitor acquisition, improve conversion and maximize the value of audiences These capabilities are enabled through the following 8 solutions… (name and describe them) The focus of today’s course is on _______.
• #4: AEM is the main focus of this presentation. AEM is a comprehensive content management system for building websites, mobile apps, and forms. It makes it easier to manage marketing content and digital assets
• #5: Within AEM there are 5 main modules – Sites, Assets, Apps, Forms, Communities.
• #6: This presentation is focused on AEM communities 6.1 It allows up to build thriving communities and inspire engaging conversations across audiences from customers to employees to partners.
• #7: Within AEM, there are two different scenarios to use Mongo You can use Mongo at the Microkernel layer to persist all of AEM’s DB to Mongo OR you can use Mongo to store only User Generated Content for a company’s online communities
• #8: Within AEM, there are two different scenarios to use Mongo You can use Mongo at the Microkernel layer to persist all of AEM’s DB to Mongo OR you can use Mongo to store only User Generated Content for a company’s online communities
• #9: The are a few reasons why Mongo works well with storing data for AEM. First off, flexible JSON document model – this is actually what AEM’s persistence microkernel layer is based upon efficient searching with indexing - Compared to other noSQL solutions Built in replication for high availability – Mongo uses replica sets which allow the primary MongoDB to replicate to secondaries for redundancy of the data layer Scaleable – Mongo allows for both horizontal and vertical scaling High concurrent writes - which is good for high traffic on a community site reduces operational overhead — AEM doesn’t have to worry about persistence
• #10: Anyone in the audience that wants to participate can go to this website This Communities website uses MSRP or Mongo Social resource provider to store all user generated content Feel free to answer
• #11: When we start to talk about AEM Communities, there are 3 things that we need to consider, 1 – This allows for content to be served up in the fastest manner possible.
• #12: This is a basic AEM deployment. Your author server is your internal server where you manage all of your content. The publish farm is your publish instances that hold published content of your website. Since AEM builds your pages dynamically for every request, there is a static webserver we call a dispatcher in from of your publish farm The dispatcher caches your static content so that your site is faster
• #13: This is a basic AEM deployment. Your author server is your internal server where you manage all of your content. The publish farm is your publish instances that hold published content of your website. Since AEM builds your pages dynamically for every request, there is a static webserver we call a dispatcher in from of your publish farm The dispatcher caches your static content so that your site is faster
• #14: The second idea that was considered was replicating content wasn’t good enough for last scale solutions 2 – replicating content backwards allows for publish instances to be out of sync for short periods of time
• #15: This concept allow transfer of content from the publish server to the author for review, moderation, and spam check without opening the firewall from the DMZ to the author server.
• #16: This concept allow transfer of content from the publish server to the author for review, moderation, and spam check without opening the firewall from the DMZ to the author server.
• #17: The third idea is that read/write had to be fast for a large scale of transitions and there cannot be data collisions 3 – When dealing with large community site that have hundreds of transactions per second
• #18: Create a common store for user generated content. Where all UGC created is stored in a central repository and is only accessed on request. Where the website itself is still on the publish instance and only the UGC on the current page is needed for rendering
• #19: The Social Resource provider is the solution for the common store This allows for us to have multiple solutions to store UGC depending on a company’s requirements JSRP - uses clustering to have a common database among publish instances where all content can be stored ASRP – Managed and hosted by Adobe MSRP – A mongo replset is used for on prem deployment
• #20: JCR Social Resource Provider. New in 6.1 Default configuration for Communities UGC persisted in publisher JCR repository. Cluster assumed if multiple publishers. Suitable for customers who can run a publish cluster on MongoMK or RdbmsMK UGC not available on author Uses Oak indices. Querying necessary for communities. Sort by date, helpfulness, number of votes. Can’t avoid queries. All of our SRPs have good/flexible querying No more bucketing. With Oak, buckets aren’t necessary, and it made indexing simpler without them, so we are eliminating them. Migration required for all SRPs Open Source tool will be available to export and import to *SRP
• #21: The Adobe Social Resource Provider. Publishers can be a farm. Point all of your instances, including author, at Adobe Social Cloud. All instances look at same copy of data. Shipped in AEM 6.0 UGC is persisted via a cloud service provided and supported by Adobe Integrates with the Adobe Social Analytics Pipeline and Moderation No need to invest in infrastructure to hold UGC
• #22: The Mongo Social Resource Provider or MSRP. New in 6.1. A whole lot like the last picture, except UGC is persisted in a local and dedicated MongoDB and Solr. Suitable for a large volume of UGC Compatible with on-prem publish farm topologies The external solr server is used to create non-biased indexers between the publish instances and the Mongo UGC cloud
• #23: Only the actualy UGC is stored in the Common store. Things such as:
• #24: The other aspects of a community would be users, groups, and the actual site itself. Non of these are stored in the UGC,
• #25: External users and groups should not be available on the internal author server And therefore are only replicated on the publish servers so users can gain the proper rights to the community site on any publish server The Community site itself is managed on the author instance and then replicated to the publish farm just like any normal piece of content
• #26: External users and groups should not be available on the internal author server And therefore are only replicated on the publish servers so users can gain the proper rights to the community site on any publish server The Community site itself is managed on the author instance and then replicated to the publish farm just like any normal piece of content ASC – Adobe Social Cloud
• #27: The next question concerning UGC is the dispatcher….
• #28: If we recall, the dispatcher is used to cache static content from our dynamic AEM instances. Which means that UGC specific information can’t be stored on the dispatcher, which means community pages pose a problem
• #29: If we recall, the dispatcher is used to cache static content from our dynamic AEM instances. Which means that UGC specific information can’t be stored on the dispatcher, which means community pages pose a problem
• #30: Utilizing Mongo as the common store, we can see how much performance increases for User Generated content As a benchmark we found that reverse replication allows for about 5 transactions per second, no matter how many active nodes Where when we use Mongo SRP, we find that we can massively increase transactions by 20x per instance. Which means transactions are linearly scaleable by the amount of AEM publish nodes in the front end.
• #31: Utilizing Mongo as the common store, we can see how much performance increases for User Generated content As a benchmark we found that reverse replication allows for about 5 transactions per second, no matter how many active nodes Where when we use Mongo SRP, we find that we can massively increase transactions by 20x per instance. Which means transactions are linearly scaleable by the amount of AEM publish nodes in the front end.
• #32: If a customer wants to utilize the fastness of a publish farm and wants to have an on prem solution, MSRP is the best solution for the UGC. Lets see how we can set one up
• #33: The collection used for UGC is called msrp-communities, which can be configured. The solr cloud I’m using the default collection collection1 for indexing
• #34: The collection used for UGC is called msrp-communities, which can be configured. The solr cloud I’m using the default collection collection1 for indexing
• #35: We first need to start 3 separate mongo instances listening for an aem replica set
• #36: Then we need to connect to the mongo instance we want to become our primary
• #37: We then can initialize our replica set and check to make sure it’s configured correctly Once the replica set has been initialized your prompt will change to show the current replica set and the current servers state
• #38: We then can initialize our replica set and check to make sure it’s configured correctly Once the replica set has been initialized your prompt will change to show the current replica set and the current servers state
• #39: Once our replica set, aem, is setup we can add the other two mongo instances as members. Remember the least amount of configuration for a replica set requires 3 mongo members
• #40: We then want to check the configurations to make sure all 3 mongo instances are added to the replset
• #41: There are 2 Solr config files that need to be added to the solr configuration before starting These can be obtained from any AEM instance in the communities section Once copied, you can start the solr server on the default port 8983
• #42: Once the mongo replica set and solr server is up and running, The SRP configuration needs to be setup on every AEM server that will be using the UGC We can see an example of that configuration here Notice that we are going to persist UGC to a collection called msrp-communities
• #43: Let’s take a moment to see a demo in action!
• #44: Let’s take a moment to see a demo in action!