WSO2 Dep Sync for Artifact Synchronization of Cluster Nodes
3 likes1,681 views
The document discusses deployment synchronization between cluster nodes in WSO2 products. It introduces the need for deployment synchronization in clustered environments to distribute artifacts and metadata transparently. It demonstrates how the Deployment Synchronizer (DepSync) feature works in WSO2 products using a SVN repository. It shows how to configure a minimal WSO2 cluster with one management node and one worker node connected by a load balancer, by enabling DepSync and making the appropriate configuration changes.
WSO2 Dep Sync for Artifact Synchronization of Cluster Nodes
- 1. DepSync for Deployment Artifact Synchronization Between the Cluster Nodes Pradeep Fernando Kasun Gajasinghe Senior Software Engineer Software Engineer pradeep@wso2.com kasung@wso2.com
- 2. Agenda • The need for deployment synchronization • Brief introduction to WSO2 product clusters • Demo - Complete Application server cluster with DepSync + fronted by WSO2 ELB
- 3. Why Deployment Synchronization? • Artifact distribution and deployment should be transparent to the end-users • Manual artifact copying is acceptable for some extent in standalone product clusters. • Will that work in an elastically scaling cloud environment ??
- 4. Typical WSO2 Product Cluster • Manager and Worker nodes. • Manager is for administration purposes - artifact upload, applying security,etc • Worker nodes responsible for serving requests. • The setup is only valid for products that support artifact deployment: • Application Server • Enterprise Service Bus • etc
- 5. Management and Worker Node Separation
- 6. Why Management & Worker Node Separation Crash Course • Proper separation of concerns - management nodes specialize in management of the setup while worker nodes specialize in serving requests to deployment artifacts • Only management nodes are authorized to add new artifacts into the system or make configuration changes • Worker nodes can only deploy artifacts & read configuration • Lower memory foot in the worker nodes because the management console related OSGi bundles are not loaded • Improved security - management nodes can be behind the internal firewall & be exposed to clients running within the organization only, while worker nodes can be exposed to external clients.
- 7. Deployment Synchronization • Distributing deployment artifacts & related metadata to all nodes in a homogeneous cluster is a typical requirement for a clustered deployment of any middleware platform • Provides a consistent, and reliable cluster • Automated synchronization without need of any user interaction • Artifact Metadata contains service policies, and other important service configuration details
- 8. Deployment Synchronizer • WSO2 products provides this through Deployment Synchronizer (DepSync) mechanism • A new feature addition to WSO2 products • Synchronization done using a central repository, usually, a SVN server • We call it SVN-based Deployment Synchronizer • Extensible - may be a Git based one in the future.
- 9. The Deployment Synchronization Mechanism
- 10. What it synchronizes? • Web Applications • JAXWS / JAXRS Applications • Proxy Services • Data Services • BPEL • Basically all the deployable artifacts in axis2 repo of Carbon products (ex. repository/deployment/server) • Service Metadata – (details on policies, transports etc.)
- 11. Enabling DepSync : Carbon.xml DepSync Configuration <DeploymentSynchronizer> <Enabled>true</Enabled> <AutoCommit>true|false</AutoCommit> <AutoCheckout>true</AutoCheckout> <RepositoryType>svn</RepositoryType> <SvnUrl>https://10.100.3.115/svn/repos/as</SvnUrl> <SvnUser>wso2</SvnUser> <SvnPassword>wso2123</SvnPassword> <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId> </DeploymentSynchronizer> ex. local-file system based SvnUrl - file:///C:/demo/svnrepo
- 12. Multicast vs. WKA Multicast • Cluster is going to be set up in a network where multicasting is allowed Well-Known Address • Cloud based deployment • Members are distributed across datacenters & regions • Multicasting blocked
- 13. DepSync and Multi-Tenancy • DepSync synchronizes the artifacts among the product cluster in a tenant aware manner. • Ghost deployment is supported in the depsync aware manner in order reduce the initial deployment and request serving latencies.
- 14. Configuring a minimal WSO2 cluster setup w/ Deployment Synchronization DEMONSTRATION
- 16. Process 1. Set up the Environment for Deployment Synchronizer 2. Configure Load Balancer 3. Configure Management Node - Application Server 4. Configure Worker Node - Application Server
- 17. PART 1 Set up the Environment for Deployment Synchronizer
- 18. Set up the Environment for Deployment Synchronizer • We will setup the SVN-based DepSync • For this, you need the following o svnkit osgi bundle o TortoiseSVN / Silk SVN (Windows), Subversion command line package(Unix) -optional
- 19. Set up the Environment for Deployment Synchronizer • Download svnkit bundle from here - http://goo.gl/CVR2F • If you will be using a remote svn location for depsync, you may skip the creation of local svn repository. • To create a local svn repository, o if you have svn command line client, enter following command. svnadmin create C:demosvnrepo o if you are using TortoiseSVN, then first create a folder at any place you like. Say - C://demo/svnrepo . Now go in to that folder, right-click -> TortoiseSVN -> "Create Repsitory here" o Now, the SVN URL for this location is - file:///C:/demo/svnrepo
- 20. Set up the Environment for Deployment Synchronizer • We will use this svn url when configuring the manager and worker nodes. So, remember the location where you created this. DepSync config will also be configured there.
- 21. Enabling DepSync : Carbon.xml DepSync Configuration <DeploymentSynchronizer> <Enabled>true</Enabled> <AutoCommit>true|false</AutoCommit> <AutoCheckout>true</AutoCheckout> <RepositoryType>svn</RepositoryType> <SvnUrl>https://10.100.3.115/svn/repos/as</SvnUrl> <SvnUser>wso2</SvnUser> <SvnPassword>wso2123</SvnPassword> <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId> </DeploymentSynchronizer> ex. local-file system based SvnUrl - file:///C:/demo/svnrepo
- 22. PART 2 Configure Load Balancer
- 23. Configure Load Balancer - Loadbalancer.conf • Download and extract wso2elb-2.0.0.zip o Let the extracted directory be WSO2_LB_HOME • Open loadbalancer.conf file in the WSO2_LB_HOME/repository/conf
- 24. Configure Load Balancer • Configuring the WSO2 Elastic Load Balancer is now complete • To start the server o Go to the WSO2_LB_HOME/bin folder using command line o Enter the command `wso2server.bat` o Notice the logs printed by TribesClusteringAgent
- 25. PART 3 Configure Management Node - Application Server
- 26. Configure Management Node • Download and extract wso2as-5.0.0.zip o Rename extracted directory to wso2as-5.0.0-manager o Let the extracted directory be WSO2_AS_MGR_HOME
- 27. Configure Management Node - axis2.xml • Enable clustering at axis2 level. <clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true"> • Change the membershipScheme of clustering to wka (well- known address) <parameter name="membershipScheme">wka</parameter> • Set the cluster domain. This must be the same which we defined in loadbalancer.conf <parameter name="domain">wso2.as.domain</parameter>
- 28. Configure Management Node - axis2.xml • Set localMemberHost <parameter name="localMemberHost">mgt.appserver.wso2.com</parameter> • Set localMemberPort <parameter name="localMemberPort">4250</parameter> • Add a new property named "subDomain"and set it to 'mgt' inside the parameter "properties" <parameter name="properties"> <property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/> <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/> <property name="subDomain" value="mgt"/> </parameter>
- 29. • Add load balancer as a well-known member. o Use the IP/hostname and port defined in LB's axis2.xml <members> <member> <hostName>127.0.0.1</hostName> <port>4000</port> </member> </members>
- 30. Configure Management Node – catalina-server.xml • Since the mgt node is fronted by an LB, we need to configure proxy ports associated with HTTP and HTTPS connecters. o Defaults for http and https are 8280 and 8243 respectively. <Connector protocol="org.apache.coyote.http11.Http11NioProtocol" port="9763" proxyPort="8280“ --- <Connector protocol="org.apache.coyote.http11.Http11NioProtocol" port="9443“ proxyPort="8243“ ---
- 31. Configure Management Node – carbon.xml • Set port offset to avoid port conflicts with other nodes (LB). <Offset>1</Offset> • Update mgtHostName and HostName elements in carbon.xml as shown below <HostName>appserver.wso2.com</HostName><MgtHostName>mgt.appserver .wso2.com</MgtHostName> • Copy svnkit bundle to repository/components/dropins
- 32. Configure Management Node - Deployment Synchronizer • The deployment synchronizer config is in carbon.xml. • Uncomment the DeploymentSynchronizer xml segment. • Set AutoCommit to true. • Set the SVNUrl • No need to set username/password if you didn't set it for local svn repo <DeploymentSynchronizer> <Enabled>true</Enabled> <AutoCommit>true</AutoCommit> <AutoCheckout>true</AutoCheckout> <RepositoryType>svn</RepositoryType> <SvnUrl>file:///C:/webinar-setup/demo/final/svnrepo</SvnUrl> <SvnUser>username</SvnUser> <SvnPassword>password</SvnPassword> <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId> </DeploymentSynchronizer>
- 33. Configure Management Node • Configuring the WSO2 Application Server is now complete • To start the server o Go to the WSO2_AS_MGR_HOME/bin folder using command line o Enter the command `wso2server.bat` o Notice the logs printed by TribesClusteringAgent and corresponding cluster joining logs in load balancer. • You can now login to management console via https://mgt.appserver.wso2.com:8243/carbon/ • Notice that requests we sent to services (including WSDL) will be directed to worker nodes.
- 35. PART 4 Configure Worker Node - Application Server
- 36. Configure Worker Node • We can use a copy of a management node, and convert it to a worker node easily since it will have most of the settings • So, instead of using a fresh appserver copy, make a copy of the management node we just created • Rename the new folder as wso2as-5.0.0-worker • Let this directory be WSO2_AS_WORKER_HOME
- 37. Configure Worker Node - axis2.xml • In addition to the settings we already have, do the following. • Set the localMemberPort to 4251 <parameter name="localMemberPort">4251</parameter> • This node belongs to the "worker"subDomain, so, change the property "subDomain" to worker (instead of mgt) <parameter name="properties"> <property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/> <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/> <property name="subDomain" value="worker"/> </parameter>
- 38. Configure Worker Node – carbon.xml • First set a new port offset <Offset>2</Offset> • Comment out/remove the mgtHostName configuration • Set the AutoCommit property inside Deployment Synchronizer to false. This MUST be done because the worker nodes of a cluster should NOT commit (write) artifacts <DeploymentSynchronizer> <Enabled>true</Enabled> <AutoCommit>false</AutoCommit> <AutoCheckout>true</AutoCheckout> <RepositoryType>svn</RepositoryType> <SvnUrl>file:///C:/webinar-setup/demo/final/svnrepo</SvnUrl> <SvnUser>username</SvnUser> <SvnPassword>password</SvnPassword> <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId> </DeploymentSynchronizer>
- 39. Configure Worker Node – carbon.xml • • Now, it's time to start the server. o NOTE: The workerNode system property must be set when starting the worker nodes every time. wso2server.bat -DworkerNode=true
- 40. Testing the cluster There are many ways to test our cluster deployment. Let's follow a simpler path. • Log in to the management console of Application Server management node • Deploy a new Axis2 Web service (Go to Manage --> Axis2 Services --> Add --> AAR service) • Once the service is deployed in the management node go to the services list and click on Tryit • Invoke the service
- 41. QUESTIONS?
- 42. THANK YOU!!