2018 open apereomontreal-moving-a large-sakai-installation-to-the-cloud
- 1. Credit: Photo by Loïc Romer MOVING A LARGE SAKAI INSTALLATION TO THE CLOUD Francois Campbell OPENCOLLAB TLP
- 2. Who is this for? This presentation will focus on large mature sakai instance(s) that are integrated with a variety of other systems. Some of the lessons learnt can be applied to smaller or “fresh” installations. Installations larger than 35,000 users per year.
- 3. Credit: Photo by Tourisme Montréal, Stéphan Poulin Structure Phase 1: Deciding to go to the cloud. Phase 2: Where do we start? Phase 3: What decisions do we need to make? Phase 4: How do we make the move? Phase 5: Preparing to move. Phase 6: Moving … future feedback.
- 4. Credit: Photo by Tourisme Montréal, Stéphan Poulin PHASE 1: DECIDING TO GO TO THE CLOUD
- 5. Credit: Photo by Sofitel Montreal Golden Mile Self Reflection Before going down this path, answer a few questions: ● Why is the cloud your end destination? ● Are there specific benefits that it offers over that of self-hosting? ● Are there shortcomings to your current installation and infrastructure? ● Will being in the cloud give you more or less flexibility and control? ● Increased performance not always guaranteed. ● Increased scalability Upwards and downwards, are you going to benefit from this? ● Redundancy? ● Disaster Recovery? What are the measurable objectives you hope to achieve by moving to the cloud?
- 6. Credit: Photo by Sofitel Montreal Golden Mile External Concerns Things to bare in mind! ▪ When moving to the cloud you will be losing complete control and accessibility to your sakai instance. ▪ The complexity of your Sakai installation and related systems and services will increase. ▪ The architecture for the entire solution will grow in scope. Monitoring ▪ Back-ups ▪ Access
- 7. Credit: Photo by Olivier Bousquet PHASE 2: WHERE TO START
- 8. Credit: Photo by Tourisme Montréal, Stéphan Poulin Beginning Do you have an existing LMS? ▪ No - Much easier for you, start in the cloud if possible ▪ Yes- Buckle Up and let's begin.
- 9. Credit: Photo by Tourisme Montréal, Stéphan Poulin Auditing What resources and services does your LMS require. The obvious dependencies: ▪ Database ( MySQL DB - easy , Oracle HARD or MS SQL DB - do-able ) ▪ LDAP / AD ▪ File Storage ( Sakai Resources ) ▪ Student Information System ( SIS ) The not so obvious ▪ Any custom tools that are connecting to other databases or services ▪ Any custom technology that is datacenter specific ( Networking work arounds ) ▪ Dependencies from Sakai to other systems ▪ Dependencies from other systems to Sakai
- 10. Credit: Photo by Tourisme Montréal, Stéphan Poulin Auditing Actions we recommend. Perform an audit on the following: ▪ Specs - of sakai nodes/ load balancers / resource housing servers of services ▪ Specs Dependent services or systems ▪ Configurations that are environment specific and would need to be removed or modified for the cloud transition. ▪ Ideally you would have an architecture plan already in place. ▪ Understand all the dependencies your LMS has on other services or systems and vice versa. ▪ The output of the audit will inform your next set of decisions. Single points of failure need to be identified.
- 11. Credit: Photo by Tourisme Montréal, Stéphan Poulin Determining Scope of Move Micro Move Will you only move the LMS into the cloud and retain all other servers and services on site? Partial Move Will you only move the essentials of the LMS into the cloud..e.g. Sakai, DB, Resources Full Move Will you move everything related to the LMS, including all dependent tools, services and systems into the cloud.
- 12. Credit: Photo by Tourisme Montréal, Stéphan Poulin What Cloud Type should be chosen? What Cloud Type should be chosen? ▪ VMs in the cloud or Platforms as a Service ▪ Containers in the cloud ( Dockerized Sakai ) ▪ Services in the cloud ( Sakai broken into services - untried & untested) VMs or Platforms as a Service This has the closest resemblance to the architecture of the existing LMS and is what we refer to as Sakai in the cloud. Containers in the cloud This is not something we can recommend at this point. We have spent some time determining how Sakai behaves as a containerized solution and the architecture poses many obstacles. The benefits are many but the challenges to mold Sakai to this solution are not trivial.
- 13. Credit: Photo by Tourisme Montréal, Stéphan Poulin What Cloud Type should be chosen? Services in the cloud Our view is that Sakai cannot be broken down into a service architecture like Amazon or Google app engine, based on our skills and experience. We are not saying that it is impossible to do so, but it is not something we could pursue. Each type of Cloud solution will require its own tooling, techniques and strategies. VMs or Platform as a service give the institution the ability to retain as much tooling, and strategies in place as they did on their own infrastructure. Things like monitoring, alerts, backups and disaster recovery will still be possible with their existing infrastructure it will now just be slightly more complex, with added layer(s)
- 14. Credit: Photo by Tourisme Montréal, Stéphan Poulin Identifying the cloud providers 1. Listing your requirements for the your solution and ensuring that the service/ provider can cater to your needs. 2. Establish what the cost will be for an equivalent setup for hardware and services will be, compared to your existing infrastructure. 3. Does the provider make provision for Scaling upwards and downwards, and is this automated or will a solution have to be developed? 4. Debugging and troubleshooting, are there existing tools or will you need to develop new tools, metrics, and techniques? 5. Deployment pipelines, will this be easier and faster or will it be complex and time consuming. 6. Staff experience with the technology 7. Also real world usage experience for users. 8. Licensing, what are the implications of moving your licensed software to the cloud, are there any other services or software that would need to be taken into consideration.
- 15. Credit: Photo by Alain Régimbald “ Any services and systems that will be moving into the cloud also need to be scalable. They will need to be part of your solution architecture. “
- 16. Credit: Photo by Tourisme Montréal, Stéphan Poulin Software Architecture ( LMS & Dependencies) ▪ Moving into the cloud theoretically should allow one to be able to gain performance, scalability and stability improvements. ▪ However this is not guaranteed. ▪ Any existing issues ( software, and dependencies ) you are experiencing on your current infrastructure will be transferred to the cloud with additional layers of complexity added on. ▪ The software solutions have to be able to take advantage of the cloud technology and tools. New software and software solutions which were written with the cloud in mind will unlock these benefits out of the box, but solutions written for older architectures will be required to adapt to this new architecture.
- 17. Credit: Photo by Tourisme Montréal, Stéphan Poulin Software Architecture ( LMS & Dependencies) Legacy and monolithic systems may actually perform worse in the cloud than in their current environments. This is of course dependent on the cloud solution chosen. Legacy or monolithic systems will perform the same if they are deployed to Cloud VMs or Platform as a cloud solutions. Will perform worse in a containerised solution. Will not even work as is in a services cloud architecture ( Not much research done here, but we were unable to envision moving Sakai to this scenario. So we do acknowledge that this opinion may be due to our lack knowledge / research )
- 18. Credit: Photo by Environment and Climate Change Canada PHASE 3: WHAT DECISIONS DO WE NEED TO MAKE? Cloud Decision Making Criteria
- 19. Credit: Photo by Sofitel Montreal Golden Mile Cloud Decision Making Criteria Knowing what you wish to achieve by moving to the cloud is critical. ▪ Stability? ▪ Cost? ▪ Performance? ▪ Redundancy? ▪ Scalability? Any single factor can be enhanced by moving to the cloud, but when we try to incorporate multiple factors into one solution. The complexity and cost increase as well. Poorly written software will not be made inherently stable by going to the cloud. Poorly performant software will not magically perform x times better simply by going to the cloud. If your solution is built on resource intensive application server, your costs will not decrease by moving to the cloud. It may in fact increase as you are now able to unlock vertical scaling not present in physical hardware without going with a rack based virtual server solution. Think petabytes of data and terabytes of Memory. Again if your software solution cannot take advantage of these resources then the solution will not bring any benefit despite the costs
- 20. Decision Making Criteria - Single Instance
- 21. Decision Making Criteria - Clustered
- 22. Decision Making Criteria - Clustered High Availability
- 23. Credit: Photo by Sofitel Montreal Golden Mile Cloud Decision Making Criteria Because Sakai is not a store bought product or purpose built for a particular cloud solution it needs to be modified somewhat to work in these new environments. VM/ PaaS offers the simplest path to getting into the cloud. Even with this method, monitoring will be crucial and new monitoring metrics will need to be measured. Metrics such as: ▪ Geographic performance ( access from various campuses and key locations ) ▪ Network(s) performance ( Data center network performance ) ▪ Cost ( Where automatic up or down scaling occurs, costs are variable and as such need to be monitored to avoid shocking invoices ) ▪ Scaling up and down ( monitor when the system scales up and down. Ensuring that it is performing correctly and tuned finely. ) ▪ Synchronization ( Data and services will need to monitored for synchronization. If database synchronization or network time goes out of sync, issues that are very difficult to isolate, troubleshoot and resolve can arise )
- 24. Credit: Photo by Sofitel Montreal Golden Mile Cloud Decision Making Criteria Metrics such as, cont.: ▪ Tool performance ( Each tool that is a depency for Sakai will need to be monitored as these tools present a point of failure for all of sakai ) ▪ Service Monitoring ( Similar to the tools, services that are a depency for Sakai will need to be monitored as well. They are even more critical for the smooth performance of Sakai )
- 25. Credit: Photo by Olivier Bousquet PHASE 4: HOW DO WE MAKE THE MOVE?
- 26. How do we make the move? The audit will also be a good reference in the future. At the end of Phase 3 you will have the beginnings of : ▪ A list of software/services/systems that will need to move to the cloud. ▪ A list of network configurations that will need to be configured on your firewall. ▪ A list of resources required in the cloud. ▪ A list of systems which will need to be changed to take the absence of software/ services / systems from the local environment.
- 27. How do we make the move? Determine how your architecture will compensate for these changes to ensure that all required functionality is still catered for. ( direction connections on non-standard ports "BAD", building web services "Good" ) things like adding: ▪ Single Sign On ▪ Portals ▪ Replication of Data/ Configurations / services between campus and cloud if necessary. ▪ A high level list of work to be done, before making the move. ▪ Migrating actual data from Datacenter to cloud. ▪ A list of vulnerabilities
- 28. Credit: Photo by Tourisme Montréal, Stéphan Poulin PHASE 5: PREPARING TO MOVE.
- 29. Credit: Photo by Tourisme Montréal, Stéphan Poulin Preparing to move. All of these lists will need to be broken up into work items and completed. It is not a trivial task. If your choice of Cloud types requires extensive rewriting or re-architecture, then plan accordingly. For our large clients moving to the cloud this is the majority of the work and we are currently in this phase.
- 30. Credit: Photo by Tourisme Montréal, Stéphan Poulin Key Takeaways ▪ You will need to be patient. ▪ You will need to be methodical & persistent. ▪ You will need to track each item on all the lists. ▪ All items are not of the same priority, adapt as conditions prevail. ▪ Training and familiarisation of the new systems and technology will need urgent attention and should be your first priority after creating a realistic plan with forgiving time frames. ▪ You will need to resolve existing issues on your LMS or campus in order to reap performance or stability benefits. Otherwise undesired behaviour will be repeated in a more complex and difficult to troubleshoot environment. ▪ Security will need to be more restrictive and a major ongoing focus.
- 31. Credit: Photo by Loïc Romer THANK YOU! Francois Campbell francois@opencollab.co.za OPENCOLLAB
- 32. Credit: Photo by Loïc Romer
- 33. Photo Credits Thank you to the following photographers and organizations for permission to use these photos for our Open Apereo 2018 conference in Montreal, QC. ▪ “View of the Old Port of Montréal - La Grande Roue” by Loïc Romer ▪ “Mary Queen of the World Cathedral and 1000 de La Gauchetière and Sun Life buildings” by Stéphan Poulin for Tourisme Montréal ▪ “Skyline panoramique” by Stéphan Poulin for Tourisme Montréal ▪ “Notre-Dame Basilica of Montréal” by Alain Régimbald ▪ “Sofitel Montreal Golden Mile - Renoir” by Sofitel Montreal Golden Mile ▪ “Biosphère, Environment Museum” by Environment and Climate Change Canada ▪ “The multicolored balls of the Village on Sainte-Catherine.” by Susan Moss ▪ “MU, Ville-Marie, ElMac Gene Pendon, 2017” by Olivier Bousquet ▪ “Montreal skyline from Mount Royal” by Stéphan Poulin for Tourisme Montréal