Building a scalable learning platform - Erik Veld - Codemotion Amsterdam 2018
0 likes186 views
Instruqt is a scalable learning platform that offers hands-on DevOps and cloud challenges, allowing users to learn practical skills using real infrastructure. Built on Google Cloud Platform with Terraform and Golang, the platform utilizes Kubernetes for resilience and scalability while managing environments dynamically. Key insights include leveraging existing technologies for efficiency and the ease of migrating cloud providers.
![# The volume is mounted in the pod from a git repository.
volume {
name = "bootstrap"
git_repo {
repository = "https://url-to-git-repo/bootstrap.git"
}
}
volume_mount {
name = "bootstrap"
mount_path = "/opt/bootstrap"
}
# Wait until the volume is mounted, then execute the setup script.
args = ["until [ -d /opt/bootstrap/ ]; do sleep 1; done;
/opt/bootstrap/setup.sh"]
36instruqt](https://image.slidesharecdn.com/buildingascalablelearningplatform-codemotionamsterdam-erikveld-180529155502/85/Building-a-scalable-learning-platform-Erik-Veld-Codemotion-Amsterdam-2018-36-320.jpg)
Building a scalable learning platform - Erik Veld - Codemotion Amsterdam 2018
- 1. Building a scalable learning platform “how to make scaling the cloud’s problem” instruqt
- 2. I like building and learning new things, and sharing knowledge with others. ERIK VELD 2instruqt
- 3. Founder of instruqt, focussing mostly on the technical side. ERIK VELD 3instruqt
- 5. Hands-on challenges that build skills on DevOps and cloud. WHAT IS INSTRUQT 5instruqt
- 6. Each user gets real infrastructure to complete the challenges in. WHAT IS INSTRUQT 6instruqt
- 7. Solutions are validated against the user infrastructure. WHAT IS INSTRUQT 7instruqt
- 8. Enough talking, demo time. 8instruqt
- 10. HOW IT STARTED 10 2016 2017 2018 HashiConf EU instruqt
- 11. 11instruqt
- 12. 12instruqt
- 13. HOW IT STARTED 13 2016 2017 2018 HashiConf EU HashiConf US instruqt
- 14. 14instruqt
- 15. HOW IT STARTED 15 2016 2017 2018 HashiConf EU HashiConf US Google Cloud Next instruqt
- 16. 16instruqt
- 17. HOW IT STARTED 17 2016 2017 2018 HashiConf EU HashiConf US Google Cloud Next Current iteration instruqt
- 18. How did we build this platform? 18instruqt
- 19. HOW IT IS BUILT 19 Google Cloud Platform Terraform Golang instruqt
- 20. Entirely on Google Kubernetes Engine. GOOGLE CLOUD PLATFORM 20instruqt
- 21. GKE gives us resilience, isolation, service discovery and scaling for free. GOOGLE CLOUD PLATFORM 21instruqt
- 22. Everything as a service from Google. GOOGLE CLOUD PLATFORM 22instruqt
- 23. GOOGLE CLOUD PLATFORM 23 more Container Registry Datastore Kubernetes Engine Cloud SQL Cloud PubSub Cloud Storage Compute Engine Cloud Functions Firebase instruqt
- 24. GOOGLE CLOUD PLATFORM 24 more Container Registry Datastore Kubernetes Engine Cloud SQL Cloud PubSub Cloud Storage Compute Engine Cloud Functions Firebase instruqt
- 25. GOOGLE CLOUD PLATFORM 25 more Container Registry Datastore Kubernetes Engine Cloud SQL Cloud PubSub Cloud Storage Compute Engine Cloud Functions Firebase instruqt
- 26. Terraform manages our platform and all of the services we use. TERRAFORM 26instruqt
- 27. TERRAFORM 27 Describe what the desired state is, don’t worry about how it’s achieved. instruqt
- 28. GOLANG 28 With the small footprint and fast start time we scale in seconds not minutes. instruqt
- 29. GOLANG 29 With many interdependencies, a monorepo solves a lot of problems. instruqt
- 31. Let’s take a look behind the scenes. 31instruqt
- 32. Terraform creates environments on the fly, inserting variables where needed. BEHIND THE SCENES 32instruqt
- 34. BEHIND THE SCENES 34 > PODSSERVICES CONFIG SECRETS PROXY TRACK NAMESPACE OTHER INFRA instruqt
- 35. Any container will work, platform dependencies are injected at runtime. BEHIND THE SCENES 35instruqt
- 36. # The volume is mounted in the pod from a git repository. volume { name = "bootstrap" git_repo { repository = "https://url-to-git-repo/bootstrap.git" } } volume_mount { name = "bootstrap" mount_path = "/opt/bootstrap" } # Wait until the volume is mounted, then execute the setup script. args = ["until [ -d /opt/bootstrap/ ]; do sleep 1; done; /opt/bootstrap/setup.sh"] 36instruqt
- 37. Because terraform stores the state of each user environment, cleanup is easy. BEHIND THE SCENES 37instruqt
- 39. How does terraform know what to build? 39instruqt
- 40. The SDK creates a skeleton, which is then filled out and sent to the pipeline. TRACK PIPELINE 40instruqt
- 42. Golang templates generate terraform code that creates user environments. TRACK PIPELINE 42instruqt
- 43. # The variables that need to go into the template. data := struct { Containers Container }{ … } # Create the destination file. f, err := os.Create(outputPath) # Parse the templates. t := template.New(templateName) _, err = t.ParseFiles(inputPath) # Execute the template with the data. for _, tmpl := range t.Templates() { tmpl.ExecuteTemplate(f, tmpl.Name(), data) } 43instruqt
- 44. # Loop over the containers in the config.yml file. {{range .Containers}} resource "kubernetes_replication_controller" "{{.Name}}" { spec { template { container { name = "{{.Name}}" image = "{{.Image}}" } # Etc. } } } {{- end}} 44instruqt
- 46. Because of Container Builder we do not have to run a build server anymore. TRACK PIPELINE 46instruqt
- 48. The platform and build pipeline are scalable. The challenge is scaling ourselves. LESSONS LEARNED 48instruqt
- 49. Stand on the shoulders of giants and use this to focus on your product. 49 LESSONS LEARNED instruqt
- 50. Migrating to a different cloud provider is not as much work as you might think. LESSONS LEARNED 50instruqt