Gatekeeper: API gateway
Download as PPTX, PDF7 likes1,035 views
The document discusses the architecture and management of a microservices-based system using Kubernetes and Kong as an API gateway. It highlights the complexities involved in deploying and scaling services, as well as the advantages and challenges of using containers and orchestration tools. Key points include the importance of health monitoring, logging, and deployment strategies to ensure robust service operations.
Gatekeeper: API gateway
- 2. Speaker @GregWeng First languages: Haskell, ECMAScript Second languages: Python, Ruby, C/C++ Domain:Functional Programming, Compiler, Dynamic Language Engine, Distributed System (as an user), Web FrontEnd Natural Language: Chinese, English, Japanese 勉強中 Slides:
- 3. Microservice
- 4. Microservices Frontend app User Backing Recipe Service Recipe/Ingredient/Rating RESTful rate limiting oauth2, ... 3rd-party service Search RESTful
- 5. Microservices Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User rate limiting oauth2, ... Backing Recipe Service Search RESTful 3rd-party service
- 6. Microservices Frontend app User Backing Recipe Service Ingredient Recipe Rating Microservices should be easy to scale out...
- 7. Microservices Frontend app User Backing Recipe Service Ingredient Recipe Rating And recover from disaster w/o crashing other instances or services as well
- 8. Microservices Frontend app User Backing Recipe Service Ingredient Recipe Rating More important: auxiliary services soon grow to help the operation
- 9. Microservices Frontend app User Backing Recipe Service Ingredient Recipe Rating Health Monitor Logging Load Balancer Deployment CI/CD
- 10. Microservices Frontend app User Backing Recipe Service Ingredient Recipe Rating Health Monitor Logging Load Balancer And the system get complicated soon... Deployment CI/CD
- 11. Microservices Frontend app User Backing Recipe Service Ingredient Recipe Rating Logging Balancer Logging Logging Logging Health Monitor And the system get complicated soon... Deployment CI/CD CI/CD CI/CD
- 14. Container For development & deployment (unit) Deployment CI/CD CI/CD CI/CD Ingredient Recipe Rating Logging Balancer Logging Logging Logging Health Monitor IngredientRecipe Rating Deployment Container (Docker)
- 15. Container Container (Docker) File System Network Process OS Kernel File System Network Process File System Network Process Process Rating Service Ingredient Service Native Processes Hardware
- 16. Container Container (Docker) CentOS 172.12.56.124 python rating.py Core OS Kernel Debian 192.168.21.24 node ingredient.js CoreOS 10.35.23.88 JVM nginx Rating Service Ingredient Service Native Processes Hardware
- 17. Container Virtual Machine 172.12.56.124 python rating.py 192.168.21.24 node ingredient.js Windows 10.35.23.88 JVM nginx Rating Service Ingredient Service Native Processes AMD-V/VT-x/AVIC/IO MMU Virtualization... CentOS Windows Kernel Debian
- 18. Container Containers for servers and services Hardware Guest OS Host OSGuest OS Container Service
- 19. Container Containers for servers and services Physical Servers Service
- 20. Container Nodes ~= Virtual Machine/Physical Machine Containers for servers and services
- 21. Container Develop & deploy a Dockerized service Nodes Write code Write Docker file Build Docker image Deploy image to nodes
- 22. Container Develop & deploy a Dockerized service Nodes Write code Write Docker file Build Docker image Deploy image to nodes Install code & dependencies into the image Instructions to build the image (many details here, actually) For your service
- 23. Container Update a Dockerized service Nodes Patch the code or Dockerfile Re-build Docker image Re-deploy the image (much more details here)
- 24. Container Why Docker/Container? Lightweight than VM (runtime overhead & image size) but still keep isolation among services* Hardware Host OS File System Network Process Kernel Process Network Process File System Guest OS Process * Isolation enough for most of cases, but can be leaky sometimes
- 25. Container Why Docker/Container? Build your own images based on others' work Rating Service Image Python3 Image Debian Image based on based on ## In the Dockerfile ## FROM python:3.6.3-jessie COPY ./app /opt/app WORKDIR /opt/app ...
- 26. Container Production Why Docker/Container? Staging CILocal Easy to deploy everywhere w/ exactly the same dependencies encapsulated rating.py virtualenv Debian packages static content environment variables external volumes
- 27. Container Why Docker/Container? Populate container supported by Kubernetes
- 28. Kubernetes
- 29. Kubernetes Kubernetes* Ingredient Recipe Rating Deployment Balancer Health Monitor *well, not best for all, but enough for most of cases
- 30. Kubernetes Microservice or microhazards? Any Container/Node/Network may fail
- 31. Kubernetes How to automatically deploy 1 image to 1000 nodes? ...... While make sure some deployment failures won't disturb the whole procedure
- 32. Kubernetes And how to upgrade to new version of image not to interrupt the service? ...... Plus the capability to auto-rollback when new version actually has regression
- 33. Kubernetes Annoying service config and secrets... Database: table, username/password Service port, host Logging level, ... Database: table, username/password Service port, host Logging level, ... Database: table, username/password Service port, host Logging level, ... Service A Service B Service C Where to put, keep, apply and update?
- 34. Kubernetes Any map for exposing service to public, or connect containers as internal services for each other? Public
- 35. Kubernetes Kubernetes can help to solve these issues
- 36. Kubernetes Different types of resources in Kubernetes Deployment Job Service Ingress ... Define these files and command Kubernetes to create/delete/update on Nodes
- 37. Kubernetes Tiny Kubernetes-based microservice in brief Containers Pod Containers Pod Containers Pod Containers Pod replica = 3 label: app=rating label: app=recipe Rating Service selector: app=rating Service Recipe Service selector: app=recipe Service /rating: Rating Service /recipe: Recipe Service Ingress Backing Recipe Ingress Public
- 38. Kubernetes Create a Deployment in Kubernetes Containers PodName Label Annotation Port restartPolicy imagePullPolicy readinessProbe livenessProbe env image command ... kubectl create -f rating-deployment.yml rating-deployment.yml Kubernetes will start the deployment and pull the image Build & push Docker image to registry
- 39. Kubernetes Update a Deployment in Kubernetes Containers PodName Label Annotation Port restartPolicy imagePullPolicy readinessProbe livenessProbe env image' command ... kubectl replace -f rating-deployment.yml rating-deployment.yml Kubernetes will start to roll out the new version *can be done w/o file updated: `kubectl set image deployment/rating rating=rating:new-version`
- 40. Kubernetes Create a service in Kubernetes kubectl create -f rating-service.yml Kubernetes will create the service and set rules to connect selected pods Name Label Annotation Type Port Selector How to expose the service (and balancing) What is the service port & Pods port Which Pods should be the backend Containers Pod Rating Service selector: app=rating Service label: app=rating *actually Service can be very powerful & complicated. Please refer the official k8s document
- 41. Kubernetes Load Balancing in Kubernetes* kube-proxy kube-proxy kube-proxy Service rating Pods Service recipe Pods Node A Node B Node C Load Balancer (Provider or External) *can be very detailed (ex: userspace proxy or iptable based) and have different ways to set up; reference: slides from KubeConf EU 2016
- 42. Kubernetes How Kubernetes detect unhealthy Pod? Containers Pod readinessProbe livenessProbe rating-deployment.yml httpGet: path: / port: 8001 initialDelaySeconds: 30 periodSeconds: 180 Kubernetes detected it not alive after the container get ready respect restartPolicy to restart the container *For multi-container Pod, the overall Pod status depends on the restartPolicy and container state
- 43. SQL connection Kubernetes Why multi-container Pod? GCloud SQL Proxy Pod: api-gateway Gateway Kong Google Cloud SQL Proxied connection External service GCloud SQL secrets Mounted secrets Raw username/ password
- 44. Mounted shared Kubernetes Why multi-container Pod? File Uploader Pod: recipe Recipe Web Server Volume Service FrontEnd App Web port File uploading port
- 45. Kubernetes Regression test for microservice (the most low-tech way) test-job.yml app test-db Job Pod HTTP client script Test Container Service Containers trigger job collect result & kill the pod create job resource
- 46. Kubernetes And test it locally before push to production Minikube on local console (local kubernetes) Kubernets platform for Production https://github.com/kubernetes/minikube
- 47. Frontend app User So now we have the infrastructure to manage our service Ingredient Recipe Rating Logging Balancer Logging Logging Logging Health Monitor Deployment CI/CD CI/CD CI/CD Kubernetes
- 48. Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User But we still left some complexity for different endpoint for requests Search RESTful Kong
- 49. API Gateway
- 50. Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User Complexity from request management and spec Search RESTful Kong rate limiting oauth2, ... different endpoints for different services
- 51. We want to manage all request & feature at once Kong Add and manage rate limiting, oauth2, logging, IP whitelist... easily RESTfulRESTfulRESTful RESTful
- 52. But to manage them on each service is just stockpiling the complexity from management Kong
- 53. Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User And front-end hopes manage only few endpoints rather than micro-100 different services Search RESTful Kong
- 54. Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User Then we have Kong as API Gateway Search RESTful Kong API gateway By Mashape
- 55. Kong Kong is... Kong core.lua OpenResty lua modules Nginx Plugin lua modules Admin API API CRUD Plugin CRUD API endpoints RESTfulRESTfulRESTful "Proxy" Interface Consumer CRUD User Postgres/ Cassandra Operator
- 56. Kong Kong is... Admin API API endpoints RESTfulRESTfulRESTful "Proxy" Interface User Postgres/ Cassandra OperatorResource CRUD plugin states Rate Limiting OAuth 2.0 Logging plugins CORSACL Bot Detection JWT Datadog AWS Lambda (and more) +plugins
- 57. Kong Kong is... Admin API API endpoints RESTfulRESTfulRESTful "Proxy" Interface User Postgres/ Cassandra OperatorResource CRUD Kong Kong Kong Kong Kong Kong Kong cluster states cluster
- 58. Kong is... External Load Balancer ex: from Kubernetes Kong Kong Kong Frontend app User Internal Upstream Load Balancer (DNS record based) service.upstream.domain 192.168.0.1 192.168.0.2 192.168.0.3 ...
- 59. How plugin works diagram from: https://github.com/openresty/lua-nginx-module Kong handler.lua nginx-kong.conf access_by_lua_block rewrite_by_lua_block header_filter_by_lua ... plugin :rewrite :access :header_filter ... iterate each plugin (defined priority)
- 60. How plugin works Kong handler.lua nginx-kong.conf access_by_lua_block rewrite_by_lua_block header_filter_by_lua ... plugin :rewrite :access :header_filter ... iterate each plugin (defined priority) Kong dao plugin daos.lua migrations Postgres/ Cassandra
- 61. Kong Kong Kong Kong Kong Kong Kong Kong & database migration Postgres/ Cassandra Kong migration up
- 62. Kong Kong Kong Kong Kong Kong Kong Kong & database migration Postgres/ Cassandra Kong migration up Kong Kong Kong Kong Kong Kong Postgres/ Cassandra Kong migration up Kong migration up Kong migration up Kong migration up Kong migration up Kong migration up Kong migration up
- 63. Kong Combine all the things together Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User Search RESTful API gateway Deployment Balancer Health Monitor Logging CI/CD Postgres Cassandra Operator Admin API Endpoints +plugins
- 64. Advantages 1. Nginx is fast, therefore Kong is fast 2. Built-in plugins are good 3. Plugin system make customisation easy and clear 4. OpenResty is a great module collection 5. Docker ready, and also for other cloud formats, like AMI 6. Admin interface is also RESTful: it is possible to set up in pure cURL 7. Lua is easy to learn, and it is also friendly to bind C code 8. All the benefits we want for manage multiple RESTful microservices 9. Mashape provides enterprise support Kong
- 65. Issues 1. Migration racing issue (Job + Deployment in k8s) 2. Plugins are not dynamic loaded 3. No resource limiting for individual plugin 4. Kong is not multi-tenant 5. No official web dashboard 6. Multiple configuration for Nginx and Kong 7. Restriction from the underlying Nginx & OpenResty 8. Lua is good, but not so good (also, hiring problem) 9. Cannot only upgrade Nginx for security reason means to re-build whole the whole stack (maybe possible in embedded mode) Kong
- 66. Recap
- 67. Microservices Frontend app User Pre-microservice Recipe/Ingredient/Rating RESTful rate limiting oauth2, ... 3rd-party service Search RESTful
- 68. Microservices Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User rate limiting oauth2, ... Microservice Search RESTful 3rd-party service
- 69. Microservices Frontend app User Problem: complexity Ingredient Recipe Rating Logging Balancer Logging Logging Logging Health Monitor Deployment CI/CD CI/CD CI/CD
- 70. Container For development & deployment (unit) Deployment CI/CD CI/CD CI/CD Ingredient Recipe Rating Logging Balancer Logging Logging Logging Health Monitor IngredientRecipe Rating Deployment Container (Docker)
- 71. Container Develop & deploy a Dockerized service Nodes Write code Write Docker file Build Docker image Deploy image to nodes Install code & dependencies into the image Instructions to build the image (many details here, actually) For your service
- 72. Container Why Docker/Container? Lightweight than VM (runtime overhead & image size) but still keep isolation among services* Hardware Host OS File System Network Process Kernel Process Network Process File System Guest OS Process * Isolation enough for most of cases, but can be leaky sometimes
- 73. Container Production Why Docker/Container? Staging CILocal Easy to deploy everywhere w/ exactly the same dependencies encapsulated rating.py virtualenv Debian packages static content environment variables external volumes
- 74. Kubernetes Kubernetes* Ingredient Recipe Rating Deployment Balancer Health Monitor *well, not best for all, but enough for most of cases
- 75. Kubernetes Different types of resources in Kubernetes Deployment Job Service Ingress ... Define these files and command Kubernetes to create/delete/update on Nodes
- 76. Kubernetes Tiny Kubernetes-based microservice in brief Containers Pod Containers Pod Containers Pod Containers Pod replica = 3 label: app=rating label: app=recipe Rating Service selector: app=rating Service Recipe Service selector: app=recipe Service /rating: Rating Service /recipe: Recipe Service Ingress Backing Recipe Ingress Public
- 77. Kubernetes Create a Deployment in Kubernetes Containers PodName Label Annotation Port restartPolicy imagePullPolicy readinessProbe livenessProbe env image command ... kubectl create -f rating-deployment.yml rating-deployment.yml Kubernetes will start the deployment and pull the image Build & push Docker image to registry
- 78. Frontend app User So now we have the infrastructure to manage our service Ingredient Recipe Rating Logging Balancer Logging Logging Logging Health Monitor Deployment CI/CD CI/CD CI/CD Kubernetes
- 79. Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User Complexity from request management and spec Search RESTful Kong rate limiting oauth2, ... different endpoints for different services
- 80. Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User Then we have Kong as API Gateway Search RESTful Kong API gateway By Mashape
- 81. Kong Kong is... Kong core.lua OpenResty lua modules Nginx Plugin lua modules Admin API API CRUD Plugin CRUD API endpoints RESTfulRESTfulRESTful "Proxy" Interface Consumer CRUD User Postgres/ Cassandra Operator
- 82. Kong Kong is... Admin API API endpoints RESTfulRESTfulRESTful "Proxy" Interface User Postgres/ Cassandra OperatorResource CRUD plugin states Rate Limiting OAuth 2.0 Logging plugins CORSACL Bot Detection JWT Datadog AWS Lambda (and more) +plugins
- 83. Kong Kong is... Admin API API endpoints RESTfulRESTfulRESTful "Proxy" Interface User Postgres/ Cassandra OperatorResource CRUD Kong Kong Kong Kong Kong Kong Kong cluster states cluster
- 84. Kong Combine all the things together Recipe RESTful Ingredient Rating RESTfulRESTful Frontend app User Search RESTful API gateway Deployment Balancer Health Monitor Logging CI/CD Postgres Cassandra Operator Admin API Endpoints +plugins
- 85. Advantages 1. Nginx is fast, therefore Kong is fast 2. Built-in plugins are good 3. Plugin system make customisation easy and clear 4. OpenResty is a great module collection 5. Docker ready, and also for other cloud formats, like AMI 6. Admin interface is also RESTful: it is possible to set up in pure cURL 7. Lua is easy to learn, and it is also friendly to bind C code 8. All the benefits we want for manage multiple RESTful microservices 9. Mashape provides enterprise support Kong
- 86. Issues 1. Migration racing issue (Job + Deployment in k8s) 2. Plugins are not dynamic loaded 3. No resource limiting for individual plugin 4. Kong is not multi-tenant 5. No official web dashboard 6. Multiple configuration for Nginx and Kong 7. Restriction from the underlying Nginx & OpenResty 8. Lua is good, but not so good (also, hiring problem) 9. Cannot only upgrade Nginx for security reason means to re-build whole the whole stack (maybe possible in embedded mode) Kong
- 87. Advantages 1. Nginx is fast, therefore Kong is fast 2. Built-in plugins are good 3. Plugin system make customisation easy and clear 4. OpenResty is a great module collection 5. Docker ready, and also for other cloud formats, like AMI 6. Admin interface is also RESTful: it is possible to set up in pure cURL 7. Lua is easy to learn, and it is also friendly to bind C code 8. All the benefits we want for manage multiple RESTful microservices 9. Mashape provides enterprise support Kong
Editor's Notes
- #4: JS -> Bytecode -> x64/x86 asm A simple example Retrval is just because we make this not in any assignment. If put it as assignment it's okay.
- #13: JS -> Bytecode -> x64/x86 asm A simple example Retrval is just because we make this not in any assignment. If put it as assignment it's okay.
- #25: Isolation enough for most of cases, but can be leaky sometimes
- #29: JS -> Bytecode -> x64/x86 asm A simple example Retrval is just because we make this not in any assignment. If put it as assignment it's okay.
- #50: JS -> Bytecode -> x64/x86 asm A simple example Retrval is just because we make this not in any assignment. If put it as assignment it's okay.
- #73: Isolation enough for most of cases, but can be leaky sometimes