Testing in Production (TiP)
0 likes300 views
The document discusses various testing methodologies for production environments, highlighting the importance of confirming build quality with production data and minimizing business impact. Techniques like blue/green deployments, canary releases, A/B testing, chaos testing, and shadowing are explained, each aimed at ensuring controlled exposure and reduction of risks during new deployments. Overall, it emphasizes the necessity of rigorous testing in production to maintain service quality and reliability.
Testing in Production (TiP)
- 1. Testing in Production (TiP) ~ Gir
- 2. There are tons of testing methodologies are available to qualify production grade build. Still, Why do we need another testing method that too in Production? ● contains impact to smaller deployment or users group ● confirms the quality of build with production data ● addresses environment specific issues, if any ● make sure monitoring, DR planning and coverage ● method to reduce, control and contain Business impact... Why?
- 3. Blue/Green How?, At any time, only one of the environments is live, with the live environment serving all production traffic. For this example, Blue is currently live and Green is idle or vice versa Covers, Blue-green deployment is a technique that reduces downtime and risk by running two identical production environments called Blue and Green. Web Server Application Server Database UI Service Application Service Database Web Server Application Server Database UI Service Application Service Database Router Users v1v2
- 4. Canary (Controlled Exposure) Web Server Application Server Database UI Service Application Service Database Web Server Application Server Database UI Service Application Service Database Router Users 10% Users 90% Users ... v1v2 How?, Roll-in new deployment in controlled manner, starting with exposing smaller percentage of user and increase exposure, if successful else rollback. For example route 10% of users to V2 and 90% of users to V1. Covers, Controlled Exposure will contain the impact with in small percentage of user group.
- 5. Canary (Traffic Shaping / Shifting) Web Server Application Server Database UI Service Application Service Database Web Server Application Server Database UI Service Application Service Database Router Users 10% Traffic 90% Traffic ... v1v2 How?, Roll-in new deployment in controlled manner, starting with exposing smaller percentage of traffic and increase exposure, if successful else rollback. For example route 10% of traffic to V2 and 90% of traffic to V1. Covers, Controlled Exposure will contain the impact with in small percentage of traffic (requests).
- 6. Canary (Gradual Release) Web Server Application Server Database UI Service Application Service Database Web Server Application Server Database UI Service Application Service Database Router Users 10% Components ... 90% Components v1v2 How?, Roll-in new deployment in controlled manner, starting with exposing smaller percentage of capacity and increase exposure, if successful else rollback. For example deploy 10% of server capacity with V2 while 90% with V1. Covers, Controlled Exposure will contain the impact with in small percentage of deployment and traffic that it intakes.
- 7. A/B Testing Web Server Application Server Database UI Service Application Service Database Web Server Application Server Database UI Service Application Service Database Router Users 50% Users 50% Users ... 1b1a Conversion Rate 20% Conversion Rate 65% How?, Deploy 2 versions of UI services, side by side on infrastructure at equal capacity and direct 50% of users traffic to Version 1a while remaining to 1b and measure the acceptance rate to choose the best. Its implementation of statistics “two-sample hypothesis testing”. Covers, Helps to identify one of 2 UI or Applications with higher user conversion rate.
- 8. Chaos Testing Web Server Application Server Database UI Service Application Service Database Web Server Application Server Database UI Service Application Service Database Router Users ... v2v2 Web Server Application Server Database UI Service Application Service Database ... v2 How?, Failure is unavoidable, why not deliberately introduce it to ensure your systems and processes can deal with the failure? Can be done by breaking components, services, data or communication channel randomly. Covers, Helps to identify failure impacts at services and components level and plan for DR policy
- 9. Shadowing Pre-Production Web Server Application Server Database UI Service Application Service Database Web Server Application Server Database UI Service Application Service Database Router Users v1v2 Quality Check shadow traffic How?, Deploy production parallel application or services on shadow environment with new build (v2) and wiretap production data traffic, copy into shadow service to simulate and qualify. Covers, Qualify a build with production parallel data.
- 10. AnswersQuestions