1,2,3 … Testing : Is this thing on(line)? with Mike Martin
Download as PPTX, PDF0 likes122 views
The document details various Microsoft testing tools and services, including Playwright for web automation, Azure Load Testing for performance testing, and Azure Chaos Studio for fault injection to enhance application resilience. It also highlights tools like Microsoft Clarity for user behavior analysis and PsRule for Azure to align with best practices in infrastructure as code. Additional insights into integrating testing into CI/CD workflows and accessibility features are provided, emphasizing the importance of continuous feedback and optimization in software development.
![Experiment Json Example {
"location": "eastus2euap",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"steps": [
{
"name": "Step1",
"branches": [
{
"name": "Branch1",
"actions": [
{
"type": "continuous",
"name": "urn:provider:agent-v2:Microsoft.Azure.Chaos.Fault.CPUPressureAllProcessors",
"parameters": [
{
"key": "PressureLevel",
"value": "90"
}
],
"duration": "PT5M",
"selectorid": "Selector1"
}
]
}
]
}
],
"selectors": [
{
"id": "Selector1",
"type": "List",
"targets": [
{
"type": "ResourceId",
"id": "/subscriptions/42707c56-5cd5-4343-82b5-
8c63c64def11/resourceGroups/ChaosDemo/providers/Microsoft.Compute/virtualMachines/demoVM1/Microsoft.Chaos/chaosAgents/chaosAgent"
},
{
"type": "ResourceId",
"id": "/subscriptions/42707c56-5cd5-4343-82b5-
8c63c64def11/resourceGroups/ChaosDemo/providers/Microsoft.Compute/virtualMachines/demoVM2/Microsoft.Chaos/chaosAgents/chaosAgent"
}
]
}
]
}
}
Action or Fault to be preformed
Target or Azure Resources that will be
targeted
Selector containing a set of targets
Selector where these actions are to be
preformed
Step that will be preformed
Branch in this step preformed](https://image.slidesharecdn.com/123testsuisse-230323190642-032633dc/85/1-2-3-Testing-Is-this-thing-on-line-with-Mike-Martin-24-320.jpg)
1,2,3 … Testing : Is this thing on(line)? with Mike Martin
- 2. 1, 2, 3… testing: Is this thing on(line)? Meet your new Microsoft Testing tools
- 3. Thursday,March 23, 2023 19:06 Mike Martin Technical Evangelist #Azure Ex-MVP, Advisor, User group Social • @techmike2kx • mimar@microsoft.com //what • Flexible • Passionate about tech • Family • Global reach • Knowledge sharing //also • curious • Straightforward • Feedback eager!
- 4. So what is this about?
- 5. One word : FEEDBACK
- 6. Actually more like 4 : FINO FAILURE IS NO OPTION
- 8. Life’s a stage ... You have to Playwright …! https://playwright.dev
- 9. What is Playwright? • Automation library • automate Chromium, Firefox, and WebKit with a single API • evergreen, capable, reliable, and fast cross-browser web automation • Using websockets and no browser hook-in • So benefitting the devtools directly
- 10. Visual Studio Code Extension
- 11. trace
- 12. report
- 14. Tryouts ….. And general rehearsal Azure Load Testing
- 15. Generate high-scale load with ease Generate high-scale load without the need for complex infrastructure Run existing test scripts with high-fidelity JMeter support Eliminate infrastructure needs with a fully managed service Experience frictionless testing on Azure
- 16. Optimize performance at scale Catch and identify app performance issues at scale Generate deep, actionable insights and recommendations Understand how tests impact all parts of your app Compare across load tests to understand change over time
- 17. Build load testing into DevOps workflows Integrate load testing into existing CI/CD workflows Shift load testing left to continually improve performance Establish a performance baseline and set clear pass/fail criteria Automatically export test results to GitHub with full context Actions based on test result Build and Deploy CI CD Workflow Source code Azure Load Testing Service Load testing Pass/Fail workflow based on test result Test plan configuration Test results
- 18. Get started quickly with a fully managed service optimized for Azure Streamline testing with Azure- specific networking patterns Protect Azure workloads with automated throttling detection Test efficiently with a service optimized for Azure
- 19. Break a leg!
- 20. Fully managed service for validating Azure application and service resilience Deep Azure integration, including Azure Portal, Azure Resource Manager, and Azure Monitor Expandable library of common resource pressure and service dependency disruption faults Advanced workflow orchestration of manual and automated fault injection experiments that simulate real-world scenarios with parallel and sequential actions Safeguards to minimize impact radius and to stop and roll back experiments Azure Chaos Studio Measure, understand, improve, and maintain product resilience Your Azure Environment Fault Library Portal UI REST API Experiments Chaos Resource Providers Your Service Service infrastructure VM | VMSS | AKS Chaos agent Service-Direct Agent Service Dependencies
- 21. Fault Library Expanding library of faults across the entire Azure stack • Current investments in Guest OS (agent-based) and Azure resource (service-direct) fault types. • Windows and Linux agent support • Destructive, pressure, latency, and configuration change faults • CPU pressure Agent-based Service-direct • CPU pressure • Physical memory pressure • Virtual memory pressure • Disk I/O pressure • Stress-ng stress • Stop service • Change time • Kill process • DNS failure • Network latency • Network disconnect • Network firewall disconnect • VM shutdown or kill • VMSS shutdown or kill • Cosmos DB failover • Azure Cache for Redis reboot • Network security group set rule • AKS network disconnect/ packet loss/latency • AKS pod failure, container failure/kill • AKS CPU, memory pressure • AKS file I/O latency/failure • AKS DNS failure • AKS time change • AKS HTTP delay, modification
- 22. In-process Fault Injection • Perturb individual API calls • Azure SDKs • Customer code Agent-based guest OS faults • Windows and Linux VM agent support • Resource pressure (CPU, memory, disk, network) • Dependency disruption (process/service, network) Agentless (Service-direct, Mocks/Proxy) faults • Dependency disruption • Configuration changes Infrastructure Fault Injection • Disrupt the host • Used by Microsoft to improve the Azure platform Application code Guest OS Azure resource Azure infrastructure Disruptions Expanding library of faults across the entire Azure stack
- 23. Chaos Studio Experiment Experiment Metadata – Container for consisting of experiment metadata such as azure region where the test is to be deployed, and Identity to be used. Step – Top level container consisting of Branches that will be run sequentially. Branch – Second level container consisting of Actions that will be run in parallel. Action – Fault that will be executed against a Selector. Selector – A container of Targets. Target – A Azure Resource that will. {Experiment Metadata} {Step Id} {Branch Id} {Action} {Step Id} {Branch Id} {Action} {Action} {Selector Id} {Target} {Target} {Selector Id} {Target} {Target} {Selector Id} {Selector Id}
- 24. Experiment Json Example { "location": "eastus2euap", "identity": { "type": "SystemAssigned" }, "properties": { "steps": [ { "name": "Step1", "branches": [ { "name": "Branch1", "actions": [ { "type": "continuous", "name": "urn:provider:agent-v2:Microsoft.Azure.Chaos.Fault.CPUPressureAllProcessors", "parameters": [ { "key": "PressureLevel", "value": "90" } ], "duration": "PT5M", "selectorid": "Selector1" } ] } ] } ], "selectors": [ { "id": "Selector1", "type": "List", "targets": [ { "type": "ResourceId", "id": "/subscriptions/42707c56-5cd5-4343-82b5- 8c63c64def11/resourceGroups/ChaosDemo/providers/Microsoft.Compute/virtualMachines/demoVM1/Microsoft.Chaos/chaosAgents/chaosAgent" }, { "type": "ResourceId", "id": "/subscriptions/42707c56-5cd5-4343-82b5- 8c63c64def11/resourceGroups/ChaosDemo/providers/Microsoft.Compute/virtualMachines/demoVM2/Microsoft.Chaos/chaosAgents/chaosAgent" } ] } ] } } Action or Fault to be preformed Target or Azure Resources that will be targeted Selector containing a set of targets Selector where these actions are to be preformed Step that will be preformed Branch in this step preformed
- 25. It’s Showtime! Secure your seat!
- 26. What is PSRule for Azure? Tests and docs for Azure Infrastructure as Code Azure Well-Architected Framework aligned Plugs into continuous integration pipelines and DevOps mindset.
- 27. It makes it easier for you to do this code main dev
- 29. Key points 200+ releases shipped to date
- 30. Reporting in Workbooks and Log Analytics
- 31. Supply Chain Development Lifecycle Code Platform for Security Governance GitHub secures your complete software lifecycle Shift security left with Advanced Security code scanning and secret scanning
- 32. What is CodeQL? • CodeQL is an industry-leading semantic code analysis engine • It helps discover security vulnerabilities across a codebase through automation • CodeQL lets us query the code as though it were data. These queries can be shared and reused by other Engineering orgs. • Security vulnerabilities, bugs, and other errors are modeled as queries that can be executed against the code (extracted into a DB).
- 33. Devops CLI (aka Guardian vNext)
- 36. Existing today Accessibility Insights for Web Windows Android
- 38. What YOU can do: Keep accessibility in mind in every step of the process Download our tools at accessibilityinsights.io Run the tools and address issues early in your product cycle Contribute on GitHub Spread the word and spread the love! E2E
- 39. Microsoft Clarity The Applause, the laughter, the feedback….
- 40. Why use Microsoft Clarity in testing? Understand the user journey, such as the path to conversion, by replaying user sessions. Understand engagement on your pages, including clicks, scrolls and movement through heat maps. Gain insights into interesting user sessions to root out problems and analyze engagement by campaigns, channels and browsers. It’s continuous feedback in usability.
- 41. Explore user session recordings, including path to conversion after ad clicks Understand “why” when problems are discovered Reason for metric movements by watching recordings to learn what part of your site works well for your users and where your users are struggling. ➔ Optimize path to conversion. ➔ Validate product changes. ➔ Identify target audience. ➔ Identify product opportunities.
- 42. Explore heat maps to optimize ad landing pages Visualize behavior in aggregate Find out which part of your pages gets the most clicks and drives conversions to grow your key metrics across device types, browsers and interaction methods. ➔ Optimize placement of web elements. ➔ Detect drop-off areas. ➔ Show impact to regression and utilisation. ➔ Landing page testing.
- 43. Explore dashboards and metrics to optimize user experience Discover opportunities and eliminate problems Make informed and data- driven decisions from the dashboard insights and drill down to specific sessions to streamline your website and remove annoying bugs. ➔ Quick-back clicks. ➔ Rage clicks. ➔ Dead clicks. ➔ Excessive scrolling. ➔ JavaScript errors.
- 44. Additional Clarity features Free service – Use Clarity as much as you want and on as many sites as you want without being charged. Simple setup – Guides for major third-party platforms (ex. Squarespace, WordPress, Shopify, Wix, etc.). No sampling – Analyze 100% of your website traffic. Nearly real time – Get your insights in nearly real time without having to wait. Lightweight – Clarity doesn’t get in the way of your site’s performance. Compatible – Works with major web analytic tools you may already be using (ex. Google Analytics, Google Tag Manager, Adobe Tag Manager, etc.) Masking support – Protect your user privacy and mask sensitive data directly onto the client. Built for scale – Clarity can handle terabytes of data with millions of users, and with no website traffic limits. Filters – Supports custom filters and tags to present the data the way you want it presented. GDPR and CCPA compliant – Compliant with General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA).
- 45. So … is that it? Well actually …..