ABTesting: Não há nada como testar em produção!

Notas do Editor

• #11: Today, every company is a software company, “whether you know it or not, you are almost certainly in the software business.” Software runs the critical infrastructure of nearly every modern business, and for the best companies, it provides a key competitive differentiator. As companies start embracing agility and taking advantage of the elasticity and economics of consuming cloud services the complexity increases, creating more points of failures and making more complicated to ensure those applications that run critical parts of your company and engage with your customer are optimally working and when they don’t you can restore a good state as soon as possible. So how do you increase responsiveness without creating unnecessary risk? How can you stay flexible and still keep a strategy and goals in view? How do you juggle agility with all of the other new demands on the success of your business? And ensure you continue investing in solutions that will give you a differentiator advantage?
• #20: Release Management Server provides valuable insights into the entire release process. Armed with relevant and timely information, managers can achieve greater efficiencies by monitoring for continuous improvement. For example, you can keep track of the trend in releases, the amount of traffic in the release pipeline, any possible bottlenecks, and even get performance insights.
• #35: Release Management Server provides granular control for the entire release process that includes many stakeholders, including Development, QA and Ops. You can limit which group can view, edit, or create a given release template. You can also decide who can edit approvers for each release stage. For example, the development group can modify the approvers in the Dev stage but cannot modify approvers for the Production stage. Another example is how you can limit which group can edit the configuration values for each release stage. For example, the development group can modify the values in the Dev and Integration stages but cannot modify Production. Finally, you can limit the stage types a given environment can support. For example, you can limit which production servers can be used.
• #39: Nov: restated # TFS tickets. Team did problem management and broke down general LSI tickets to specific events Availability model: Outside in views not shown due to noise in current tests. Outside of our control – customers network bandwidth, working towards measuring to the Time to First byte sent Vs., total time. Total time is usually bad if customer network bandwidth is poor AI Availability Model: Multi-step synthetic transaction test covers more surface area, includes more steps and is more sensitive to customer experience. 10 E2E scenario tests cover full surface area, new model aggregates availability across all these tests. Example sensitive measures: Single PoP node failures, data latency issues all count against availability model – these were not part of Gomez before. Availability model continues to be refined as new scenarios are uncovered.
• #40: ELS Active accounts based on 60 days activities are: Sep-13 to Jan 14: 194, 263, 343, 400, 459
• #42: For the success of features we’re investing in a “Build, Measure, Learn” culture. <talk about BML> Let me give you a recent example…
• #56: Time: 2 minutes Flow: We have had an end to end ALM solution for many years with Team Foundation Server on-premises which is built on a core set of capabilities to help development teams: Source control Agile planning tools Team rooms Test case management Feedback management and Automated builds Layered on top of that core ALM functionality, Team Foundation Server has the ability to Manage, deploy to, and test on environments with Lab Management Configure and manage releases with Release Manager Perform on premises load testing Integrate completely with SharePoint and customized SharePoint sites Integrate seamlessly with Project Server and System Center We are now introducing Visual Studio Online which builds on top of that core ALM functionality as well and provides additional capabilities and unique services like: Cloud Build Service Cloud Load Testing Service Application Insights Continuous deployment to Windows Azure Lightweight Code Editing Experience Microsoft’s ALM toolset is flexible and allows you to adopt a mixture of on-premises or cloud-based services. You can even use TFS on-premises and use services on Visual Studio Online that make sense to your team like the Load Testing Service and Application Insights. Transition to next slide: We’ll explore these services and more in depth throughout the rest of this presentation.