Testing SharePoint solutions overview
Download as PPTX, PDF4 likes6,687 views
This document discusses testing SharePoint solutions using Visual Studio 2010 tools. It provides an overview of common testing challenges and scenarios that lack testing rigor. It then demonstrates the Visual Studio 2010 testing capabilities for unit testing, load testing, coded UI testing, and lab management to help address these issues and establish a more robust testing practice. Automated testing tools are shown to integrate with the development lifecycle in Visual Studio and Team Foundation Server.
Testing SharePoint solutions overview
- 1. Testing SharePoint SolutionsOverviewErvin LohVisual Studio ALM MVPK365Labs SdnBhdLight Up SharePoint
- 2. PresenterErvin LohProfileErvin Loh is currently the Application Lifecycle Management Program Manager at K365Labs Sdn Bhd. He works on a variety of Application Lifecycle Management, Software Configuration Management, and Lab Management products. He is also active in the IT community space by contributing contents to Microsoft Malaysia's START.NET and Ultimate program workshops and talks in conferences, events and user groups.
- 3. ASP.NET Developers, SharePoint DevelopersTarget Audience
- 4. Is this how you kill bugs?
- 8. An Unfortunately Common ScenarioDeveloper writes codeDeveloper makes sure the code compilesDeveloper checks in codeRepeat x15 developers for 4 weeksDeveloper lead does a build of all code from developersApplication is installed in test environmentTester tries to test the applicationApplication doesn’t workDeveloper blames tester
- 9. The Demonstrations ScenarioThe ConfigurationsA fully configured Visual Studio 2010, Team Foundation Server 2010, and SharePoint Server 2010 environment.The TasksA lot of tests.
- 10. TopicsOverviewofthetestingtools in Visual Studio 2010Testing SharePoint projectswith Visual Studio 201010
- 11. Have you heard any of these?“my testers are spending too long testing the same thing”“tooling is expensive (time, licenses, people)”“the developers say the defects are useless”“when is my software ready to ship?”“developers and testers work in silos and don’t communicate/speak the same language”“we have a changed requirement, what do I need to test?”
- 12. What if you could…reduce the time it takes to determine the root cause of a bugreduce the time it takes to replicate a bug uncovered by user actionsreduce the time it takes to isolate differences between the test and production environmentenable users to easily run acceptance tests and track their resultsreduce the time it takes to verify the status of a reported bug fix
- 13. Where does testing happen?70% of testing happens hereMajority of test tools target hereBlack Box TestingWhite Box TestingAPI Testing
- 14. Visual Studio 2010 Test CapabilitiesLoad Test Web Test Coded UI Test Microsoft Test RunnerUnit Testing Test Case ManagementLab ManagementData Collectors (Historical Debugging, Test Impact, Metrics)Team Foundation ServerReporting
- 16. Why is fixing bugs difficult?Insufficient information to reproduceNot every step is documentedCan’t see exactly how the bug was triggeredDifferent environmentsOS, service packs, installed software,…Creating a clean environment takes time
- 17. Microsoft Test Manager 2010Plan, Manage and Execute (manual) tests from one placeCreate test casesBuild test suitesDefine configurationsRun test casesFile bugs into TFSIncluding captured dataAutomate testingIntegrated with TFS
- 18. VS Agents: Diagnostic Data AdaptersSystem InfoIntelliTrace™Video CaptureSteps PerformedSystem InfoIntelliTrace™Visual Studio Agents 2010Separate downloadTest Controller & Test AgentsCapture data while testingSave data with bug reportEasy for developer to reproduce bug
- 19. Lab Manager 2010Setup, Teardown, and Restore virtual environmentsMaintain known stateBuild automationBuild deploymentTest executionNetwork isolationRun multiple copiesConnect from VS2010Linked to bug form
- 20. 70%of all functional testing is still done manually.
- 21. DemoFunctional Testing with Microsoft Test Manager
- 22. Automated Testing for SharePointWeb Performance Testing
- 23. Load Testing
- 24. Coded UI Testing
- 25. Unit TestingWeb Performance TestingSimulate a single use caseInternet Explorer Web Testing toolbarRecords the user’s web requestsImportant: This is not a UI testValidation RulesExtraction RulesParameterized TestsData-Driven Tests
- 26. Load TestingStress-testing: simulate many users at onceNeed multiple servers to create substantial loadMultiple agents generate load on the applicationSingle controller coordinates the agents and captures dataPerfMon-style output
- 27. Coded UI TestingSimulate a single use caseUses a recording toolbar, like Web Performance testRecords the user’s interaction with the UIGenerates code to reproduce that interactionCode can be edited and customized at willHas some issues with SharePointe.g. no support for datasheetsVisual Studio Feature Pack 2 adds support for Silverlight
- 28. Unit TestingUsing built-in Unit Test with SharePoint has problems:Issues with 32-bit vs 64-bit, and .NET 3.5 vs .NET 4.0Need to stub/mock SharePoint APIe.g. SPSite, SPWeb, SPListAdditional tools enable unit testing for SharePoint projectsMicrosoft Research: Pex & MolesIntegrates with VS2010, free to MSDN subscribershttp://research.microsoft.com/en-us/projects/pex/pexsharepoint.pdfThird party: NUnit, TypeMock
- 29. DemoAutomated Testing with Microsoft Visual Studio
- 31. Are we ready to ship?Are we making progress on running test plans?How are our builds doing over time?Are we fixing bugs?What’s the quality of our bug fixes?Ready to Ship?
- 32. DemoReporting
- 35. ResourcesTest Manager 2010 and Lab Managerhttp://msdn.microsoft.com/en-us/library/ms182409.aspxhttp://msdn.microsoft.com/en-us/library/dd997438.aspxVisual Studio 2010 Agentshttp://www.microsoft.com/downloads/en/details.aspx?FamilyID=a3216d2a-0530-4f6c-a7c9-0df37c54a902Pex and Moleshttp://research.microsoft.com/en-us/projects/pex/default.aspx
- 36. Questions?
- 37. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Visual Studio, the Visual Studio logo, and [list other trademarks referenced] are trademarks of the Microsoft group of companies. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Editor's Notes
- #12: These are some of the comments that we have heard from our customers. Are any of these being said by your testers?Broken down another way, can you answer these affirmatively?Do you know consistently when your software will ship?Are the defects created by your tester truly effective?No? <click to next>
- #13: Does this sound like a wish list? How much money is lost in the time that it takes to track down a bug?How much money is lost in the time it takes to isolate the differences between your development in test environments?
- #14: It is also important to understand where most testing happens in the spectrum of general testing to the more technical specialist testing.The Generalist Testers are usually professional testers with no coding background. Often these testers are experts in the business process or tool that is being developed. On the opposite side of the spectrum is the Specialist. This is a tester with strong coding skills.A fun side note: Microsoft’s testers are usually converted developers and tend to be on the specialist side of the graph.Black-box testing is a method of testing software that tests the functionality of an application as opposed to its internal structures or workings (see white-box testing). Specific knowledge of the application's code/internal structure and programming knowledge in general is not required. Test cases are built around specifications and requirements, i.e., what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and design to derive test cases. These tests can be functional or non-functional, though usually functional. The test designer selects valid and invalid inputs and determines the correct output. There is no knowledge of the test object's internal structure.White-box testing (a.k.a. clear box testing, glass box testing, transparent box testing, or structural testing) is a method of testing software that tests internal structures or workings of an application as opposed to its functionality (black-box testing). An internal perspective of the system, as well as programming skills, are required and used to design test cases. The tester chooses inputs to exercise paths through the code and determine the appropriate outputs. It is analogous to testing nodes in a circuit, e.g. in-circuit testing (ICT). While white-box testing can be applied at the unit, integration and system levels of the software testing process, it is usually done at the unit level. It can test paths within a unit, paths between units during integration, and between subsystems during a system level test. Though this method of test design can uncover many errors or problems, it might not detect unimplemented parts of the specification or missing requirements. White-box test design techniques include: Control flow testing Data flow testing Branch testing Path testingAPI testing (application programming interface) – is a specific type of White Box testing of the application focusing on public and private APIs<Question to Audience>Looking at this spectrum, where does most testing happen today? <collect answers and click>Where do most testing tools target today? <collect answers and click>
- #15: Describe the pieces as it applies to the spectrum as the slide builds out.
- #21: Currently about 70% of functional testing is done manually—that is, a software tester follows a script to execute a series of steps to verify the outcome of a test. Microsoft Test Manager 2010 is an example of a tool that someone doing manual testing would use to be more productive. Imagine if you could still get the benefits of functional testing without the overhead and resource cost of doing in manually.Consider a regression test—a test designed to identify if a bug fix begins to fail. With a regression test (and even other functional tests) you are repeatedly testing something that was known to work at one point. These too are typically done manually. This is a massive resource hit. Software testers spend countless hours testing functionality that works, solely to ensure it still works. Imagine how much more productive they could be if they could focus their efforts on creating and running new test cases that covered parts of the system not currently tested instead of spending their time testing the same thing, over and over again.
- #29: Here are two examples <Build> of the dashboard you get when running on top of one of the full versions of SharePoint Server. First, we have the Burndown dashboard showing project progress. <Build> Second, we have the Quality dashboard. So what does the quality dashboard tell us? <next>
- #30: As you can see the Quality dashboard has four main graphs. <Build>First you can see if your test team is making progress on running test plans. <Build>Second, you can see how build are doing over time. What’s the trend like? Are you having lots of success or failure? <Build>Third, what’s your bug trend like. Are you closing out bugs or are you stagnate. Or is the velocity of your bug filing far exceeding your team’s ability to fix, test, and close out bugs? <Build>Finally, are you seeing a bad trend related to bug reactivations—bugs that were closed reopened by test as not fixed? All of this information is there for you in a quick, heads up dashboard format.