Performance benchmark for CRM Online 2016 update 1
0 likes313 views
The document provides a benchmark for the performance of Microsoft Dynamics CRM Online 2016 Update 1 handling 17,868 concurrent users. Testing showed CRM Online 2016 Update 1 could support over 17K users performing tasks every 10 minutes with average page response times of 0.58 seconds and average transaction times of 0.91 seconds, totaling nearly 680,000 transactions over 2 hours. Customizations, roles, security, and over 650 million records were included to replicate a realistic enterprise environment.
Performance benchmark for CRM Online 2016 update 1
- 1. Performance Benchmark for Microsoft Dynamics CRM Online 2016 Update 1 PUBLISHED: DECEMBER 2016
- 2. Copyright This document is provided "as-is". Information and views expressed in this document, including URL and other Internet website references, may change without notice. You bear the risk of using it. Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. The videos and eBooks might be in English only. Also, if you click the links, you may be redirected to a U.S. website whose content is in English. © 2016 Microsoft. All rights reserved. Microsoft, Active Directory, Excel, Hyper-V, Internet Explorer, Microsoft Dynamics, Microsoft Dynamics logo, MSDN, Outlook, Notepad, SharePoint, Silverlight, Visual C++, Windows, Windows Azure, Windows Live, Windows PowerShell, Windows Server, and Windows Vista are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners.
- 3. Contents Applies to .......................................................................................................................4 Overview.........................................................................................................................4 Objective.........................................................................................................................4 Results summary............................................................................................................4 Data load ........................................................................................................................5 Organization structure ..................................................................................................5 Customizations...............................................................................................................6 User data.........................................................................................................................6 Test environment configuration ...................................................................................8 Testing method..............................................................................................................8 Setup configuration .............................................................................................................................8 Tuning and optimizations ....................................................................................................................9 Test run details ....................................................................................................................................9 Conclusion ......................................................................................................................9
- 4. This performance paper provides a benchmark for performance of a Dynamics CRM organization instance running Microsoft Dynamics CRM Online 2016 Update 1. Applies to Microsoft Dynamics CRM Online 2016 Update 1 (Version 8.1) Overview Microsoft Dynamics CRM delivers intelligent customer engagement to the market, helping companies create customer experiences that are personalized, proactive, and predictive. Dynamics CRM provides data anywhere, across a wide array of devices, ranging from phones and tablets to PCs, and through a number of popular client types, such as smartphone apps, tablet apps, and Microsoft Dynamics CRM for Outlook. This paper highlights the scalability and performance that can be achieved in terms of concurrent users and feature functionality with the release of Dynamics CRM Online 2016 Update 1. Objective This white paper benchmarks the performance and demonstrates the scalability and performance of Dynamics CRM Online 2016 Update 1. Results summary Performance testing was conducted for an organization that was set up in a standard Dynamics CRM Online environment. The client PCs used in the test infrastructure were standard Microsoft Azure virtual machines. Testing was carried out to benchmark the performance of Dynamics CRM Online 2016 Update 1 serving 17,868 concurrent users, performing tasks with a think time of 10 minutes. In this environment, Dynamics CRM Online 2016 Update 1 demonstrated the following performance characteristics. Concurrent Users + Total Record Count Web Requests per Hour Average Page Response Time Average Business Transaction Time Business Transactions* 17K 656,549,587 594K 0.58 secs 0.91 secs 679,963 + 17,868 users, each performing a business transaction once every 10 minutes. *A business transaction is a full user scenario, which is simulated by the various steps in a test case. This test case is not a measure of system capacity or throughput, but reflects the total number of business transactions that completed successfully during the more than two-hour test run. These results reflect the scalability and performance achieved on a specific Microsoft Dynamics CRM Online 2016 Update 1 implementation running in a Dynamics CRM Online environment. Factors like specific customizations, solutions, and features imported by enterprise organizations, and geographic distribution of users can affect how enterprise organizations use the CRM system. Therefore, results will vary for each implementation. Θ Important
- 5. This benchmark focuses on server-side performance and metrics. The response times reported here are for clients in a test environment, running Azure Virtual Machines. As a result, response times here are in no way indicative of exact client responsiveness that might be seen when rendering in a browser from a client located somewhere else. Data load User load was modeled to reproduce 15,040 concurrent users, in a model similar to an enterprise-class deployment of a Microsoft Dynamics CRM organization. The actual load generated was 17,868 users. The increase in the number of users occurred because some test cases executed on more than one virtual machine for the same virtual user. The target load was determined after reviewing the baseline from previous releases and the capacity (memory, compute power) of the Dynamics CRM Online servers. The organization database size was 1.024 TB. Organization structure To test the scale of a business unit (BU) defined security model, a complicated business unit hierarchy with 47 sectors (4 BUs per sector for a total of 188 BUs) was created. The depth of business units was set to four. To test the performance of a CRM system in a realistic manner, each of the business units was assigned users of different security roles. Nine security roles were selected for all users in the Dynamics CRM organization, of which eight were security roles that come “out-of-the-box” with Microsoft Dynamics CRM; one role, named Sales Associate, was a custom security role. The security roles were: System Administrator Sales Manager Marketing Manager Customer Service Representative Manager Scheduler Marketing Professional Salesperson Customer Service Representative Sales Associate
- 6. This diagram showcases the distribution of users and teams in each of the business units. Customizations To model a realistic enterprise, the organization was configured with four customizations. Commonly-used entities like account, contact, lead, and task were modeled with custom attributes. Field-level security was also enabled on custom attributes to reflect a realistic enterprise organization. User data Based on customer research, each user of a specific role was assigned a realistic set of data. The data that the user would own was based on the user’s role. Before the test, the total data in the test database included more than 656,549,587 business records with a total database size of 1.024 TB. This table provides a summary of the record count, which shows the top 51 entity tables by row count Table Name Row Count ActivityPartyBase 184,652,344 PostRoleBase 83,441,322 ActivityPointerBase 59,481,420 PostBase 50,213,237 PostCommentBase 44,538,712 PostRegardingBase 26,455,649 AnnotationBase 25,710,513
- 7. TaskBase 24,108,341 LeadAddressBase 22,344,376 PrincipalObjectAccess 20,609,782 CustomerAddressBase 19,195,039 EmailSearchBase 18,896,180 ImageDescriptor 18,730,149 BusinessProcessFlowInstanceBase 15,536,906 LeadBase 11,172,188 IncidentBase 4,895,570 KnowledgeArticleBase 4,028,276 AccountBase 4,005,329 ContactBase 3,728,127 PostFollowBase 2,555,946 DocumentIndex 2,014,138 InvoiceBase 859,624 InvoiceDetailBase 859,540 SalesOrderDetailBase 856,622 SalesOrderBase 855,039 OpportunityBase 625,519 OpportunityProductBase 607,093 new_imBase 538,159 QuoteBase 532,208 PrincipalEntityMap 471,777 PrincipalAttributeAccessMap 447,886 QuoteDetailBase 402,299 ConnectionBase 393,280 ListMemberBase 365,361 SystemUserBusinessUnitEntityMap 258,718 ResourceGroupExpansionBase 188,446 ListBase 162,972 SystemUserPrincipals 157,415 MailboxBase 116,211 QueueMembership 93,409 UserEntityUISettingsBase 90,809 DependencyBase 87,522 QueueBase 84,208 RecurrenceRuleBase 82,180 CampaignBase 77,211
- 8. DependencyNodeBase 71,249 InternalAddressBase 68,144 CalendarBase 64,327 CalendarRuleBase 64,326 TeamMembership 61,406 Test environment configuration Four test client PCs (virtual machines), hosted in Azure, were used to simulate concurrent user load. Here is an overview of the environment configuration. Testing method Test scenarios were based on customer research and were created using the Microsoft Dynamics CRM Performance and Stress Toolkit (Performance Toolkit), which is designed to formalize performance testing of Microsoft Dynamics CRM by facilitating load testing of simulated customer environments. Setup configuration A batch of 17,868 concurrent users performed create, update, and delete (CRUD) operations within Dynamics CRM Online. A warm-up time of 30 minutes was needed to get each client to load all the users. Performance data was captured every five seconds.
- 9. A think time of 10 minutes was used per user to simulate a real-world scenario. Therefore, virtual users were running tests continuously with a lag time of 10 minutes between each test run. Every test run carried additional overhead of authenticating the virtual users before initiating the test run. The authentication was cached for subsequent test runs. A single virtual user performed all the test cases sequentially before moving to the next test run, repeating all the tests. Internet Explorer 11 was used to render the pages. Tuning and optimizations Standard optimization techniques that are part of the Dynamics CRM Online optimization process, such as adding recommended indexes, were included with the Dynamics CRM Online organization used in this benchmark. Test run details Metric Value Duration 2 hours Max User Load 17,868 Tests/Sec 26.72 Average Test Time 2.47 seconds (test included several pages, web test plugin executions, and an additional SQL Server connection to randomly select records) Average Page Time 0.58 seconds Total Records 656,549,587 Web Requests/Sec 165 Conclusion The results reflect the scalability and performance achieved in a standard Dynamics CRM Online organization running Dynamics CRM Online 2016 Update 1. Actual performance observed by customers may vary based on factors like customizations, solutions, other features, geographic distribution of users, and network latency.
- 10. These results demonstrate the robustness of Dynamics CRM Online 2016 Update 1 and its capability to handle concurrent user activities with ease for enterprise CRM scenarios.