Highly available and scalable architectures
4 likes377 views
The document discusses the architectural considerations for highly available, scalable, and performant SharePoint 2010 systems, detailing crucial aspects such as load balancing, redundancy, and performance optimization. It emphasizes the importance of service configuration and resource allocation for maintaining system availability and performance, while also providing recommendations for SQL management and server clustering. Additionally, it covers monitoring and health assessments to ensure optimal system performance and user experience.
Highly available and scalable architectures
- 1. Highly Available, Scalable, and Performant ArchitecturesFor SharePoint 2010SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 2. About Me?About Me: Over 6 years of experience SharePoint Architecture Experience, trained SharePoint architecture and development classes for nationally renown SharePoint-focused training organization.Training & Certifications:MCTIP: SharePoint 2010 Administration
- 3. MCPD: SharePoint 2010 Development
- 4. Microsoft’s SharePoint Masters Training (Redmond, WA)Other Notable Experience:Published Author on SharePoint 2010 (SharePoint 2010 Workflows in Action)
- 5. National Speaker at SharePoint Conferences (The Experts Conference-LA, SharePoint Technology Conference-San Fran, TechFuse-MN, SharePoint Saturdays)
- 6. SharePoint Blog: www.philwicklund.comAgendaWhat is availability?2007 vs. 2010 ConsiderationsWeb TierApp TierSearch and FAST SearchExcel Services, PowerPivot, and PerformancePointOffice Web Apps, Word, and Visio ServicesWorkflowSandboxesSQL TierVirtualizationNew health and monitoring featuresSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 8. What is availability?No single point of failurePerformance – horrible performance can mean it’s not availableDisaster recoverySLA (.9999, .999, etc)SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 9. What is availability?No single point of failureSharePoint Services: IIS (WFEs)? Search crawl? Search query? Excel Services? OWA? Workflow? Visio? Word automation? Performance Point? Etc.Network: NICs? Routers? load balancers? Etc.SQL & Storage: RAID? Mirroring? Shipping? SAN? Etc.Datacenter: redundant across DCs?2 WFEs != Redundant!SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 10. What is availability?10,000 users DOES NOT EQUAL 2 WFEs, 1 app server, and 1 SQL server EVERY TIME!!!!Performance considerations:STOP LOOKING AT # OF USERS!Instead focus on what services, and RPSRPS – cue your Business AnalystYou could host 1 million SharePoint users on a laptop if you only get 1 RPH!THEN TEST IT!
- 11. 2007 vs 20102007: Index server wasn’t scalable2010: TONS MORE SERVICES!Know what services you enable, and why because there can be huge performance implications.EXPECT TO BUY MORE HARDWARE (or dedicate more virtual servers and virtual resources)Some throw out 2x, but you can do better than thumb in the air hereSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 12. Load BalancingWeb Front End and SQL ratioWeb Tier
- 13. Web TierLoad balancing:WNLB (software LB)F5 (or other hardware LB)Network RedundancyPerformance sizing:3 to 1 optimal WFE to SQL ratioMinimum 8GB RAM w/ 2 coresTry to keep application services off WFEsSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 14. SearchFAST SearchExcel Services / PowerPivotPerformance PointOffice Web Apps, Word, and Visio ServicesWorkflowSandboxesApplication Tier
- 15. SearchComponents and TerminologyAdmin ComponentCrawl ComponentCrawl DatabasesQuery ComponentIndex PartitionQuery ProcessorProperty DatabasesTopology ExamplesSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 16. SearchAdmin Componentprocesses config changes (search administration)no redundancy but search still works if it goes downAdmin DatabaseMSSSecurityDescriptorstable ALWAYS in RAM on SQL redundant through mirroring - search stops if DB goes downSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 17. SearchCrawl ComponentSupports Active/PassiveCan have multiple active crawlersEven multiple per server where CPU is main limitationCrawl DatabaseCrawl Queue – only one crawl component can crawl a host at a time!Get around this by adding more crawl components.Place on dedicated spindles (target 3.5k – 7k IOPs)Optimize speed by adding DBs and using Host Destination Rules to allocate hosts to a particular DB.Tends to reset SQL cache (isolate this DB!)SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 18. SearchQuery ComponentSupports Active/PassiveCan have multiple active query seversEven multiple per server where CPU is main limitationNote – 1 per index partitionIndex partitions“partition” of the index as a wholeMax 10m itemsSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 19. SearchQuery ProcessorInitiates query to query servers and partitionsPerforms meta data lookup w/ Property DBGets results and…Performs security trimmingCalculates relevancyfinalizes result set and returns resultsState-less. Can add this role wherever, just note CPU costs and memory costs.RAM – ensure you can dedicate size of MSSSecurityDescriptorstable wherever this component exists.SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 20. SearchProperty DBIndex Partition assigned to a prop DBProp Db can have multiple partitions.Max 50m itemsLargest search DBRecommendations:Keep 33% of critical tables in SQL’s RAM (MSSDocsSDIDs, MSSDocProps, and MSSDocresults)Dedicate spindles that can support 7k IOPsSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 23. SearchSummary of recommendations:Pair Active/Failover components – try to avoid having 2 active components of same type on same server.Use host distribution rulesDeploy DBs on dedicated spindles with guarantee of 7k IOPsAgain, crawl DB does not play nice!Have 2 cores available for each active component, and one additional core for each failoverPair with other services with great caution!SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 24. FAST SearchMedium farm supporting roughly 40m documents.PROBLEM? NOT REDUNDANT!Courtesy - technet
- 25. FAST SearchHow many index columns?1 index column= roughly 3.6TB of indexed content.Content size * multiplier / 3.6 = # of index columns (round up and account for growth)Eg: 10TB * .5 = 5 TB5TB / 3.6 = 2 index columns (rounded up for growth)Multiplier.3-.5 for binary (Word, PowerPoint, etc)3-5 for .txt1.5 – 2.5 for HTMLSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 26. FAST SearchMinimum to achieveredundancy for indexingand query matchingcomponents, given 10TB of binary content.Note – the other components are stateless and can go anywhere, but depending on hardware may want more than 4 to accommodateSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 27. FAST SearchHowever, index performance is inhibited because of potential query load, so add another search row to free up the index server.
- 28. FAST SearchEach query matcher component can handle 5-50 QPS (depending on features) – so scale out to match your expected QPS
- 29. FAST SearchHowever, each index server can handle about 35 docs / second, so you may want to add more index columns to meet freshness requirements:
- 30. FAST SearchDon’t forget about redundancy in your SharePoint Farm!People search (>= 2 crawl components and query components)FAST SharePoint ConnectorSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 31. Excel Services &PowerPivot1 Technology, 2 tools: PP for SharePoint & PP for ExcelLoad BalancingRound robinHealth basedMem firstCPU secondCached files will greatly affect the selection of the final target serverFile size relates to RAM on client/server2x ratio (if file is 1GB, account for 2GB ram)PP is NOT claims aware - can't run on a web app that is setup as claims - only works in Classic modeSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 32. Excel Services & PowerPivotRecommendations (especially for PP):8GB RAM minimum – reality, a lot more!4 procs, 16 coresObvious conclusion – PHYSICAL SERVERSTrusted file locations (PP “Side Effect”)Maximum workbook sizeAllow external data (enables PP)Warn on refreshWeb app settingsMax upload sizeSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 33. Performance PointHEAVY MEMORY UTILIZATION! Cache, cache, cache!Examples - App objects, Query metadata, Filter data, Query results, Rendered viewThere are a bunch of PPS counters to look at cache effectivenessCache overviewStandard first big hitHowever, rendered views are usage basedEach PPS server is responsible for it's own cacheCache settingsCan toggle how long things are cached, includingUser identity and privilegesFiltersHowever - SSAS data is ALWAYS up to date because PPS can look at the timestamp on the data to see if it changed. All other datasources are cached until cache dies.SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 34. Performance PointData retrievalONLY AS PERFORMANT AS THE DATA LAYER UNDERNEATHA scorecard is individual row(s), eg for each row in the card you have a query – so BE CAREFUL!SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 35. Office Web AppsTakes an Office doc and converts it to HTML/CSS/JS/ImagesCheck cache > SA gets doc > SA spins up WP to convert > store in cache > RenderIncludes Word, PowerPoint, and Visio servicesRESOURCE INTENSE!Worker process for each sessionEach doc cached in memory and temporarily to diskSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 36. Office Web AppsEnable Word, PP, Visio service app on 2 or more app serversRecommendations:Word & PowerPoint worker processesRecommendation - run 2 worker processes for each core (configured via PS)Configure OWA Cache site and expiration periodConfigure Cache database location (which SQL server)SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 37. WorkflowWhere is workflow processed?WFE if <= threshold (default is 15 WF across farm)Timer job > thresholdStart the OWSTimer service (every 5 minutes):Workflow timer batch size: default is 100Batch size is per content DBSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 38. WorkflowRecommendations:Set threshold to 0 forcing workflow load onto OWSTimerSet-SPFarmConfig –WorkflowPostponeThreshold 0Start WF service on 2+ app servers, keeping WFEs free of loadCranking up batch size seems wise, but be careful!SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 39. SandboxesPLAN WHERE TO START THIS SERVICE!Recommendations:By default there is only 1 worker process per serverFor optimal performance change this to # of cores +1 via PowerShellSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 40. SQL ClusteringSQL MirroringSQL LoggingDatabase Availability
- 41. ClusteringServer instance redundancy:Note: NOT database redundancySharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 42. MirroringData & Server instance redundancy:Synchronous or Asynchronous (think latency/bandwidth)SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 43. Clustering AND MirroringSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 44. Log ShippingSend log and forgetGood for test and stage environmentsGood for PATCHING!(next slide)SharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 45. Clustering, Mirroring, AND Shipping! – OH MY!
- 47. VirtualizationYes: WFEs & App serversNo (although fully supported): Excel Services (PP), SQLHost can be single point of failureDon’t over subscribe hosts NUMA boundariesSharePoint FREEWAREwww.PhilWicklund.com SharePoint CONSULTINGwww.RBAconsulting.com
- 48. StretchingMDHA – multi datacenter highly available2 close DCs< 1ms latency>= 1GB/sec bandwidth
- 50. Health Analyzer and ScoreHealth Analyzer“Health Rules” – OOB and build your ownHealth ScoreFarm scored 1-10, Request throttling starts at 10User requests last to be throttledPUT requests never throttled
- 51. Health ScoreHealth Score Example:CPU cycles/sec0-500 score = 2501-1000 = 51001-1500 = 8> 1500 = 10(start throttling HTTPrequests)Customized via PowerShellScore returned in Response Header
- 52. Usage LoggingDefault this is not enabled. Only enabled via PowerShell.Configure what counters to log, and how often.What’s logged? Requests (type, source, machine, user), and any counters (think perfmon, like RPS, queue depth, etc)Logging DBCan grow very largeNeeds its own monitoring strategyOOB retains 14 days by default, and extended up to 31 days.Build a warehouse and cubes!