Take Cloud Hybrid Search to the Next Level
2 likes572 views
The document discusses hybrid search between on-premises SharePoint and SharePoint Online. It provides an overview of different approaches to hybrid search including split workload, split user, and search providing a unified view. It then discusses three approaches to hybrid search including classic federated search, crawling Office 365 content, and using the cloud hybrid search service application. The document also includes several case studies on implementing hybrid search and considerations for a hybrid search solution.
Take Cloud Hybrid Search to the Next Level
- 2. we love hybrid search - it's amazing how fast usage is growing Jeff Teper @jeffteper
- 4. Focused on Search and SharePoint since 2004 Longtime Search Nerd • CTO, BA Insight • Senior PM, Microsoft • VP, FAST • SVP, LingoMotors About Jeff Fried Passionate About • Search • SharePoint • Search-driven applications • Information Strategy Blog: BAinsight.com/blog Technet Column “A View from the Crawlspace” jeff.fried@bainsight.com
- 5. About BA Insight – Connectivity – Applications - – Classification - – Analytics
- 8. Search is not stationary
- 11. The Evolution of SharePoint: HYBRID ManagementExtensibilityExperiences | Server Experiences ManagementExtensibility | Server | Server HYBRID Team Sites Portals Enterprise Content Mngt BI
- 12. Approaches to Hybrid SharePoint Split Workload different tools in different places Split User task uses content or sites across ‘the divide’ Exchange, SharePoint, Skype OneDrive, Yammer, PowerBI, Delve Extranet, Mysites, Team Sites, Project Sites Portals, Intranet, Services/Applications Links Search
- 13. Search Provides a Unified View
- 14. – – – – 3 approaches for hybrid search 14
- 15. “Classic” Hybrid Search is Federated not a single result set OOB
- 16. SharePoint 2013/2016 Search Architecture Web Service (CEWS)
- 19. Benefits of Cloud Hybrid Search 2) Makes finding content easy, wherever the content lives 1) Simpler, easier, and less costly to run search
- 20. SharePoint Server (On-premises or Hosted) Office 365 SharePoint Online Content Onedrive for Business Content SharePoint Content Cloud Hybrid Search
- 21. Case Study C: Split Users with SharePoint
- 22. Setting up Cloud Hybrid Search • • 1. 2. 3. 4.
- 24. 24 New Sites to bookmark
- 25. The Cloud SSA
- 26. Use search verticals with Cloud Hybrid Search SharePoint Online Custom result source using Local SharePoint results plus a filter which excludes results from on-premises TIP: Can be used during validation of hybrid search in the production tenant. Result source query: {searchTerms} NOT(IsExternalContent:1)
- 27. Start with “Everything”? This is the default result source using Local SharePoint results but it has been renamed to «Everything» in the Search Navigation configuration. SharePoint Online Everything
- 28. Result Sources are your friend The Support Search vertical only searches sites that are relevant to the Support team. It uses Local SharePoint results plus a filter on which sites to include in the search results Result source query: {searchTerms} ( Path:»http://sp2010» OR Path:»file://fileshare» OR Path:»http://demohybrid... /../supportforum») SharePoint Online Support Search
- 29. SharePoint 2016 Hybrid Cloud Hybrid Search User Profiles Following Extranet Compliance (DLP/e- Discovery) Config Experience Built on Search
- 30. Differences between Cloud Hybrid Search in SP2013 versus SP2016?
- 33. External Content (on-premises and/or in the cloud) SharePoint Server (On-premises or Hosted) Office 365 SharePoint Online Content Onedrive for Business Content Connectors SharePoint Content Adding External Content Cloud Hybrid Search
- 34. Connectors to MANY Enterprise Systems • • • • ERP and Portal Systems • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •
- 35. External Content in O365 UX Unified view across all content - on-premises and on-line - inside and outside SharePoint
- 36. DLP Sensitive Data Search works with hybrid Search for sensitive data across on-premises and SharePoint Online All Built-in sensitive types Identification and export Extends to data in OneDrive Sensitive Information type detection through KQL searches Get instant statistics Preview & export results
- 37. Current Caveats: 1) don’t see thumbnails, just file icons 2) Have to query for it to show up
- 38. – – – – Case Study C: Cloud SSA, external content Large global company in materials science
- 39. Scaling
- 40. Item Limits and Pricing 1M items of external content in index for every 1TB storage in O365 1TB included by default + 0.5 GB per licensed O365 user No limit on number of items from O365 in the index 2000 users x 0.5 GB = 1TB + 1TB default = 2 TB total -> 2M external items indexed + Can also buy the “Office 365 Extra File Storage” Add-on $0.20/GB/Month = $200/TB/Month = $200/M items/Month 50,000 users x 0.5 GB = 25TB + 1TB default = 26 TB total -> 26M external items indexed
- 41. External Content (on-premises and/or in the cloud) Custom Processing CEWS Bottlenecks: 1) Source systems 2) Content Processing 3) Indexer ….
- 42. External Content (on-premises and/or in the cloud) Bottlenecks: 1) Uplink 2) Source systems ….
- 43. 43 Performance
- 44. 500K items crawled on an Azure D3 50 DPS 100 DPS 1 hour
- 45. SUPPORTED – Custom IFilter – BCS connectors – Partner connectors Customizations with Cloud Hybrid Search SUPPORTED – Tenant level schema mapping – Query rules – Result sources Cloud SSA SCS/O365 NOT SUPPORTED • Content that requires custom security trimming NOT SUPPORTED • Site collection level schema mapping • Custom security trimming • Custom entity extraction • Content enrichment web service
- 46. Issues with Cloud Hybrid Search (1) Cloud Hybrid Search "annoyances" Performance Characteristics slower query latency for on-prem queries against Cloud SSA SharePoint Online Limitations no synonyms no site-level schema no full trust code access Hybrid Administration Weaknesses clunky metadata mapping can't remove on-premises search results from Cloud SSA trickier to test & debug crawls can't reset index from Cloud SSA
- 47. Should I run index reset? NO!
- 48. Issues with Cloud Hybrid Search OOB 48 Content Enrichment no CEWS no Entity Extraction Security no Custom Security Trimming Can't crawl across Multiple Domains Can't Crawl SP in Classic Auth Mode Data Sovereignty export-restricted content can't be put in O365 index Limitations of Cloud SSA
- 49. External Content (on-premises and/or in the cloud) SharePoint Server (On-premises or Hosted) SPO Content OneDrive Content Connectors SharePoint Content Connector Framework Office 365 AutoClassifier (app version) CEWS Custom Processing
- 50. Case study D: Content Enrichment Content Cloud SSA Connector Framework Indexing Connectors Smart Pipeline AutoClassifier Custom Stage A Custom Stage C Custom Stage B
- 51. Online On-Prem Cloud Hybrid Search under the covers Security = identity sync + ACL mapping Cloud SSA Cloud SSA ParseCrawl SCS ACL Map Process Blob store queue
- 52. • • Directory Synchronization SID S-1-5-21-1212121212-1212121212-1212 jaden@corp.hybridsearch.com msOnline- OnPremiseSecurity Identifier S-1-5-21-1212121212-1212121212-1212 PUID PUID-XXXX-XXXXXXXXXX
- 53. Mapping of Access Control Lists Allow: S-1-5-21-1212121212-1212121212-1212 Allow: PUID-XXXX-XXXXXXXXXX • User SIDs are mapped to PUIDs • Group SIDs are mapped to Object IDs • «Everyone» and «Authenticated users» are mapped to «Everyone except external users»
- 54. Case Study E: Crawling Cross-Domain
- 55. A global single index solution Cloud SSA Cloud SSA Cloud SSA Cloud SSA Cloud SSA BUT export-restricted content can’t be in the global index
- 57. “Classic” Hybrid Search is Federated not a single result set OOB
- 59. Case study F: Data Sovereignty & Federation
- 60. Issues with Cloud Hybrid Search OOB Content Enrichment no CEWS no Entity Extraction Security no Custom Security Trimming Can't crawl across Multiple Domains Can't Crawl SP in Classic Auth Mode Data Sovereignty export-restricted content can't be put in O365 index Limitations of Cloud SSA BA Insight Solution Connector Framework AutoClassifier Connector Framework can 'map down' to AD groups can 'map across' cross-domain can crawl and map security Federator
- 62. Key Considerations for Hybrid: Workloads, Environment, Data, Customizations Availability of features Online versus On-Premises on particular workloads Significant investments in customization of On-Premises workloads Concerns over global network performance with remote sites Regulatory considerations Manageability concerns
- 67. Thank you! Toronto Enterprise Collaboration User Group Change Management, Governance, SharePoint, Office 365, Yammer, PowerBI, etc http://www.meetup.com/TSPBUG/ Toronto SharePoint Users Group http://tspug.com/ THANK YOU & See you next year! Join us for SharePint after the event @ 5:30pm 6982 Financial Dr. and don’t forget to submit feedback after each session for your chance to win great prizes at the end of the day! https://www.surveymonkey.com/r/spstoronto2016