O365Engage17 - Configuring share point hybrid search
1 like103 views
The document discusses configuring SharePoint hybrid search. It provides an overview of hybrid search history and the current hybrid search solution. It then covers prerequisites, crawling and query flows, topology considerations, customizations, and integrating on-premises data. Specifically, it outlines how content is crawled from on-premises sources to the cloud search service application and indexed, and how queries are processed from both on-premises and the cloud. It also provides examples of result sources and managed properties that can be used for hybrid search implementations.
![4
Slide
4
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
The Hybrid Search Solution from 2013
• In SharePoint Server 2013 Microsoft introduced hybrid search
using query federation.
• Search results came as separate results sets.
• This solution required hosting all search components on-
premises.](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-4-320.jpg)
![5
Slide
5
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Hybrid Results
Previously,
hybrid results
were not
comingled.
Results from
SharePoint
On-Premise
Results from
Cloud](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-5-320.jpg)
![10
Slide
10
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
#ITDevConnections
of Enterprise Customers
are “on the road” to the
cloud.66%](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-10-320.jpg)
![16
Slide
16
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Prerequisites
• The Cloud Search service application
• Logical architecture: crawling
• Security and Identity
• Combining the old and the new hybrid search
• Logical architecture: query](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-16-320.jpg)
![21
Slide
21
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Search Content Service
• Service hosted in Azure providing storage for
SharePoint content intended for use in Search and
Graph scenarios.
• Routes data from SharePoint crawlers
• Allow for quick refeeding of content to provide DR
features and support reprocessing when document
processors or schema has changed.](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-21-320.jpg)
![27
Slide
27
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Validating the Implementation
• ULS / Fiddler
• Call IsExternalContent:1 on Search API url
• _api/search/query?querytext=‘isexternalcontent:true’](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-27-320.jpg)
![31
Slide
31
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Combining hybrid search approaches
• Dog House Toys is also considering hybrid search, but
they have some business sensitive content that they need
to keep indexed on-premises for legal reasons.
• Dog House Toys can combine the two hybrid search
approaches
• They implement query federation from the site that has the
high-business impact content on-premises](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-31-320.jpg)
![33
Slide
33
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Mapping of Access Control Lists
As Items are indexed in Office 365, the access control
entries are looked up in the cloud directory service.
• User SIDs are mapped to PUIDs
• Group SIDs are mapped to Object IDs
• “Everyone” and “Authenticated Users” are mapped to “Everyone except
External Users”
Sync your users and groups BEFORE you crawl!
Allow: PUID-XXXX-XXXXXXXXAllow: S-1-11-21-555-555-1212](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-33-320.jpg)
![34
Slide
34
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Topology: Single Node
• Sizing
• Scaling up and out may not improve performance
• Follow SharePoint 2016 Minimums
• 12GB RAM, 4 CPU, 80GB disk
• Crawl DB is the same ~20GB/1M items
• Fault Tolerance
Queries in Office 365
Outbound Hybrid Queries
Crawling
#ITDevConnections
Single Server
Admin
Crawl
QPC
APC
CPC
IndexerSSA (4)](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-34-320.jpg)
![35
Slide
35
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Topology: Multi-node
• Fault Tolerance
Queries in Office 365
Outbound Hybrid Queries
Crawling
Throttling
Servers busy/near capacity or tenant quota reached
No SLA on DPS, index freshness or outbound query rate
Bottlenecks
Customers Uplink
Content repository
Crawler CPU
Search Server #2
Admin
Crawl
QPC
Search Server #1
Admin
Crawl
QPC
APC
CPC
Indexer
SSA
SSA
unused
HA](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-35-320.jpg)
![37
Slide
37
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Hybrid Search and Customization
• With the new hybrid search solution all index and query related
settings are managed in SharePoint Online
• Crawling and parsing are still managed on-premises.](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-37-320.jpg)
![38
Slide
38
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Customizations
SUPPORTED
• Custom IFilter
• BCS connectors
• Partner connectors
• On-Prem Cloud SSA
SUPPORTED
• Tenant level schema mapping
• Query rules
• Result sources
• ...
](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-38-320.jpg)
![39
Slide
39
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
On Premises Data
•Traditional Hybrid BCS solution
• Complex to configure
• Additional hardware for inbound connectivity
• Odata Endpoint required
•Search based solution
• On premises BCS
• Cloud Hybrid Search
• Display templates optional](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-39-320.jpg)
![44
Slide
44
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
References
Microsoft Press
• Planning an Preparing for Microsoft SharePoint Hybrid
• https://blogs.msdn.microsoft.com/microsoft_press/2016/04/26/free-ebook-planning-
and-preparing-for-microsoft-sharepoint-hybrid
• Configuring Microsoft SharePoint Hybrid Capabilities
• https://blogs.msdn.microsoft.com/microsoft_press/2016/07/06/free-ebook-
configuring-microsoft-sharepoint-hybrid-capabilities
TechNet
• Configure Hybrid for SharePoint Server 2013
• https://technet.microsoft.com/en-us/library/jj838715.aspx
Office Hybrid Resources Center
• http://hybrid.office.com/](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-44-320.jpg)
![45
Slide
45
Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER]
Follow us:
#O365ENGAGE
Questions? | Thank You
Matthew McDermott
matthew@aptillon.com
@MatthewMcD
We’d like to know what you think!
Please fill out the evaluation form you
received at the registration desk for this
session
Session recordings and materials:
Materials will be available on
Office365Engage.com soon](https://image.slidesharecdn.com/configuringsharepointhybridsearch-170717095434/85/O365Engage17-Configuring-share-point-hybrid-search-45-320.jpg)
O365Engage17 - Configuring share point hybrid search
- 1. 1 Slide 1 Configuring SharePoint Hybrid Search Matthew McDermott, MVP Director, Aptillon, Inc. matthew@aptillon.com • @MatthewMcD
- 4. 4 Slide 4 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE The Hybrid Search Solution from 2013 • In SharePoint Server 2013 Microsoft introduced hybrid search using query federation. • Search results came as separate results sets. • This solution required hosting all search components on- premises.
- 5. 5 Slide 5 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Hybrid Results Previously, hybrid results were not comingled. Results from SharePoint On-Premise Results from Cloud
- 6. Search from anywhere, even on-premises sites Office 365 enterprise search and Delve for your on- premises content Preview and access the content when connected to your corporate network Relevancy and deep refiners across the entire result set
- 7. 7 Slide 7 Follow us: #O365ENGAGE Hybrid Today Is he going to show us a demo?
- 9. SharePoint Hybrid Scenarios Hybrid OneDrive Hybrid Sites Hybrid B2B Extranet Hybrid Search (Cloud SSA) Hybrid Profile Redirect Hybrid Ext App Launcher Modern Attachments (Ex OD4B) Hybrid Taxonomy Hybrid Auditing Hybrid Self Service Site Creation http://bit.ly/ABVideos
- 10. 10 Slide 10 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE #ITDevConnections of Enterprise Customers are “on the road” to the cloud.66%
- 11. 11 Slide 11 Support forum SupportSales & Marketing Knowledge Articles Dog House Toys has 2 SharePoint Server deployments (2013 and 2010), plus content in fileshares. Hardware requirements: 10 application servers 4 content + 6 search On-premises
- 12. 12 Slide 12 They roll out OneDrive for Business to their users, and start migrating sites to Office 365. Hardware required after migration: 9 application servers 3 content + 6 search site migration On-premises Office 365 SupportSales & Marketing Knowledge Articles NEW! OneDrive Support forum
- 13. 13 Slide 13 DHT learned that they can index their on-premises sites in the search index in Office 365, without upgrading their existing deployments. Current hardware requirements: 5 application servers 3 content + 2 search SupportSales & Marketing Knowledge Articles OneDrive Support forum On-premises Office 365 SP 2013 SSA
- 16. 16 Slide 16 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Prerequisites • The Cloud Search service application • Logical architecture: crawling • Security and Identity • Combining the old and the new hybrid search • Logical architecture: query
- 17. On Premises Infrastructure InternetMicrosoft data center IntranetPerimeter network Customer network AD Servers Office365 tenant AAD Connect Sync Server Azure AD Directory Service ACS Trust Azure AD Tenant Azure AD Proxy SharePoint STS User Profile Sync Service Secure Store Target App SharePoint Reverse Proxy ADFS Proxy ADFS ServersFederation Gateway Identity Platform SharePoint
- 18. 18 Slide 18 Prerequisites for Office 365 Hybrid Search SharePoint Server with Cloud Search service application 2013 or 2016 Directory Synchronization of AD users and groups Hybrid environment with Office 365 Basic hybrid search requirements Additional requirements for search previews Reverse proxy back to on-premises WAC server
- 20. 20 Slide 20 Follow us: #O365ENGAGE Cloud SSA What’s it take to build this?
- 21. 21 Slide 21 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Search Content Service • Service hosted in Azure providing storage for SharePoint content intended for use in Search and Graph scenarios. • Routes data from SharePoint crawlers • Allow for quick refeeding of content to provide DR features and support reprocessing when document processors or schema has changed.
- 22. CRAWL FLOW
- 23. 23 Slide 23 Follow us: #O365ENGAGE Crawling Content How does it work?
- 24. DirSync SP 2013/16 SP 2010 SP 2007 Fileshares BCS Cloud SSA SPO Search Index 1 2 34 5 6 7 Logical architecture: crawling Corporate network Office 365
- 25. SCS under the hood Crawler Content Indexing API Blob store Document state table Work queues Backend API Index/Graph On-Premises content source Search farm Azure Broker Crawler Content SPO content source The Hybrid feature allows tenants to index and search on-premises data alongside SPO data
- 26. 26 Slide 26 What is pushed to the SCS Endpoint? SharePoint 2013/2016 FileShares
- 27. 27 Slide 27 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Validating the Implementation • ULS / Fiddler • Call IsExternalContent:1 on Search API url • _api/search/query?querytext=‘isexternalcontent:true’
- 29. QUERY FLOW
- 30. SP 2013 Cloud SSA SPO Search Index Logical architecture: query SP 2010 1 2a Cheyenne issues a query from Office 365. Her user token contains her online identity and group memberships. 1 2a 2b 2b Office 365 Corporate network Cheyenne issues a query from a site on-premises. This sends over her on-premises claims to SPO Her user token gets rehydrated with her online claims as she is authenticated against Office 365.
- 31. 31 Slide 31 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Combining hybrid search approaches • Dog House Toys is also considering hybrid search, but they have some business sensitive content that they need to keep indexed on-premises for legal reasons. • Dog House Toys can combine the two hybrid search approaches • They implement query federation from the site that has the high-business impact content on-premises
- 32. SP 2013 Cloud SSA SPO Search Index Logical architecture w/ query federation Corporate network SP 2010 1 2a 2b Office 365 On Premises Search Service 3 Ruby issues a query from the site with sensitive content. He gets back search results from on-premises and online as separate result sets 3
- 33. 33 Slide 33 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Mapping of Access Control Lists As Items are indexed in Office 365, the access control entries are looked up in the cloud directory service. • User SIDs are mapped to PUIDs • Group SIDs are mapped to Object IDs • “Everyone” and “Authenticated Users” are mapped to “Everyone except External Users” Sync your users and groups BEFORE you crawl! Allow: PUID-XXXX-XXXXXXXXAllow: S-1-11-21-555-555-1212
- 34. 34 Slide 34 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Topology: Single Node • Sizing • Scaling up and out may not improve performance • Follow SharePoint 2016 Minimums • 12GB RAM, 4 CPU, 80GB disk • Crawl DB is the same ~20GB/1M items • Fault Tolerance Queries in Office 365 Outbound Hybrid Queries Crawling #ITDevConnections Single Server Admin Crawl QPC APC CPC IndexerSSA (4)
- 35. 35 Slide 35 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Topology: Multi-node • Fault Tolerance Queries in Office 365 Outbound Hybrid Queries Crawling Throttling Servers busy/near capacity or tenant quota reached No SLA on DPS, index freshness or outbound query rate Bottlenecks Customers Uplink Content repository Crawler CPU Search Server #2 Admin Crawl QPC Search Server #1 Admin Crawl QPC APC CPC Indexer SSA SSA unused HA
- 36. CUSTOMIZATION
- 37. 37 Slide 37 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Hybrid Search and Customization • With the new hybrid search solution all index and query related settings are managed in SharePoint Online • Crawling and parsing are still managed on-premises.
- 38. 38 Slide 38 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Customizations SUPPORTED • Custom IFilter • BCS connectors • Partner connectors • On-Prem Cloud SSA SUPPORTED • Tenant level schema mapping • Query rules • Result sources • ...
- 39. 39 Slide 39 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE On Premises Data •Traditional Hybrid BCS solution • Complex to configure • Additional hardware for inbound connectivity • Odata Endpoint required •Search based solution • On premises BCS • Cloud Hybrid Search • Display templates optional
- 40. 40 Slide 40 Follow us: #O365ENGAGE BCS Solution On-Premises Data Integration to the Cloud
- 41. Managed Property for Hybrid Search Results • The IsExternalContent managed property is set to 1 for content that is crawled on-premises.
- 42. Example search verticals SharePoint Online Custom result source using Local SharePoint results plus a filter which excludes results from on-premises TIP: Can be used during validation of hybrid search in the production tenant. Result source query: {searchTerms} NOT(IsExternalContent=1)
- 43. Example search verticals This is the default result source using Local SharePoint results. Result source query: {searchTerms} (ContentType=SupportItem) It uses Local SharePoint results plus a filter on which content to include in the search results
- 44. 44 Slide 44 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE References Microsoft Press • Planning an Preparing for Microsoft SharePoint Hybrid • https://blogs.msdn.microsoft.com/microsoft_press/2016/04/26/free-ebook-planning- and-preparing-for-microsoft-sharepoint-hybrid • Configuring Microsoft SharePoint Hybrid Capabilities • https://blogs.msdn.microsoft.com/microsoft_press/2016/07/06/free-ebook- configuring-microsoft-sharepoint-hybrid-capabilities TechNet • Configure Hybrid for SharePoint Server 2013 • https://technet.microsoft.com/en-us/library/jj838715.aspx Office Hybrid Resources Center • http://hybrid.office.com/
- 45. 45 Slide 45 Session Title (Keep title BOLD) | Presenter Name (normal) | Time and Date of Session (normal) [CHANGE THIS IN THE MASTER] Follow us: #O365ENGAGE Questions? | Thank You Matthew McDermott matthew@aptillon.com @MatthewMcD We’d like to know what you think! Please fill out the evaluation form you received at the registration desk for this session Session recordings and materials: Materials will be available on Office365Engage.com soon