Azure search
Download as PPTX, PDF0 likes149 views
This document provides an overview of Azure Search including supported data sources, pricing plans, indexing and querying capabilities. Key points include: - Azure Search supports Cosmos DB, SQL databases, Blob Storage and Table Storage as data sources. - Pricing plans range from free to $2,850 per month based on storage, indexes, search units and replicas. - Features include scoring profiles, boosting, suggesters, autocomplete, simple and Lucene query syntax, analyzers and escaping special characters. - Data change detection policies support change tracking in SQL and high watermarks in Cosmos DB. Deletion detection also supported. - Cognitive Search enables natural language processing and image processing skills
![Scoring profiles
"scoringProfiles": [
{
"name": "newDocuments",
"text": {
"weights": {
"title": 2,
“info": 1 }
} ]](https://image.slidesharecdn.com/azuresearch-211103201136/85/Azure-search-7-320.jpg)
![Boosting
"functions": [ {
"type": "freshness",
"fieldName": "date",
"boost": 10,
"interpolation": "quadratic",
"freshness": {
"boostingDuration": "P7D"
} } ]](https://image.slidesharecdn.com/azuresearch-211103201136/85/Azure-search-8-320.jpg)
![Suggesters
"suggesters": [
{
"name": "sg",
"searchMode": "analyzingInfixMatching",
"sourceFields": [“title", “name"]
}
]](https://image.slidesharecdn.com/azuresearch-211103201136/85/Azure-search-9-320.jpg)
![Escaping and encoding special
characters
Следующие символы
+ - && || ! ( ) { } [ ] ^ " ~ * ? : /
необходимо экранировать с помощью
Небезопасные для использования в URL символы
" ` < > # % { } | ^ ~ [ ]
необходимо энкодировать. Например, символ # станет %23](https://image.slidesharecdn.com/azuresearch-211103201136/85/Azure-search-16-320.jpg)
![Regular expression search
lucene.apache.org/core/4_10_2/core/org/apache/lucene
/util/automaton/RegExp.html
bit.ly/2Ug2CKq
Place RegEx expression between forward slashes "/“
/[mh]otel/](https://image.slidesharecdn.com/azuresearch-211103201136/85/Azure-search-20-320.jpg)
![Data Change Detection Policy for SQL
"dataChangeDetectionPolicy" : {
"@odata.type" :
"#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy",
"highWaterMarkColumnName" : "[a rowversion or last_updated
column name]" }
"dataChangeDetectionPolicy" : {
"@odata.type" :
"#Microsoft.Azure.Search.SqlIntegratedChangeTrackingPolicy" }](https://image.slidesharecdn.com/azuresearch-211103201136/85/Azure-search-22-320.jpg)
Azure search
- 1. Welcome to the BELARUS, MINSK
- 3. Supported database types ■ Azure Cosmos DB ■ Azure SQL Database ■ SQL Server hosted in an AzureVM ■ Azure Blob Storage ■ AzureTable Storage
- 4. Azure Search elements ■ Datasources ■ Indexes ■ Indexers
- 5. Pricing ■ Free (3 indexes, up to 50 Mb storage) ■ Basic – 75$ month (15 indexes, 2Gb storage, up to 3 search units, up to 3 replicas) ■ Standart – 250$ month (50 indexes, 25Gb storage, up to 36 search units, up to 12 replicas, up to 12 ) ■ … up to 2850$ (with High Density support)
- 6. Create datasource with portal UI ■ Если вы используете Cosmos DB, то добавьте Database=ИМЯ_ВАШЕЙ_БАЗЫ к connection string
- 7. Scoring profiles "scoringProfiles": [ { "name": "newDocuments", "text": { "weights": { "title": 2, “info": 1 } } ]
- 8. Boosting "functions": [ { "type": "freshness", "fieldName": "date", "boost": 10, "interpolation": "quadratic", "freshness": { "boostingDuration": "P7D" } } ]
- 9. Suggesters "suggesters": [ { "name": "sg", "searchMode": "analyzingInfixMatching", "sourceFields": [“title", “name"] } ]
- 10. Suggester vs Autocomplete Suggestions API suggests documents and returns document Ids which contain the query term Autocomplete API returns potential terms from the index which match the partial term in the query
- 11. Simple query syntax wifi+luxury searching for wifi and luxury at same time “luxury hotel” searching for phrase wifi | luxury searching for wifi or luxury wifi –luxury searching for wifi without luxury motel+(wifi | luxury) you can combine with parenthesis lux* searching for words starting from lux
- 12. Lucene query syntax - queryType=full range searches are constructed in Azure Search through $filter expressions OR or || AND, && or + NOT, ! or -
- 13. NOT ! или - searchMode=any wifi –luxury поиск содержит wifi или не содержит luxury searchMode=all wifi –luxury поиск содержит wifi и не содержит luxury
- 14. Стемминг и лемматизация Сте́мминг — это процесс нахождения основы слова для заданного исходного слова. Лемматиза́ция — процесс приведения словоформы к лемме — её нормальной (словарной) форме.
- 15. Analyzers default analyzer is Standard Lucene Lucene's English analyzer applies stemming as per Porter Stemming algorithm Microsoft's English analyzer performs lemmatization instead of stemming
- 16. Escaping and encoding special characters Следующие символы + - && || ! ( ) { } [ ] ^ " ~ * ? : / необходимо экранировать с помощью Небезопасные для использования в URL символы " ` < > # % { } | ^ ~ [ ] необходимо энкодировать. Например, символ # станет %23
- 17. Fuzzy search "blue~" or "blue~1" would return "blue", "blues", and "glue“ but… "business~analyst" means business OR analyst.
- 18. Proximity search "hotel airport"~5 will find the terms "hotel" and "airport" within 5 words of each other in a document
- 19. Term boosting rock^2 electronic documents that contain the search term rock will be ranked higher than the other search term electronic
- 20. Regular expression search lucene.apache.org/core/4_10_2/core/org/apache/lucene /util/automaton/RegExp.html bit.ly/2Ug2CKq Place RegEx expression between forward slashes "/“ /[mh]otel/
- 21. ODATA support { "name": "Scott", "filter": "(age ge 25 and and lt 50) or surname eq 'Guthrie'" }
- 22. Data Change Detection Policy for SQL "dataChangeDetectionPolicy" : { "@odata.type" : "#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy", "highWaterMarkColumnName" : "[a rowversion or last_updated column name]" } "dataChangeDetectionPolicy" : { "@odata.type" : "#Microsoft.Azure.Search.SqlIntegratedChangeTrackingPolicy" }
- 23. SQL Server ChangeTracking ALTER DATABASE AdventureWorks SET ALLOW_SNAPSHOT_ISOLATION ON; ALTER DATABASE AdventureWorks SET CHANGE_TRACKING = ON (CHANGE_RETENTION = 2 DAYS, AUTO_CLEANUP = ON)
- 24. Data Change Detection Policy for Cosmos DB { "@odata.type" : "#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy" , "highWaterMarkColumnName" : "_ts" }
- 25. Data Deletion Detection Policy "dataDeletionDetectionPolicy" : { "@odata.type" : "#Microsoft.Azure.Search.SoftDeleteColumnDeletionDetectionPolicy", "softDeleteColumnName" : "IsDeleted", "softDeleteMarkerValue" : "true" }
- 26. Development NuGet package Microsoft.Azure.Search REST API Node.js and Java samples
- 27. Cognitive Search
- 28. Natural language processing skills entity recognition language detection key phrase extraction text manipulation sentiment detection
- 29. Image processing Optical Character Recognition (OCR) textExtractionAlgorithm "handwritten“ (for English only) textExtractionAlgorithm "printed“ Identification of visual features provided by ComputerVision in Cognitive Services
Editor's Notes
- #4: https://docs.microsoft.com/en-us/azure/search/search-what-is-azure-search
- #5: Indexers are available for Azure Cosmos DB, Azure SQL Database, Azure Blob Storage, and SQL Server hosted in an Azure VM.
- #6: High Density tier is targeted at SaaS providers who build applications which support a large number of relatively small indexes in a single search service Max indexes for partition - 1000 (max 3000/service)
- #10: https://docs.microsoft.com/en-us/azure/search/index-add-suggesters
- #11: https://azure.microsoft.com/en-us/blog/autocomplete-in-azure-search-now-in-public-preview/
- #12: This syntaxis is default https://docs.microsoft.com/en-us/azure/search/query-simple-syntax
- #13: Отличие Azure-овского от классического Lucene синтаксиса только в отсутствии range ( mod_date:[20020101 TO 20030101] – вот так в Azure Search нельзя ) https://docs.microsoft.com/en-us/azure/search/query-lucene-syntax
- #14: https://docs.microsoft.com/en-us/azure/search/query-simple-syntax#not-operator
- #15: Stem (англ.) – основа, стебель, происхождение Лемма (лингвистика) - каноническая, основная форма слова Стемминг – использует алгоритмы (зачастую обрезает слова удаляя суффиксы и окончания, получая основу слова) Лемматизация использует поиск по словарям содержащим различные формы слов
- #16: Lucene's English and Microsoft's English analyzers are better than default
- #22: Odata filter could be used additionally with simple or full query syntax used in a search parameter https://docs.microsoft.com/en-us/rest/api/searchservice/support-for-odata https://docs.microsoft.com/en-us/azure/search/query-odata-filter-orderby-syntax
- #23: 2 варианта для SQL Второй только для tables и не для таблиц у которых составной первичный ключ
- #24: https://docs.microsoft.com/en-us/sql/relational-databases/track-changes/work-with-change-tracking-sql-server?view=sql-server-2017 https://docs.microsoft.com/en-us/sql/t-sql/statements/set-transaction-isolation-level-transact-sql?view=sql-server-2017 SET TRANSACTION ISOLATION LEVEL SNAPSHOT; BEGIN TRAN -- Verify that version of the previous synchronization is valid. -- Obtain the version to use next time. -- Obtain changes. COMMIT TRAN
- #26: Soft delete if HighWaterMarkChangeDetectionPolicy was selected
- #27: https://docs.microsoft.com/en-us/azure/search/search-get-started-nodejs https://docs.microsoft.com/en-us/azure/search/search-get-started-java
- #29: Sentiment анализирует текст на положительный или отрицательные отзывы/сентименты/эмоции
- #30: generate tags for image, or identify celebrities and landmarks .JPEG .JPG .PNG .BMP .GIF .TIFF