![Indexing Data
● Iterate over all creatives
● Call our GraphQL server
with a query
● Put the result into
Elasticsearch
query getCreativesById($creativeIds: [UUID]) {
creative(creativeIds: $creativeIds) {
id
...AllCreativeFields
show {
...AllShowFields
titleRanking {
...AllTitleRankingFields
}
}
comments {
...AllCommentFields
}
}
}](https://image.slidesharecdn.com/graphqlsummitpresentation-191101011730/85/GraphQL-Search-9-320.jpg)
GraphQL Search
- 3. How can we quickly search and aggregate data spread across multiple sources?
- 4. Roadmap Indexing the Data Keeping It Updated Data Flow Learnings
- 5. Marketing Tech at Netflix Creatives
- 6. Our Data query getAllCreatives() { creative { id ...AllCreativeFields show { ...AllShowFields titleRanking { ...AllTitleRankingFields } } comments { ...AllCommentFields } } } Creative Service Show Service Title Ranking Algo Comment Service
- 7. 50 Million
- 8. Roadmap Indexing the Data Keeping It Updated Data Flow Learnings
- 9. Indexing Data ● Iterate over all creatives ● Call our GraphQL server with a query ● Put the result into Elasticsearch query getCreativesById($creativeIds: [UUID]) { creative(creativeIds: $creativeIds) { id ...AllCreativeFields show { ...AllShowFields titleRanking { ...AllTitleRankingFields } } comments { ...AllCommentFields } } }
- 10. After Our Data is Indexed ● Sort, group, and filter the data on arbitrary fields ● Provide type-ahead suggestions ● Implement faceted search ● Display results with infinite scroll ● Our page loads much faster
- 11. Roadmap Indexing the Data Keeping It Updated Data Flow Learnings
- 12. Keeping the Index Up To Date ● If the creative changes, then reindex that creative ● if the show changes, then reindex the creative with that show id ● If the title ranking changes, then find the related show, use the show to find the creative, and reindex it ● if a comment changes, then reindex the creative with that comment id If our data changes, we need to reindex
- 13. GraphQL Defines Relationships in Our Data Title Ranking Show Creative Comments
- 14. Auto Search Indexer Title Ranking Show Creative Comments 1. Change event
- 15. Auto Search Indexer 1. Change event 2. Fan out one level Title Ranking Show Creative Comments
- 16. Auto Search Indexer 1. Change event 2. Fan out one level 3. Search all indexed documents Title Ranking Show Creative Comments Elasticsearch
- 17. Auto Search Indexer 1. Change event 2. Fan out one level 3. Search all indexed documents 4. Rebuild and reindex documents Title Ranking Show Creative Comments
- 18. Auto Search Indexer 1. Change event 2. Fan out one level 3. Search all indexed documents 4. Rebuild and reindex documents rebuild search fan out change
- 19. Roadmap Indexing the Data Keeping It Updated Data Flow Learnings
- 20. Data Flow: Before Indexing BrowserGraphQL aggregation on page load (read) Creative Service Show Service Title Ranking Algo Commenting Service
- 21. Data Flow: After Indexing BrowserGraphQL Creative Service Show Service Title Ranking Algo Commenting Service Elasticsearch aggregation on data change (write)
- 22. Data Flow: Abstract Elasticsearch BrowserGraphQL Creative Service Show Service Title Ranking Algo Commenting Service Elasticsearch GraphQL can choose its data source
- 23. Search becomes a plugin that you can enable on a GraphQL server with minimal configuration
- 24. Roadmap Indexing the Data Keeping It Updated Data Flow Learnings
- 25. Initial Setup of the Search Indexer Search Indexer GraphQL Kafka Elasticsearch 1. Use GraphQL - graph edges must be bidirectional 2. Listen to change events 3. Index to a search database
- 26. 1. Define a GraphQL query 2. Done How to Build a New Index
- 27. Search Indexer Performance Fan out Search Rebuild+ +Change + < 500ms =
- 28. More Information on Monday medium.com/netflix-techblog
- 29. GraphQL Search Indexing ● Robust search and instant page loads ● Minimal refactoring on the client side ● Configuration driven and automated +