Proximity Targeting at Scale using Big Data Platforms
0 likes57 views
The document outlines a presentation on proximity targeting and its technological ecosystem, highlighting the AIQ platform that optimizes data analytics for real-time marketing outcomes. It details improvements in cost and time efficiency achieved through advanced data processing techniques, resulting in significant revenue generation. Various challenges related to geocoding and data management are discussed, along with the solutions implemented to enhance operational performance.
![8
11:00 am
Your Location
9:00 am
Coffee Shop
10:00 am
Competitor
Location 1
Customer Data - Without a Story
[random & disconnected]
6:30 pm
Gym
1:00 pm
Competitor
Location 2](https://image.slidesharecdn.com/techtriveni2-191127104849/85/Proximity-Targeting-at-Scale-using-Big-Data-Platforms-8-320.jpg)
![19
❑ Shuffle partitions
○ spark.sql.shuffle.partitions = 200
❑ Broadcast joins
❑ Salting
❑ Databricks skew.hints df.hint("skew", "col1")
○ df.hint("skew", ["col1","col2"])
○ df.hint("skew", "col1", "value")
from pyspark.sql.functions import broadcast
result = broadcast(A).join(B,["join_col"],"left")
Optimizations- Joins, Shuffles & More](https://image.slidesharecdn.com/techtriveni2-191127104849/85/Proximity-Targeting-at-Scale-using-Big-Data-Platforms-19-320.jpg)
Proximity Targeting at Scale using Big Data Platforms
- 1. Proximity Targeting at Scale Presented By : Rohan Raj ( Lead Engineer, MIQ )
- 2. 2 Agenda Proximity Targeting Tech Ecosystem, Challenges & Optimizations Results
- 3. 3
- 4. 4 Activating Marketing Intelligence through AiQ AiQ is our technology that provides modular, API-based analytics services to rapidly build data solutions for successful real-time business outcomes. As a result, we take a process that might normally take a couple of weeks and reduce time-to-value to a couple of hours. CONNECT DISCOVER ACTION Onboard, unify and store any dataset, making data organized, useful and meaningful Perform advanced analytics and process datasets for insights & algorithmic deployment Export decisions to your marketing technology platform AiQ
- 5. 5 Daily Scale genda 80 Billion Ad Impressions 5000+ Strategies 10+TB Data 900,000 CPU mins 1000+ Campaigns 750 million users
- 6. 6
- 7. 7 Proximity Targeting & Hyperlocal
- 8. 8 11:00 am Your Location 9:00 am Coffee Shop 10:00 am Competitor Location 1 Customer Data - Without a Story [random & disconnected] 6:30 pm Gym 1:00 pm Competitor Location 2
- 9. 9 11:00 am Your Location 9:00 am Coffee Shop 10:00 am Competitor Location 1 1:00 pm Competitor Location 2 6:30 pm Gym Coffee enthusiast and maybe stays in the vicinity of the coffee shop to pick it up before he goes on with his day User visited the Client location at 11am along with a bunch of other competitor locations, so looks like he was actively shopping Competitors who offer similar products like the client who customers also visit Time when he visits the gym if we wanted to capture time to target users to consume fitness related products. He travels 10km to get to the gym from the coffee shop SATURDAY Customer Data - With a Story
- 10. 10 Motion - Capabilities User Journeys Audience Targeting Visit Trends : Unique, Repeat Precise Uplift Rank Store (Footfall insights) Competitor conquesting
- 11. 11
- 12. 12 Big Data Processing Ecosystem
- 13. 13 Major Technical Challenges how we overcame these
- 14. 14 Geocoding OLC ● It’s designed to be used as a street address. (similar to phone numbers) ● Not able to provide needed precision, when radii around POI varies widely. ● Nearby places don’t have shared prefixes. ● Looks Like this - 87G8Q257+5QP Geohash ● Scalable and efficient for programmatic usages and joins. ● Provides the needed precisions when size of POIs or target physical stores vary enormously. ● Hierarchical ● Nearby places have shared prefixes. ● Looks Like this - dr5ru7v
- 15. 15 Managing Parallelism - Skew cost is involved
- 16. 16 Managing Parallelism - Overheads
- 18. 18 Optimizations- Joins, Shuffles & More Registration Make Model Engine_size AB12CDE Ford Fiesta 1.0 FG23HIJ Ford Fiesta 1.1 KL34MNO Ford Fiesta 1.5 PQ45RST Nissan Qashqai 1.6 UV56WXY Hyundai i20 1.4 ZA67BCD Ford Mondeo 2.0 Make Model Engine_size Price Ford Fiesta 1.0 10110 Ford Fiesta 1.1 2500 Ford Fiesta 1.5 13653 Ford Fiesta 1.4 16700 Ford Fiesta 1.2 8965 Ford Fiesta 1.6 7320 Nissan Qashqai 1.5 14567 Nissan Qashqai 1.6 11432 Partition 1 Partition 2 Partition 3 Partition 4 ... A B
- 19. 19 ❑ Shuffle partitions ○ spark.sql.shuffle.partitions = 200 ❑ Broadcast joins ❑ Salting ❑ Databricks skew.hints df.hint("skew", "col1") ○ df.hint("skew", ["col1","col2"]) ○ df.hint("skew", "col1", "value") from pyspark.sql.functions import broadcast result = broadcast(A).join(B,["join_col"],"left") Optimizations- Joins, Shuffles & More
- 20. 20 Cluster optimization & Persistence ● Spot Nodes ● Heterogeneous clusters ● EBS-autoscaling
- 21. 21 Results 20K $ → 4.5K $ Cost 21 Hours →3 Hours Time 30 M $ / yearly Revenue generated Cluster optimizations shuffle partitions geohashingskew & join optimization
- 22. 22 ❑ https://medium.com/miq-tech-and-analytics/geohash-vs-open-location-codes-understanding-the-tw o-geocoding-techniques-in-advertising-space-8002452201c8 ❑ https://medium.com/miq-tech-and-analytics/understanding-customer-lifetime-value-in-retail-62eb62 f44994 ❑ https://medium.com/miq-tech-and-analytics/decision-trees-optimizations-for-programmatic-media-buying-e 77e4410bfe8 ❑ https://medium.com/miq-tech-and-analytics/meet-jarvis-79ad50f3ccb8 ❑ https://medium.com/miq-tech-and-analytics/https-medium-com-nagaraj-mediaiq-how-we-use-s3-select-for- schema-validation-and-filtering-data-at-miq-52cf036bf9be Further Readings ...
- 23. 23 Thank You It is a long established fact that a reader will be distracted by the readable content of a page when looking at its layout.