Q2 SEO & AI Update: How To Track & Optimize AI Search Performance
1 like2,507 views
Tracking your SEO performance on AI-driven search engines like ChatGPT, Gemini and Perplexity can ensure your data is accurate and increase visibility for your business. Watch as we dive into practical, technical strategies for monitoring and optimizing your visibility in AI-generated search results. You’ll learn: - How to track your performance on AI-powered search engines like ChatGPT and Perplexity. - The best tools and techniques for monitoring AI-driven search rankings. - How to set up scrapers - both manually and programmatically - to analyze AI search visibility. With insights from Ariel Shulman, VP of Product at Bright Data, we’ll walk through a live demo showing how to track AI search results and extract meaningful insights. We’ll also share actionable strategies to improve uptime, ensure data accuracy using APIs, and prepare for future SERP disruptions.
Q2 SEO & AI Update: How To Track & Optimize AI Search Performance
- 2. 2 The Global Leader in Public Web Data Serving 70% of leading eCommerce Marketplaces 20,000+ Customers including Fortune 500 companies 5,500+ granted patent claims 400+ employees spread worldwide GDPR & CCPA compliant 750+ partner organizations - Accelerating social impact work through the use of public web data.
- 3. The Internet is the world’s biggest database “Public Web Data - Belongs To The Public” It should be available to anyone regardless of how you choose to access it. * some companies don’t agree!
- 4. 4 In 2024, Bright Data won court cases against Facebook (Meta) and Twitter (now X). It cost $M… but these legal precedents help define the future of the web data industry. Leading the fight for public web data
- 5. 5 Intro: The Landscape of Search is Changing What is changing? And how can we adapt? As a result of GenAI, the search ecosystem is changing; 1. Answers provided are different based on user intent - transactional or research? 2. Platforms change: Google/Bing/others are evolving, and new AI tools offer web search 3. Users change: from individuals to AI agents.
- 7. 7 Answers Are Different - and marketers need to adapt Queries are changing - People ask different questions! From simple keywords → to english sentences/questions, sometimes paragraphs! AI’s Human-like answers → no need to visit site? AI-generated answers change search user experience - giving instant 1-step answers (perplexity ad…) Dynamic Structure changes = harder for experts Technical challenges of extracting and structuring AI-generated responses
- 8. 8 Queries are changing - and any language works!
- 9. 9 Nice AI experience? Too bad for website owners Poll: Are you seeing the same trend?
- 10. 10 Extracting AI answers - what does it take? Bright Data’s Scraping Cloud
- 11. 11 Technical challenges when working at scale Before Jan-25: you needed Proxies, fingerprints, headers… After: even this was no longer enough!
- 12. 12 We kept everything running… Bright Data Scraping success rate during Jan-25 change request->response Browser+JavaScript required Success rate on google.com 2am.. Few hours! And Bright Data kept prices same :)
- 13. 13 Getting the page is just step #1… Unpredictable page structure! PPC? Organic? Related? Major impact on CTR… What triggers the AI view? AI generated
- 14. 14 Demo #1 - Extract data from Google SERP with Gemini
- 15. 15 What can YOU do to help AI index your content? Build up a large FAQ section! * Clear, concise content that directly answers common questions as users ask them. Use natural language and conversational tone. Keep your content FRESH * AI uses signals such as publish date, frequency of updates to prioritize data used to generate answers Structure your data correctly * Use clear headers (H1, H2, etc.) to help AI understand. Use H1 for primary keyword, and H2 for subtopics. Use bullet/number lists. Make your content indexable Ensure your site is mobile-friendly, has a sitemap & its robots.txt file and meta tags allow indexing by all AI engines. docs.brightdata.com saw traffic increase from citations in ChatGPT
- 17. 17 Search Platforms Are Different ● Input: Text ● Little context/memory ● Gives links ● Requires additional navigation ● PPC ads ● Input: Text, voice, image ● Lots of context/memory ● Gives answers (any language) ● In many cases - 1 step ● No ads
- 18. 18 How do queries impact search? Transactional and Navigational queries are still being asked on search. The long tail queries shifted to Answer Engines. Research is being done by Agents on search engines but the user consumes the answer from the Answer Engine. 01 02 03 “Mazda 3 fuel consumption” “Which car is more reliable? Mazda 3 or Tesla 3?” “Which car has lowest cost over a year including gas, insurance and maintenance? Split by electric and gasoline cars” Types of queries Mix AI
- 19. 19 What do Brands need to monitor in Answer Engines? Brand Visibility Mentioned in key queries accurate brand descriptions Recommendation Rankings Included in “top”, “best” lists in your category Competitive Comparison “Is My Brand better than Brand Y”? 01 02 03 AI search doesn’t RANK pages - it generates answers. You need to stay visible!
- 20. 20 Demo #2 - Extracting Perplexity responses
- 21. 21 Demo #2 - User experience & response on ChatGPT What you see… What happens under the hood…
- 23. 23 Consumer is Different AI Agents are searching the web Less people and more AI Agents are accessing and searching the web, extracting data from multiple results There’s more info outside of search engines! AI agents search way beyond Google (News sites, YouTube, Reddit, high-quality publications) Time to change Businesses should adapt to AI-driven search behaviors How do AI agents collect information? Do they get blocked?
- 24. 24 ChatGPT AI Agent operator getting blocked
- 25. 25 Demo #3 - AI Agent Search + Crawl → Aggregate Response
- 26. 26 Tech Summary: The Path to a Robust Search Scraping Mimic Browser Fingerprints Set Referral Headers Manage user agents Solve CAPTCHAs Render JavaScript Handle Cookies Access from different Geos Auto-retry & IP rotation Parse HTML to structured data Request Bright Data
- 27. 27 Summary As an SEO/Marketing tool Understand what data you need to provide your customers with to provide value, Search engines aren't enough anymore. As a marketer Understand how search trends are changing and that you need to adapt to stay afloat. The technology is out there! Search marketers expect to see it from SEO Platforms.
- 28. Thank you!