Lessons From Spider Support

1. Lessons Learned from 11,000 Spider Support Tickets OLIVER BRETT, HEAD OF STRATEGY, SCREAMING FROG

8. Lots of Languages = Lots of Languages Two Languages = Bilingual One Language = English

9. Lots of Languages = Polylingual Two Languages = Bilingual One Language = English

15. http://taylortomita.com/seo-graph-portfolio/ @TVYLORTOMITV

18. 3 Rules For Today:

19. 1. This isn’t a sales pitch.

26. 2. This isn’t a walkthrough.

28. https://www.screamingfrog.co.uk/seo-spider/tutorials/

29. https://www.screamingfrog.co.uk/seo-spider/tutorials/

36. 3. Just ask if you need help!

38. 1.Was ist dein Lieblingsfacht? 2.What is Screaming Frog? 3.Spider Support 4.What People Ask Us

39. Part One:What isScreaming Frog?

42. Screaming Frog Log File Analyser

44. Screaming Frog Log File Analyser Screaming Frog SEO Spider

47. Why is it called ‘Screaming Frog’?

49. Screaming Frog is built by SEOs, for SEOs.

58. Screaming Frog (usually) runs locally. But that’s a good thing.

63. “The Swiss Army Knife of SEO” (IT’S CUTTING EDGE…)

64. https://www.seerinteractive.com/blog/screaming-frog-guide/

70. Screaming Frog Can…

71. Timeline: V.1(2011)-V.16(2022)

72. V1.0 (2011) LAU N CH

73. V2.0 (2012) Word Counts URL Rewrites Crawl Control

74. Nobigupdates (2013/2014)

75. V3.0 (2015) Tree View Insecure Content Audits Image Sitemaps + Enhanced XML Sitemaps

76. V4.0 (2015) Google Analytics Integration Custom Extraction

77. V5.0 (2015) Webmaster Tools (Google Search Console) integration Better Robots.txt Auditing/Control

78. http://taylortomita.com/seo-graph-portfolio/

79. V6.0 (2016) JavaScript Rendering Rendered Crawling Crawl Indexed Sitemaps SERP Emulator

80. V7.0 (2016) Fetch and Render Custom Robots.txt Rules Hreflang Web Form Authentication

81. V8.0 (2017) Made it look nice(r) More APIs Custom Profiles

83. V9.0 (2018) Database Storage Store HTML and Rendered HTML Better Search Functions

84. V10.0 (2018) Scheduling Visualisations CLI (Headless Mode)

85. V11.0 (2019) Structured Data- Aud it, Valid ate and Expor t

86. V12.0 (2019) Page Speed Insights API Configurable Tabs Custom Search Improvements

90. V13.0 (2020) N ear Duplicate Content Spelling & Grammar

91. V14.0 (2020) Dark M ode Google Sheets Expor t Cookies

92. V15.0 (2021) Crawl Comparison Change Detection

94. V16.0 (2021) Improved JavaScript Auto Crawl Repor ts for Data Stud io Translated U I

95. V16.0 (2021) Improved JavaScript Auto Crawl Repor ts for Data Stud io Translated U I

96. Lots of Updates… V1.0 -2011- Launch V2.0-2012- Word Counts, URL Rewrites, Crawl Control V3.0- 2015- Tree View, Insecure Content, Image Sitemaps V4.0- 2015- GA Integration, Custom Extraction V5.0- 2015- GSC Integration, Better Robots.txt V6.0- 2016- Rendered Crawling, Crawl Indexed Sitemaps, SERP Emulator V7.0- 2016- Fetch and Render, Custom Robots.txt, hreflang, Web Form Authentication V8.0- 2017- Made it look nice(r), More APIs, Custom Profiles V9.0- 2018- Database Storage, Store and Render HTML, Better Search V10.0- 2018- Scheduling, Visualisations, Headless Mode V11.0- 2019- Structured Data- Audit, Validate and Export

99. Part Two: Spider Support

101. Slide about Ewa on the phone

102. Slide about Ewa on the phone Ewa

105. Blah blah javascript blah blah html?

106. Blah blah javascript blah blah html? What?

108. Spider Support

109. Spider Support 0.25€

110. 48% of queries replied to within 1 hour

114. 97% within 24 hours

118. 0 5000 10000 15000 20000 25000 30000 35000 2015/16 2016/17 2017/18 2018/19 2019/20 2020/21 2021/22 Messages Received

119. Lessons Learned from 11,000 Spider Support Tickets OLIVER BRETT, HEAD OF STRATEGY, SCREAMING FROG

120. 1.6 1.65 1.7 1.75 1.8 1.85 1.9 1.95 2015/16 2016/17 2017/18 2018/19 2019/20 2020/21 2021/22 Replies to Resolve

121. November is our busiest month

122. Screaming Frog™ Log File Analyser™!!!11!!

123. Screaming Frog Log File Analyser Screaming Frog SEO Spider

126. Part Three: People Always Ask

127. Many spider support questions are secretly SEO questions

130. People who don’t People who like pie chats

131. People who speak German Dieser Junge auf der Bühne.

132. Useful Information

133. I looked at 108 common topics we get asked about

134. All 108 Topics

135. 108 topics ranked by no. of conversations

136. 15% licence

137. 15% licence 11% license

140. 15% licence 11% license 6% crawling

141. Operating Systems

148. 25% Mac 13% Ubuntu 7% Linux 56% Windows

154. Search Engines

156. 95% Google 2% Yandex 2% Yahoo 1% Bing

158. Functions

159. 17% Memory 12% Storage 9% Install 6% Opening 38% Crawling

161. Adjusting Your RAM Increasing allocated memory

162. Changing Storage Choosing the right mode

163. MemoryModevs Database Storage Mode Memory Mode: Crawl is saved to RAM as you go Works on older PCs More likely to explode If your PC shuts down… good luck Beyond several 1,000’s you might struggle… Database Storage Mode: Crawl is saved right to your SSD/HDD You need a faster SSD ideally Can save the crawl if your computer goes to sleep/crashes Can crawl millions of URLs

164. Modes

165. 58% Spider Mode 32% List Mode 9% Command Line

166. 58% Spider Mode 32% List Mode 9% Command Line 1% SERP Mode 

169. Technical Terms

170. 10% User Agent 9% Redirect 14% JavaScript 7% Speed 8% Security

173. 10% User Agent 9% Redirect 14% JavaScript 7% Speed 8% Security 2% hreflang

174. 10% User Agent 9% Redirect 14% JavaScript 0.1% AMP 7% Speed 8% Security 2% hreflang

175. 10% User Agent 9% Redirect 14% JavaScript 7% Speed 8% Security 2% hreflang

176. 10% User Agent 9% Redirect 14% JavaScript 0.1% AMP 7% Speed 8% Security 2% hreflang

177. https://www.mediavine.com/amp-is-dead-miley-cyrus-was-only-loosely-involved/

178. Status Codes

181. 404

182. 200

183. 301

184. 302

185. 500

187. 30% 200s 22% 404s 14% 301s 11% 302s 10% 403s 4% 503s 4% 410s 3% 500s 2% 502s

188. 200

189. 30% 200s 22% 404s 14% 301s 11% 302s 10% 403s 4% 503s 4% 410s 3% 500s 2% 502s 36% 4XX 30% 200 25% 3XX 9% 5XX

190. 404

191. Jobs

192. 19% Custom Extraction 17% Custom Search 13% Large Site 2% Migration

194. Yes, there’s a guide for that!

200. Content

203. 46% Images 17% Page Title 12% Meta Descriptions 9% Alt Text

204. 46% Images 17% Page Title 12% Meta Descriptions 9% Alt Text 4% Meta Keywords

209. <Heading Tags>

210. H1 H2 H3 H4

211. 60% H1 27% H2 13% H3

213. APIs

214. 48% Google Analytics 24% Search Console 11% ahrefs 10% PageSpeed Insights 8% Moz

215. Review PageSpeed View stats and opportunities to improve

217. We scraped all the headlines we could…

219. (And you can use the Moz API to bulk-check the DAs of coverage.)

220. Other SEO Tools

221. 28% Moz 17% SEMRush 6% DeepCrawl 4% Sistrix 4% Sitebulb 38% ahrefs 3% OnCrawl 1% Ryte

223. Cheeky Asks

224. 71% Discount 26% Stickers 2% T-shirt 1% Hoodie

227. Slide about Ewa on the phone Ewa

231. The Fan Club

232. 6% Chocolate 45% Beer 27% Wine 4% Whiskey 2% Gin 1% Pimms

233. 6% Chocolate 45% Beer 27% Wine 4% Whiskey 2% Gin 1% Pimms 15% Egg

235. @OliverBrett @ScreamingFrog Danke!

Lessons From Spider Support

More Related Content

Similar to Lessons From Spider Support (20)

Recently uploaded (20)

Lessons From Spider Support