Brighton SEO Sept 2019 PowerShell

@Fearless_Shultz #brightonSEO
Utilizing the power of
PowerShell for SEO
Mike Osolinski // Technical SEO Consultant
SLIDESHARE.NET/MikeOsolinski
@Fearless_Shultz

https://www.slideshare.net/MikeOsolinski/command-line-automation-for-repetitive-tasks

What I’m Going to Talk About
Today
A Brief Introduction to
what PowerShell is, it's
components and
interface
Some Examples of Using
PowerShell to Automate
SEO Tasks
Extending PowerShell Beyond
It’s Native Capabilities

Why Automate SEO Processes?

Spend Less Time Gathering and Formatting Data and More Time
on Analysis

Make your business more profitable by being able to do the
same work in a shorter period of time

Allow teams to easily handle greater workloads without having
to recruit new staff

Have an Easier Life…

So Why Don’t More People
Automate Processes?

Lack of Awareness of the potential time savings and that many
repetitive tasks can be easily automated

Fear either that scripting is too hard for you to learn and you
won’t be able to do it or that you might break something
*Spoiler – It’s not and You Will. Everyone does

Not having / wanting to spend time learning to write scripts

Why PowerShell?

Already Installed on Most Windows Computers so a lack of
complicated set up processes

Open Source Meaning it is Cross Platform and has a Large
Development Community

Makes Working with Data Easy and Reduces the Need to Write
Complex Formulas and Expressions

Highly Extendable Via Gallery of Custom Modules

What is PowerShell?

PowerShell Gives You Superpowers!!!

Actually, PowerShell is a task-based command-line shell and scripting
language built on .NET which helps system administrators and power-
users rapidly automate tasks that manage operating systems and
processes
But that doesn’t sound as cool…

So Why Should SEO’s Care?

• Scrape Websites
• Automate Website Interactions
• Run Screaming Frog Reports
• Consume Rest APIS
• Extract and Manipulate Data
• And Much More……..

Getting Started

https://www.peppercrew.nl/index.php/2014/06/powershell-
coding-on-a-mac/
https://wilsonmar.github.io/powershell-on-mac/
https://docs.microsoft.com/en-
us/powershell/scripting/install/installing-powershell-core-on-
linux?view=powershell-6
https://docs.microsoft.com/en-
us/powershell/scripting/install/installing-powershell-core-on-
macos?view=powershell-6
https://www.starwindsoftware.com/blog/using-powershell-
on-linux

Scripting Panel
Command Line
Cmdlets

Cmdlets

Command Line
A cmdlet (pronounced "command-let") is a lightweight Windows PowerShell script that performs a
single function.
Invoke-WebRequest Import / Export CSVInvokeRestMethod

Command Line
https://www.powershellgallery.com

Invoke-WebRequest

The Invoke-WebRequest cmdlet sends HTTP and HTTPS requests to a web
page or web service. It parses the response and returns collections of
links, images, and other significant HTML elements.
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-6

$w = Invoke-WebRequest -Uri
'https://moz.com/learn/seo/what-is-seo'
$w | Get-Member
$w = $w.RawContent
$w

$w = Invoke-WebRequest -Uri 'https://moz.com/learn/seo/what-
is-seo'
$w | Get-Member
$w = $w.Links
$w

is-seo'
$w | Get-Member
$w = $w.Links |
Select innerHTML, href |
WHERE innerHTML -like *SEO*
$w | Export-Csv 'C:scriptslinks.csv' -NoTypeInformation

innerHTML href
Free SEO Tools https://moz.com/tools
https://moz.com/learn/seo
https://moz.com/training
https://moz.com/beginners-guide-to-seo
SEO Learning Center https://moz.com/learn
Free SEO Tools https://moz.com/free-seo-tools
What is SEO? https://moz.com/learn/seo/what-is-seo
On-Site SEO /learn/seo/on-site
Local SEO /learn/seo/local
Mobile SEO /learn/seo/mobile
International SEO /learn/seo/international
Beginner's Guide to SEO /beginners-guide-to-seo

is-seo'
$w | Get-Member
$w = $w.Links |
Select innerHTML
$w | Out-File 'C:scriptslink-text.txt'
Get-Content C:scriptslink-text.txt |
New-WordCloud -Path C:scriptswordcloud.svg -ImageSize 1080p

https://github.com/vexx32/PSWordCloud

$currentRequest = Invoke-WebRequest -Uri
https://moz.com/learn/seo/what-is-seo
$currentRequest.AllElements

is-seo'
$w.AllElements | where tagname -EQ "p" |
select innerText |
where innerText -Like *seo*

is-seo'
$w.AllElements | where tagname -EQ "p" |
select innerText |
where innerText -Like *seo*|
Out-File 'C:scriptsscriptfilesparagraphs.txt'
Get-Content C:scriptsscriptfilesparagraphs.txt |
New-WordCloud -Path C:scriptsscriptfilesparagraph-cloud.svg
-ImageSize 1080p

$path = 'C:scriptsscriptfiles'
Import-Csv -Path 'C:scriptsscriptfilesseo-competitors.csv'|
ForEach-Object{
foreach ($property in $_.PSObject.Properties)
{
$currentRequest = Invoke-WebRequest -Uri $property.Value
$pageTitle = $currentRequest.ParsedHtml.title
$currentSavePath = $path + $pageTitle + '.txt'
$currentSavePath = $currentSavePath -replace("?","")
$currentSavePath = $currentSavePath -replace("/","")
$currentSVGPath = $currentSavePath -replace("txt","svg")
$paragraphs = $currentRequest.AllElements | where tagname -EQ "p"|
select innerText|
Out-File $currentSavePath
Get-Content $currentSavePath | New-WordCloud -Path $currentSVGPath -ImageSize 1080p
}
}

Invoke-RestMethod

The Invoke-WebRestMethod cmdlet sends HTTP and HTTPS requests to a
web page or web service. It parses the response and returns collections
of links, images, and other significant HTML elements.
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-restmethod?view=powershell-6

$pageSpeedData = Invoke-RestMethod -Uri
'https://www.googleapis.com/pagespeedonl
ine/v5/runPagespeed?url=https://www.brig
htonseo.com/'

'https://www.googleapis.com/pagespeedonl
ine/v5/runPagespeed?url=https://www.brig
htonseo.com/'
$pageSpeedData.lighthouseResult.audits

'https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url
=https://www.dailymail.co.uk'
$offScreenImageCount =
$pageSpeedData.lighthouseResult.audits.'offscreen-
images'.details.items.count
for($i = 0; $i -le $offScreenImageCount; $i++){
$element = $pageSpeedData.lighthouseresult.audits.'offscreen-
images'.details.items.getvalue($i)
$element | export-csv 'C:scriptsoffscreen-images.csv' -
notypeinformation -append
}

totalBytes wastedBytes wastedPercent url requestStartTime
43337 43337 100 https://i.dailymail.co.uk/1s/2019/09/04/14/18057384-0-image-m-2_1567604298582.jpg 363649.4834
33206 33206 100 https://i.dailymail.co.uk/1s/2019/09/05/01/18085400-0-image-a-1_1567644632414.jpg 363649.2957
13063 13063 100 https://i.dailymail.co.uk/i/pix/2018/10/03/tips_v3_opt1_308.png 363649.4832

Import-Csv 'C:test-urls.csv' |
Foreach-Object {
foreach ($property in $_.PSObject.Properties)
{
$currentURL =
'https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=' +
$property.Value
$testResults = Invoke-restMethod -Uri
$currentURL$testResults.lighthouseResult.audits.'render-blocking-
resources'.details.items
$arrayCount = $testResults.lighthouseResult.audits.'render-blocking-
resources'.details.items.Count
for($i = 0; $i -le $arrayCount; $i++){$element =
$testResults.lighthouseResult.audits.'render-blocking-
resources'.details.items.GetValue($i)
$element | Export-Csv 'C:scriptsrender-results.csv' -NoTypeInformation -
Append
}
}

https://www.programmableweb.com/apis/directory

+ +

# run the screaming frog crawl
cd "C:Program Files (x86)Screaming Frog SEO Spider"
$env:Path = $env:Path + ";C:Program Files (x86)Screaming Frog SEO Spider"
$startingURL = "https://mikeosolinski.co.uk"
ScreamingFrogSEOSpiderCli.exe
--crawl $startingURL
--headless
--save-crawl
--output-folder “C:scriptssf"
--export-tabs "Internal:HTML,Response Codes:Client Error (4xx)"
https://www.screamingfrog.co.uk/seo-spider/user-guide/general/#commandlineoptions

#create an excel document to contain the data
$xl = New-Object -ComObject Excel.Application
$xl.Visible = $true
#add a workbook to the excel object
$workbook1 = $xl.WorkBooks.Add()
#add the sheet to contain long page titles
$lngPageTitles = $workbook1.Worksheets.Item(1)
$lngPageTitles.Name = 'Long Page Titles'
#create the column headers for long page titles
$lngPageTitles.Cells.Item(1,1) = 'Address'
$lngPageTitles.Cells.Item(1,2) = 'Status'
$lngPageTitles.Cells.Item(1,3) = 'Title Length'

#import the title length data from the screaming frog crawl
$titledata = Import-Csv -Path ‘C:scriptssfinternal_html.csv'|
select Address, Status,'Title 1 Length'|
where Status -eq 'OK'|
where 'Title 1 Length' -gt 55
#iterate through the records and add data to the sheet
$i = 2
foreach($record in $titledata)
{
$titledata.Address
$titledata.'Title 1 Length'
$lngPageTitles.cells.item($i,1) = $record.Address
$lngPageTitles.cells.item($i,2) = $record.Status
$lngPageTitles.cells.item($i,3) = $record.'Title 1 Length'
$i++
}

https://github.com/mikeosolinski/powershell/blob/master/screaming-frog-excel

https://github.com/mikeosolinski/powershell/blob/master/site-speed-data

# CHANGE THE URL BELOW TO THE ONE THAT YOU WANT TO TEST
$urltotest = 'https://mikeosolinski.co.uk'
#update the path to save files to from C:SiteSpeedDataTOOL-OUTPUT to whatever you want. avoid spaces in
path/filenames
#any questions on how to run this ping me on twitter at https://twitter.com/Fearless_Shultz
https://github.com/mikeosolinski/powershell/blob/master/site-speed-data

Extending PowerShell

Install-Module -Name GoogleMap -Scope CurrentUser
https://geekeefy.wordpress.com/2016/05/17/powershell-module-for-google-map/

$env:GoogleGeocode_API_Key = “INSERT API KEY"
$env:GooglePlaces_API_Key = “INSERT API KEY"
("Brighton, UK" | Get-GeoCoding).coordinates | Get-NearbyPlace -Radius 1000
-TypeOfPlace hotel

Install-Module -Name PSTwitterAPI -Scope CurrentUser
https://github.com/mkellerman/PSTwitterAPI

Import-Module PSTwitterAPI
#set keys
$apiKey = INSERT KEY HERE'
$apiSecret = INSERT API SECRET HERE'
$AccessToken = INSERT ACCESS TOKEN HERE'
$tokenSecret = INSERT TOKEN SECRET HERE'
# Provide Authentication for the Twitter API
# https://twittercommunity.com/t/how-to-get-my-api-key/7033
Set-TwitterOAuthSettings -ApiKey $apiKey -ApiSecret $apiSecret -AccessToken
$AccessToken -AccessTokenSecret $tokenSecret
# Get user twitter profile
$followers = Get-TwitterFollowers_List -count 200
$followers.users.name

#set keys
$followers.users

#set keys
$followers.users|
select name, screen_name, description|
where description -Like '*SEO*'
Export-Csv 'C:scriptsseo-followers.csv'-NoTypeInformation

Resources

https://github.com/mikeosolinski/powershell/ My Github profile which contains commented versions of all of the scripts mentioned
https://docs.microsoft.com/en-us/skypeforbusiness/set-up-your-computer-for-
windows-powershell/download-and-install-windows-powershell-5-1 The download page to install PowerShell
https://www.powershellgallery.com/ PowerShelll Cmdlet Gallery
https://www.adamtheautomator.com/invoke-webrequest-powershell/ Guide to Invoke-WebRequest
https://www.gngrninja.com/script-ninja/2016/7/8/powershell-getting-started-
utilizing-the-web Another guide to invoke WebRequest
https://stackoverflow.com/questions/11885246/how-do-i-loop-through-a-line-
from-a-csv-file-in-powershell Loops in PowerShell
https://www.youtube.com/watch?v=PXBMdIkH24I Associative Arrays in PowerShell
https://vwiki.co.uk/MySQL_and_PowerShell Accessing MySQL With PowerShell
https://mcpmag.com/articles/2018/08/08/replace-text-with-powershell.aspx Finding and Replacing Text
https://techblog.dorogin.com/generate-word-documents-with-powershell-
cda654b9cb0e Generating Word Documents with PowerShell
https://www.powershellbros.com/powershell-tip-of-the-week-create-invoke-
webrequest-from-chrome/ PowerShell and Google Chrome
https://docs.google.com/spreadsheets/d/1psz6SvRqv7fjIFIiAySPNshObMINjjFeCvGqTjluGLs/edit?usp=sharing

Closing Thoughts

PowerShell is Not Just for Systems Administrators and
Network Admins

PowerShell is An Incredibly Versatile Tool and you are
Only Limited By Your Own Imagination

Even If You Don’t Want to Learn to Script Yourself you
Should Understand the Opportunities

But. . .

You CAN do it

https://mikeosolinski.co.uk
http://twitter.com/Fearless_Shultz

Brighton SEO Sept 2019 PowerShell

More Related Content

What's hot (20)

Similar to Brighton SEO Sept 2019 PowerShell (20)

Recently uploaded (20)

Brighton SEO Sept 2019 PowerShell