Fixing common problems with SEO by Steve Mortiboy

Editor's Notes

• #2: Welcome Introduction
• #3: In this session we’re going to cover how to identify and troubleshoot common problems with SEO We’re going to use a selection of tools which you will all want to become familiar with These include essential tools such as Google Search and Google Webmaster Tools as well as problem specific tools such as Chrome Developer Tools, Why No Padlock and Moz Local Search Tool I’ll show you how to use these tools to identify problems so you can then fix them on your sites
• #4: So first let’s quickly go over how Google works Google crawls websites looking for specific information as well as indexing the content on the website The first file that Google checks when it crawls a site is the robots.txt file This is a file that is either dynamically generated by WordPress or a plugin or it could be a static file in the filesystem on your web server It tells crawlers and robots where they cannot go It also tells them where they can find your XML Sitemap It’s important because if it is misconfigured then you could block search engines from seeing important files on your site or even block them totally from your site The XML Sitemap is typically a dynamically generated map of your site that lists all of the URLs for your content and when each page was last changed and how often your content changes Search engines use this file to help them determine which content needs to be indexed since their last visit and how often they should visit your site Search engines then check the source code of pages they crawl to look for meta information They may or may not use this information depending on their preferences Finally they look at your content and index this content so that people using their search engine can find your content
• #5: So when Google looks at a page of content on your site, it checks for various meta tags The two most important are the title tag and the meta description The title tag should be an accurate and concise title for the page It is a critical element in SEO and every page of content on your site should have a title tag, this includes posts, products, custom post types, archives, etc. The meta description should be an accurate and concise description of the content of your page It is also a critical element on SEO, and although they are not as important in rankings they are important for influencing someone to click on your result in search results Both the title tag and meta description are used search results to create the snippet that you see in the image above
• #6: In addition to the title and description meta tags, there are a number of other meta tags search engines check for These include robots meta tags which give direction to the search engine crawlers Noindex is by far the most important of these Noindex asks or tells search engines not to index this page of content This is important if you’re wondering why your content doesn’t appear when you search for it Nofollow tells search crawlers not to follow any links on that page This effectively makes the page a standalone and could mean search engines don’t crawl and index content linked from that page, especially bad if this is on your homepage NOODP and NOYDIR will be discussed next
• #7: The most frequently asked question I get, and I get asked this every week, is why isn’t Google showing my title or description in search results So let’s look at why this happens and how we can troubleshoot this
• #8: Google has specific quality guidelines which cover title tags and meta descriptions If you don’t follow these guidelines then you reduce the chance of Google using your title and description Even if you follow these to the letter you should understand that Google does not have to use your title tag or meta description in a search snippet The generation of search snippets is completely automated and takes into account both the content of a page as well as references to it that appear on the web The goal of the snippet is to best represent and describe each result and explain how it relates to the user's search
• #9: Another thing to be aware of when you’re troubleshooting why your title or description isn’t appearing in SERPS is that Google may not have crawled that page since you set or changed the the title or description To check this we can perform a site search in Google We do this by typing site colon in front of our domain name We can then click on Search Tools to reveal an extra filter menu Selecting Past Year will restrict results to anything indexed in the last year And sorting by date will show latest indexed pages first Here we can see that Google last indexed our blog 5 days ago So if I had made a change to the title or description on my blog page more recently than that well the new title and description would not have been picked up by Google yet so I’d have to wait until they next update their index for my blog page A final note about the date that appears in Google search results A lot of people ask whether you can remove that date, the answer is no Google will pick up that date and display it because they want to and there’s no way to force them to remove them
• #10: Troubleshooting title tag and meta description issues is helped by using Google Webmaster Tools If you go to Search Appearance, HTML Improvements you will see a report showing any issues Google has found with your titles and descriptions You can then resolve these issues However, this report does not update very often and so new issues don’t show for a while and resolved issues have a habit of remaining in the report for a while after you have fixed them It’s strongly recommended that you check this report every couple of weeks but you should really get in the good practice of writing good titles and descriptions that comply with Google’s quality guidelines Our All in One SEO Pack plugin is a great tool for managing titles It helps you by showing you which pages don’t have titles and descriptions It shows when a title or description exceeds the character limits It brands your titles for you It has checkboxes to enable NOODP and NOYDIT site wide So all you need to do is focus on writing accurate and concise titles and descriptions
• #11: Here are some useful links to help you with troubleshooting title and description issues
• #12: When you look at a web page, news article, blog post or any other content you expect to see a title at the top that tells you what the page is about This title is exceptionally important to us as consumers of content As we read content we like to see long content broken up into sections with clear titles We like to see structure to content and structure to titles with titles and sub titles Well it’s the same for search engines They pay attention to our use of titles In WordPress these are the H1 through to H6 headings we can select from the button bar above the content editor But there are rules to how we use these The most important of these rules is that every page should have a page title That title is the H1 heading There should only be one page title for any page of content and therefore there should only be one H1 heading on any page Headings in content should use H2 headings for primary headings and H3 headings for secondary headings We should never mess with this hierarchical structure The biggest problems I see with headings is with themes that do not correctly implement H1 headings for page titles Always check your page source for problems and fix any bad headings in your theme
• #13: One of the worst problems we run into is that search engines cannot or have not index content because they are blocked There are many ways you can block a search engine from indexing your content These include: Using the robots.txt file Using the NOINDEX meta tag Using a directive in your htaccess file Using the Search Engine Visibility setting in WordPress Or by using some plugins
• #14: This is my recommendation for a WordPress robots.txt file It shows the URL for the XML sitemap It have two disallow statements The first blocks crawlers and robots from going in the cgi-bin directory This is system directory found on many servers and there’s no reason for search engines to go in there The second is for the xmlrpc file I block search engines from that file because they have no reason to index that file Note that I do not block them from any of the WordPress directories This is in line with a statement put out by Google back in July 2015 where they needed access to core JavaScript files so they could index some site content
• #15: We can check for blocked resources in Google Webmaster Tool under Google Index, Blocked Resources We can also test our robots.txt file under Crawl, Robots.txt Tester And we can also perform a Fetch and Render under Crawl, Fetch as Google All on these tools will help you to identify if Google is blocked from content, files or scripts on your site that it needs in order to render and read your content You should become familiar with each of these tools
• #16: The Crawl Errors report can be accessed in Google Webmaster Tools by going to Crawl > Crawl Errors This report will show a list of URLs which Google crawled in the past but received an error when they last tried to crawl that page The report shows the URL, the HTML Error Code they received and the date they crawled the page and got the error
• #17: Crawl errors happen for several reasons. These include: You changed the permalink structure on your site without setting up redirection Doing this could affect the URL structure of your site which means that Google would get a 404 page not found error when trying to crawl your site on the old URL structure Or it could be that you changed the permalink for a specific post or page For example, you changed the page structure by moving a top level page to become a child page This changes the permalink for that page resulting in a 404 page not found error when trying to crawl the page on the old URL Or that you changed category structure which could affect the URLs for posts Another example would be you changed the slug for a page For example you changed /contact to /contact-us If Google indexed that page with the URL /contact and tries to visit that URL now it would get a 404 page not found
• #18: Another cause is that you have broken links in the content of your site For example, you typed a URL incorrectly Or you linked to a page or post and then deleted that page or post and so now the link is broken
• #19: Another cause is that you have broken back links to content on your site from other peoples sites For example, they typed the URL for your site incorrectly Or they linked to a page or post that has since been removed and so now the link is broken
• #20: Other causes can include server issues, DNS issues and plugin or theme issues Typically these issues will result in a different error code than 404 page not found For example, if the site is unreachable due to a server or DNS outage then the error code would indicate this
• #21: To troubleshoot a crawl error we first need to know the cause of the error We can determine this by clicking on each error in the table This will launch a popup that shows the error details and source The first tab shows the error details This is a blog post that is resulting in a page not found 404 error We can see the date that Google reported the problem
• #22: If the sitemaps tab shows then this would indicate that the URL was found in the XML Sitemap This is rare if you use a plugin such as All in One SEO Pack which dynamically generates the XML Sitemap whenever it is requested A dynamically generated sitemap is created based on the state of your site at that second and it lists all the URLs for your content as they are at that second This means they are typically extremely accurate
• #23: The Linked From tab will show where Google found the link that is resulting in the 404 error In this example the bad link to my post appears on multiple pages on my site This tab would show if the link is a backlink found on someone elses site It would list all the URLs of those external sites that are back linking to your post
• #24: The best way to fix crawl errors is to preempt them I use the Broken Link Checker plugin on every site I manage This plugin scans your site regularly and notifies you when it detects a broken link on your site This way you can fix the issue before Google crawls the link and reports the crawl error Another plugin I use religiously is Redirection There are many redirection plugins available so find your favorite Any time you make a change to your site that might affect the URL structure of the site or a page or post you should set up a 301 redirect to redirect old URLs to their new URLs You can also use this to redirect bad backlinks so that Google will get a valid page when it crawls a back link on someone elses site Using Broken Link Checker and Redirection should enable you to find and fix problems with URLs on your site, in your content and back links from other peoples sites Once you’ve fixed a crawl error, make sure you mark it as fixed by clicking the blue button in the popup Then make sure you check back often so you pick up new errors and fix them speedily
• #25: Bad SEO on a site can lead to bad back links Back linking schemes used to be all the rage with SEO’s a few years ago until Google started getting wise to abuse and started penalizing sites for backlinks generated purely for SEO reasons An example of a bad backlink is content posted on a free ezine site by an SEO just for the purposes of creating a backlink to your site The content has no value and is only there for the backlink If you’ve suffered from this type of bad SEO then there’s a report and tool you can use in Google Webmaster Tools The report is found under Search Traffic > Links To Your Site and shows the sites who link to you the most Check this report every so often and if you see a site linking to you that you think is bad or spammy then you can disavow this sites links
• #26: To disavow bad backlinks, follow the instructions in Google’s documentation in the first link shown here Use the Disavow Links tool which is found at the second link and is shown here It may take a while for Google to remove the bad backlinks so be patient
• #27: We all know how much the mobile web has changed over the past few years Google has recognized this and now places greater importance on mobile usability of sites in their ranking Because of this, they’ve added a Mobile Usability report to Google Webmaster Tools It can be found under Search Traffic > Mobile Usability This report shows issues that Google has detected with mobile usability on your site It shows which pages have a problem and what the problem is
• #28: Clicking on an issue and then on a page from the list of pages where the issue was found launches a popup The popup gives us access to Google’s mobile testing tool
• #29: You can use Google’s Mobile Testing Tool to check your site to ensure it is mobile-friendly But you’ll also find lots of other great mobile testing tools available If you ask around designers and developers they’ll all have their favorite tools they like to use My particular favorite is the Chrome Developer Tools inspector
• #30: Make sure you check the Mobile Usability report and fix any issues as soon as possible
• #31: We’ve already identified that the XML Sitemap is important to search engines Issues with the XML Sitemap can often go hidden for a long while, after all, how often do you look at your XML Sitemap However, Google will tell us in Google Webmaster Tools when they have discovered any issues with our XML Sitemap We can see this by going to Crawl > Sitemaps Issues will show under the Issues column
• #32: Clicking on the issues count will display the Sitemap Details page which will show us a description for any issues In this example the warnings are related to a 404 page not found for a attachment image in a trashed post This is obviously something we can ignore, especially as it was detected back in June The other warning is regarding a time out when trying to crawl a tag archive Timeouts can happen because of a slow server or server outage so they tend to be transient issues and are resolved the next time Google tries to crawl again If we can browse to that page and as long as Google doesn’t continue to report the issue then we can ignore it If Google continues to have a problem reaching that page then it could be an issue with blocked resources so we can check the Blocked Resources report and troubleshoot accordingly
• #33: Structured data, often referred to as Schema.org markup is special markup we can add to source code of our web content It’s used by Google so that they can display rich snippets in search results Some examples of these rich snippets that you’ve probably seen are: Products where Google shows the price of the product and whether it’s in stock Star ratings for a product or something else Recipe information that shows an image or video, the cooking time and number of calories Or event information that shows date, venue and ticket price
• #34: Here’s an example of product markup and a rich snippet in Google search results We can add this markup to our sites using plugins for each purpose An e-commerce plugin such as WooCommerce would output the markup for products An events plugin would output the markup for events Some plugins support Schema.org markup, some do not
• #35: When Google detects a problem with our structured data, they will notify us in Google Webmaster Tools under Search Appearance > Structured Data We can check this report for problems and then resolve the problems Most problems with WordPress sites are related to either HATOM feed or HCARD issues This is markup that comes from your themes and most themes don’t implement the markup fully or correctly Fixing these errors usually involves contacting the developer that created the plugin or theme that is outputting the markup and reporting the issue to them
• #36: Google also provides a Structured Data Testing Tool which you can use to check your site for issues It should be noted that this tool has undergone some recent changes to reflect Google’s preference for the newer JSON LD markup It doesn’t always provide the same results as the Structured Data report in Google Webmaster Tools This is one of those aspects of your site where you may well need to call on a knowledgeable developer for help
• #37: Speed is a very important ranking factor Google likes pages to load in under 3 seconds Page load times can be affected by many factors including: The speed of your internet connection The size and speed of your server What type of hosting you are on The file size of images that load on a page Whether you call any external scripts or resources such as fonts, videos, maps and social media buttons You can check the speed of your site using Google’s PageSpeed Insights Tool This tool will give you standard recommendations on how to speed up your site I have found the recommendations to be the same for almost every site I’ve tested
• #38: These are the standard recommendations I see every time with Google’s PageSpeed Insights tool You can implement a caching plugin on your site or your hosting company may implement caching on the server You defer scripts that aren’t needed for rendering above-the-fold content until later in the page load This can be very difficult to do because most scripts are added by plugins which control when they are loaded You absolutely should optimize your image file sizes Use lossless compression to get the file size of images down as small as you can without impacting the quality of the image Consult your hosting company about server speed If you’re on a shared hosting plan then consider investing in a VPS instead Use a plugin to minify your CSS and JavaScript Just be careful to test thoroughly to make sure minified scripts don’t break Avoid using any redirection from one page to another Check with your hosting provider that GZIP compression is enabled for HTTP requests Some of these are just good practice, some are out of your control and some come down to who you host with I just find the recommendations of the PageSpeed Insights tool to be very generic
• #39: I prefer to test page load times using Chrome Developer Tools If you right click on any page and select Inspect from the right click menu then click on the Network tab and do a hard refresh of your page you’ll see a report like this The magic number is in blue at the bottom In this example the page loaded in 3.12 seconds which is perfectly acceptable We can no check the list of what was loaded to see if there’s anything in red Items in red indicate a problem such as a time out or the item failed to load In my example here I have some external JavaScript from embed-player.com related to a video that failed to load This may be something I can fix by loading the video from a different source It may be caused by the way I embedded the video or by a video plugin I’m using I typically run this test with all my plugins deactivated to get a baseline I then activate each plugin one at a time and see if any specific plugin causes the load time to increase significantly I can then decide whether I need that plugin or can remove or replace it Don’t forget that the speed of your Internet connection and load on your web server can affect this report Always check the very first line in the report which shows the initial page request to the server If that changes significantly then it’s likely due to server load or network speed
• #40: Back in August 2014, Google announced that they would start using HTTPS as a ranking signal The goal was to encourage a safer Internet by rewarding website owners for implementing HTTPS Implementing HTTPS and an SSL certificate is fairly straightforward But ensuring that our page is secure and we get the little green padlock can be more challenging Luckily there is the website Why No Padlock which we can use to detect if there are any problems with HTTPS on a page Run the URLs for each page through this tool and it will tell you whether there is anything that needs to be resolved in order to get the green padlock symbol in your browser
• #41: Finally I’m going to address common issues with local SEO If you’re running a local business where you want to rank specifically for your area then you want to make sure you have the correct information on your sites and other sites The first thing is to ensure you have your Name, Address and Phone number, referred to as NAP, on your site Ideally this should be added using the correct Schema.org markup There are several sites online that will generate this markup for you, just search for NAP markup generator Make sure you have business opening hours on your site and a Google Map of your business location Next, absolutely make sure you have one official approved Google Business listing Google Business is what used to be Google Places and it merged with Google Plus Your listing here is the absolute authority for your local SEO information so the information on all other sites, including your own must match what is on Google Business Make sure you do not have missing, incomplete or conflicting business listings out there on the web
• #42: Luckily there’s a tool provided by Moz.com that helps us find inconsistencies with business listings This tool will show us what listings there are for our business and which ones are incomplete, inconsistent or duplicated It’s then a process of going through each listing and claiming the listing and correcting any problems This is painstaking and time consuming work and there are services out there that will do this for you if you pay This is an important process and one you should follow and verify regularly