When the internet first dawned, HTML was king when it came to building websites. This simple coding language was used to develop websites of all kinds, but it was just that: a simple language.
In the late 1990s and early 2000s, website development moved from simple infographics and text to high-powered interactive graphics like videos and usable tables, especially with eCommerce’s growing prevalence and necessity. This new era of web content demanded a more powerful programming language that could provide users with an enhanced experience.
The simpler your website content is, the easier it is for Google or Bing to render and index everything on your pages. When you start to layer in interactive elements like visual animations, navigation, and eCommerce facets and filters, you ask Google’s robot web crawlers to do heavier lifting as they work to render and index your website in search results.
The problem, essentially, is that Googlebot isn’t inherently just going to do that extra work.
React was created by the developers at Facebook to help with the building of their own platforms. It’s also important to note that technically, React is a library and not a true-to-definition framework. However, it is becoming so prevalent in its use that this guide would be remiss to not discuss it.
We stockpiled a list of our favorite SEO tools that can help you with your SEO overall; for JS audit specifically, we like:
First and foremost, let’s address the elephant in the room (or on the screen).
Each makes it more difficult (or impossible) for search bots to access your website and/or your website’s critical content. What’s “critical” to access?
To check if your URLs and the content is available and indexable, use a crawler like Screaming Frog or DeepCrawl to mimic a search engine bot, and create a dashboard of URLs & the data you found on them. This will tell you which pages are indexable, and which aren’t, and why that might be.
If URLs you know exist (and should be indexed!) aren’t found, that’s a problem, too.
It’s a relatively common issue for URLs to be available, but for the page copy, navigation links, and in-page content links to be hidden/inaccessible.
Both of these cases are full dealbreakers for SEO; even if Googlebot wanted to crawl and index your pages, your site quite literally isn’t structured to make that possible.
The final cause of JS issues has to do with your website’s rendering, especially if you are utilizing client-side rendering.
Generally speaking, it is less expensive (in terms of fiscal payments to your hosting provider) to utilize client-side rendering since it uses less server space and less programming.
A third hybrid rendering option is also available (in fact, there are many variations!) But generally speaking, hybrid rendering and prerendering utilize both client-side and server-side rendering. It allows Googlebot to easily access your content through selective server-side rendering, while also utilizing client-side rendering for human access when SEO needs are less important.
The “cost” of rendering can mean timeout errors for Googlebot, OR it can simply create an unequal trade-off, wherein Googlebot decides that the value your page brings is too low to be worth the work of rendering, storing, and indexing it.
In this scenario, Googlebot can access the critical URLs and content - your site just makes it too cumbersome for Googlebot to bother in most cases. This isn’t inherently a “deal-breaker” for SEO - depending on the value of the brand as a whole and the page's specific purpose/value - but it is a real concern that can dramatically limit your ranking potential in many use cases. Learn more about why this is & how it could impact your site.
Here are four main potential outcomes or scenarios that can help you in prioritizing JS issues:
The best-case scenario is that everything “shows up” without any errors. This is a good sign that Googlebot has full and immediate access to your content. This includes looking for navigation links being present, text content, images, and videos (they’ll be ugly, but that’s okay!)
Things you probably won’t see but that’s ok:
This middle-to-sometimes fine scenario occurs when all important content is accessible for search engines, and upon rendering, everything shows up correctly. (e.g. the work of rendering falls to search engines.)
When will they want to? When the overall site, and the specific page in question, warrants that additional work.
When won't they? When they don't see the need to; e.g. when other resources - perhaps with more valuable backlinks, or better content - already cover that same subject "well enough".
Do note that in many cases, this is absolutely fine. We simply encourage you to ensure that your site as a whole, and your individual pages, are as high-quality as possible. Otherwise you may run into the dreaded "Crawled but not indexed" errors in GSC.
The middle-case scenario is that some, but not all, content shows up in your source code, with or without errors. The errors you might see at this level is likely a reflection of your rendering. While your content may show up in Google’s rendered code, it still needs to be rendered. This could be causing Googlebot to be unable - or unwilling - to access your content at the time of the crawl.
The types of content that we usually see fall in this space:
* Please, oh please, just stop using sliders. And on that note, accordions or tabs are NOT always the solution to long pieces of content. Stop making the user work harder to consume your yummy content!
A middle-to-sometimes bad scenario is when there’s no content showing up in the source code, but Googlebot can access all/some critical elements upon rendering.
How can you double-check which scenario your pages fall into? By using the tools mentioned above (e.g. Mobile-Friendly Tool, rendered DOM vs. source code, browsing with JS turned off), you can usually get a clear view into the severity of the problem at hand. When critical elements are entirely missing, it's usually a bad- to worst-case scenario that should be considered a high priority.
For this, we suggest using a Chrome plugin called Wappalyzer. This tool can help you determine if the site is using React, Angular, and a whole slew of other technologies that could be ultimately affecting the crawlability/renderability of your website.
Understanding your stack will help you better identify what and how much (e.g., to what extent of) auditing is needed.
There are two primary scenarios to consider in identifying your tech stack.
Wappalyzer is effective in generally uncovering your site's technology stack, including the CMS, Frameworks, or Programming Languages being used. If you're unfamiliar with the information revealed, a simple Google search is recommended.
If one of the categories says "JS Library," you can safely deduce that it is, in fact, a JS site. However, in the cases of the latter scenario (non-JS nor SPA), a visual inspection of interactive elements via the bookmarklet may suffice.
Sitebulb's Website Auditing Tool is another great way to identify the tech stack, perhaps as a part of your normal SEO auditing process:
This report actually leverages Wappalyzer to pull this data, and is available when you use the Chrome Crawler in Sitebulb.
While above scenarios are foundational variables, there are a number of other technologies that can affect crawlability. Some of these include, but are not limited to:
Once you have a clear picture of what your tech stack looks like, the next step is to roll up your sleeves and start auditing the JS issues plaguing your site’s SEO.
Typically, these crawl reports will show you more immediate issues, such as:
The goal behind this first audit is to identify big-picture issues that can be resolved globally. In some cases, you'll be able to pinpoint problematic macro patterns that you can dig into more granularly in the next step.
Pro Tip: The new version of Screaming Frog - 16.0 - has new enhanced JS crawling and free Data Studio dashboard template you should absolutely check out:
On last PRO TIP here: don't panic if you see 307 (temporary) and 308 (permanent) status codes - these redirect types are of growing popularity.
The intention behind this step is to visually analyze and investigate issues at the page level. To get started, you'll want to determine the critical pages to prioritize auditing. These are typically your most popular/top-performing pages as well as an example page of each major template/layout type.
Once you've determined the top-performing pages and critical templates driving your site, the next phase is to audit those pages and document any issues that you find in the process. How you do so can vary, but we recommend the three following tools for this stage:
Scan a URL using Google's mobile-friendly tool and select the HTML tab. Review Googlebot rendered code with the question "can Google access this critical content?" in mind. Ideally, all critical elements should be present and thereby accessible to Google. But if you don’t see it here ("it" being critical SEO elements like copy, internal links, markup, etc.), then neither does Google. You can use this insight to dig into what's not showing, and why that might be.
This is a very handy Chrome extension that enables you to compare a page's source code vs browser-rendered HTML side-by-side. View Rendered Source makes it efficient to spot any missing elements that Google can’t access, which in turn, are limiting your site’s SEO performance.
These tools are not mutually exclusive, nor do they all have to be utilized at this stage. You may simply prefer one tool over the other, and use it heavily, while bringing in one or another tool on an ad-hoc basis for additional information (e.g. when you know something is wrong, but you just can’t figure out what or why with the initial tool.)
In the process of auditing JS SEO problems through the steps above, you'll likely identify issues that need to be documented and fixed. This may involve passing your findings along to a development team, which means providing clear information behind the what, where, and why of the issue.
This troubleshooting step is designed to equip you with the tools and processes needed to effectively pinpoint and communicate the JS issues impacting your SEO. Depending on your engineering team, however, this step may not be required.
By referring to both a page's source code and rendered DOM, you'll be able to better define and articulate discrepancies that are causing problems. Here’s what that means:
Source code: When you “View Source” while on a browser, you’ll get the HTML that was delivered to the browser initially. This is referred to as the “source code.”
Rendered DOM: Use the ever-popular “Inspect Element” toolset or just open up your Dev Console (we recommend Chrome, as it’s the most popular browser). Also known as the “rendered HTML,” this will show you how the browser has interpreted that HTML, and perhaps even applied fixes that need to be made.
Coming Soon: All About Inspect Element & Troubleshooting for Site Issues and SEO.
You’ll typically want to do this for any interactive elements on the page (e.g. drop-down menus, tabbed content, and the like), and/or any elements or sections that you identified as problematic in Steps 2 and 3.
Some of the most commonly applied fixes that you should look out for include:
These are just a handful of overarching problems we often come across. For a related read on this topic, see other common JS SEO issues and how to overcome them. In the next step, we discuss more about prioritizing fixes and communicating them to your engineering team.
Based on your findings, you'll want to prioritize issues by the importance of page type AND the criticalness of what’s missing. For example, random JS errors are far less detrimental than Googlebot not finding product page links from category pages on a webstore (as that’s the core purpose of that page, and without it, Google can’t easily find or pass page rank to the products you want to sell). Utilize the JS SEO Issue Scale outlined above in determining how big of an issue the actual JS error itself is.
In addition to prioritizing which fixes need to be made first - on what pages (or page templates), it's very helpful to segment your content by function or type. For instance, you can use your audit's visual analysis and crawl data to organize content into the following four categories:
This framework also helps in determining rendering issues based on layout features/functions, meaning you won’t have to audit every single page. Rather, you can typically just audit each module or presentation of content. In doing so, it helps to:
The tickets themselves should be grouped by page and/or page type, and content function, as outlined above.
Like all good tickets you submit to your software development team, please ensure you include the following information:
Once you're on the same page with your dev/engineering team, it's time to carry everything over the finish line and ensure it's fixed, and fixed for good.
This step is really about getting the fixes deployed, QA’d, and over the finish line.
Upon initial QA testing, note that you may need to pull in additional tools to help confirm a fix. Some tools - specifically the Mobile Friendly Tool - are only usable for publicly available pages (e.g. you can’t run it for non-indexable stage URLs.)
Changes to high-priority SEO content should also be reviewed for quality assurance. Make sure that any new content a) still exists, and b) meets (or exceeds) your expectations based on what you’re seeing.
While it may seem obvious, specific items to make sure to check include: page titles and meta descriptions, Hreflang and internationalization features, on-page copy, internal links, images, videos, and other media types.
Equally as important as fixing problems is ensuring they don't surface again in the future. There are a few tools and measures we recommend to help with ongoing QA and performance monitoring.
Post-fix SEO quality assurance testing and performance monitoring isn’t rocket science. It just takes having the right toolkit and knowing where to look. While automating as much as you can never hurts, referring back to your initial auditing tools is often a failproof way to manually test for high-priority ticket items.
Looking to get to know the vast neighborhood that is SEO, in all its forms and functions? We’d love to show you around. At The Gray Dot Company, we specialize in working as partners and external consultants with various types of SEO and digital marketing teams, and we provide advanced capabilities and experience in the complex realm of technical SEO.