The "Learn JavaScript for SEO Cohort" was a group learning experiment led by our own Sam Torres in conjunction with others from our beloved Women in Tech SEO community. The following is the final, recorded presentation of the group's learnings, as led by Sam. In it, we'll review the 8 most frequently occurring issues with JavaScript SEO we've personally encountered—as well as how to resolve them.
Video Transcript:
All right—so this is just something I put together for the cohorts about things that I've run into, things that I see. Ideally to try and help! So hopefully it does, if it doesn't, I apologize.
So just some things to know about this. And so that, like I said, this is just a summary of my experiences, the things I've commonly run into. It's definitely thought of from an SEO first perspective, and less from me "wearing my developer hat".
Video Transcript:
I will tell you, it's not going to help you learn JavaScript development, it's not a comparison of the different frameworks, it certainly does not cover all possible topics, that would be pretty much impossible. Then there could also be mistakes. So these are just things that worked for me. But it doesn't mean they're perfect. And also things update—so if we refer to this in a year, things might be different. So just -just so you know.
Slide Text:
What this presentation is:
What this presentation is not:
Video Transcript:
As far as the frameworks we're going to take a look at, we're gonna look at—I'm calling it foundational JavaScript, it's usually just JavaScript sites, or in development, we typically call that "Vanilla JavaScript." React, and then they have a tool called Gatsby. And then we're going to talk about Angular and Vue.
So React, Angular, and Vue are the three most popular JavaScript-based frameworks. But there are dozens if not hundreds more. I wanted to cover these three, since these are the ones that we'll typically be running into as SEOs—who just come across JavaScript sites.
Slide Text:
Frameworks we're discussing:
Please know that there are dozens more JS Frameworks—these are just the ones I've most run into (and/or coded myself)
Video Transcript:
When I'm going through and find out, you know, as far as my process goes with these things, the first thing I always have to do is identify the stack—so I have to identify which technology is being used. I use a Chrome extension called Wappalyzer. Highly recommend it. We can take a look at it after the slides—of just what it looks like.
Also, anytime you need it, and it can make you sound really impressive, because clients will be like, how in the world did you know what my website's built on? And you just look like a wizard. So let's all be wizards!
Then I'll do site crawls. So I either use Screaming Frog or Ahrefs with a JS rendering, and look for any kind of errors that may come up. And then also really important is investigating how things are being rendered.
What does Google Search Console have as far as the content for that page? Because there's definitely times where maybe your content's being crawled—but not indexed, maybe some of it's being crawled. You know, really being able to identify where those nuances are is pretty critical to diagnosing your issues.
Slide Text:
My process:
We've all been there.
Video Transcript (2:49):
So to dive into our first problem—so are you seeing that only a portion of your page or content is getting crawled and indexed? So maybe this is like you've gone into Google Search Console, and you see that the page URL is there, but it's only getting about 40% of the content? Chances are, there's a JavaScript error.
And then also, so you guys know, I've put these little like "pills" at the top {of the slide images}, so that I can tell you which framework it applies to.
But when you look in Inspect—and that should be Google Chrome Inspect—basically looking at the Chrome Dev Tools, and then if you see something like this on it, it could be causing a JavaScript error that is making the rest of the page unparsable. As a rule, JavaScript is very unforgiving when it comes to languages. Also, its error handling essentially sucks.
Video Transcript:
Error handling is basically the message that's delivered to you—as far as telling you what's wrong—so really, the only solution you have at this point is to ask your developer, "Hey, what does this mean?" Now, I will say, like this one, if you see an uncaught syntax error—so syntax errors are actually errors, but if you see something at the top of the console, because you'll usually see a few more messages here. If it's at the very top 9 times out of 10, in my experience, it's just an alert and not really an "error."
So just know that your dev might say that, and it's probably pretty founded. Now, of course, developers are inherently lazy—that's why we code things, to do the work for us! So if you have any questions about that, of course, pose the question in Women in Tech SEO, I'm sure—I'm sure you'll get some answers. And then also, just to note, as someone who is a developer, just remember when you're "framing" this and how you communicate, like, "Hey, what does this mean?" It can make someone go immediately on the offensive and not want to help you. You know, as much as we're always having to work with our clients or internal teams to really comfort them and make sure they know that (a)—we are the experts, and they come to us to have those answers, we need to have the mindset with our developers—THEY are the experts.
They just may not know how to implement it according to the way we want to, which is why we have to talk. So that's the first problem.
Slide #1 Text:
Possible Culprit: JavaScript Errors.
{Screenshot of Inspect > Console - Uncaught SyntaxError}
If you're seeing some content in GSC Inspect but not all, you could have a JS error that's causing it to be unparsable (unreadable).
"Find these errors by checking the Console in Chrome Dev Tools while on the page in question."
Slide #2 Text:
Possible Culprit: JavaScript Errors.
JS is unforgiving, and its error handling leaves much to be desired. Solution? Raise it to the developer.
Not the gumdrop buttons!
Video Transcript (5:19):
Next problem, maybe you have entire pages or sections of your site that aren't being crawled. Does anybody actually know what "not the gumdrop buttons" comes from? I need to deliver this to an American audience. It's from Shrek, I have a lot of like, pop culture references in here that I shouldn't have put in, but they're there.
Video Transcript:
So first off, you could have a "bad linking approach." So kind of the same that we would have with any website, if you don't have links to the content—and it makes it really impossible for Google to find it. And then of course, we've talked about this quite a bit, at length, as far as—you have to use anchor tags, and these other methods of like a router link or a span or onClick—those are not functional.
Unfortunately, developers—from everything I can tell that go to school or Bootcamp—they don't really get, or "set a standard" on how it should be used. So to a developer, all of these methods are totally valid! So our job as SEOs is: we need to provide documentation, and then I also have a link to Google's documentation, and I straight up pull the screenshot from there. So that makes it a lot easier. Developers love documentation. If they don't... that's pretty weird.
Slide Text:
Possible Culprit: Bad Linking Approach
*Links are not properly coded to use an anchor and href.
Can follow:
Can't follow:
*No industry-standard for developers on how to make links
*Solution? Provide documentation. Here's Google's.
Video Transcript:
It could also be a Single Page App, or an SPA. Is everybody familiar with what an SPA is? Yes, okay. Um, so, basically, with Single Page Apps—React, Angular, Vue—they all make SPAs at their base level. And, of course, that can be really cool, because you just kind of skip through content, there's no page load events that need to fire. But that does also mean that crawlers aren't seeing the "changing content." So that can be an issue.
Now, just a side note, if you ever have an app—where you actually don't need it to be crawled, but you do want to see it in Google Analytics, you can actually circumvent that problem by feeding "history change events." And so that's not gonna help your SEO, but just a little tracking information, since Google Analytics and data is my other love.
So as far as when you have an SPA, your options for fixing that: there's some rendering options. And then there's also router plugins or extensions for each of the frameworks. And so we're going to talk a little bit about all of those. Any questions so far?
Slide Text:
* Many JS frameworks end up creating a SPA. This makes for super quick experiences for users by skipping the page load step. It just replaces the page with the requested content.
* This means that users don’t actually change URLs, meaning the crawlers can’t see changing content — they only ever see one page.
* Solutions?
"If you’re having issues with seeing pageviews in Google Analytics, you can circumvent this problem by feeding History Change events with the new content request. This will NOT help with SEO however."
Video Transcript:
Next, alright, so for the rendering options, so I put together this chart; these are just my recommended—or what I found that works really well, or consistently come across. So the idea with this is that—depending on the framework, and depending on what style of rendering you'd like to utilize—this would be the type of tool that you could recommend to the development team. So pre-rendering—we're all familiar with that. We have Prerender.io; there's also a plugin for React called React-snap, we're going to talk quite a bit about Angular Universal. And then there's also some pre-rendering web packs and CLI for Vue. So all of those—except for the Prerender.io—are actually plugins or essentially other libraries that would get added to the—to the app that's already there.
And just, you know, anytime using React, Angular or Vue, even if it's a website, it's actually always called an "app" on there. There's straight up an "app tag" that houses everything.
Then you have server side. So of course, that's where all of the calculations are being done—on the server, and not in the client browser. So React does this by using Node, or a plugin library called NextJS.
Angular Universal. So to talk about that a little bit, Angular is actually the framework that Google uses, so they spend a lot of resources and development on making improvements to Angular. There is another kind of sub-library of Angular called Angular Universal, and this puts together both pre rendering options, server side, and really handles a lot of the—what's considered "best practices for SEO." It also has quite a bit of other uses in other kinds of functions—more than just SEO. So it's pretty common that if you have Angular, it's probably going to also include Angular Universal.
However, I will say Angular Universal is more useful in Angular Version 2 versus Version one. We can talk a little bit—basically, if you see someone saying AngularJS, that's Version 1, if they're just saying Angular, that's Version 2. Why it's that way, I have no idea! But—just you know, when you're talking about it.
And then Vue has a library called nuxt.js that can be used for the server-side. Now, with server side, they almost all require some special server setup. Like for the React, you actually have to have Node running on the server—it can sound really scary, but actually, pretty much every major hosting service that can do apps like this, can also support Node. The difference is if you're ever using Azure, or a Microsoft-based server, they have their own unique things going on—it can still be done, but I would just recommend that you probably need to at least get a consultant to help set up that server correctly.
And at that point, that's probably out of SEOs purview, and it's on engineering. Or if you say it's on Windows, maybe you just run, run far away, if you're an agency! Because IIS is the bane of my existence.
Next, we have a Dynamic. So of course, that's one of the ones that we studied from—that's referenced in Google's documentation. So there are a couple tools that do that. And remember that dynamic is where basically—when the query is made, like to pull the page data,—the server looks at who is asking for the data, and will either give the response of the live JavaScript client-enabled version, or they will serve kind of the static version. And that will get served to crawlers and bots.
So that's what Dynamic is. There are a couple different tools—so Prerender.io, again, Rendertron is also a very popular tool use for it, I personally have not used it, I've used Prerender.io. But those are kind of the two biggest ones that are "accepted."
Next, you have Isomorphic. So Isomorphic is probably one of the things that I absolutely love. Um, so Isomorphic is a little bit different than Dynamic in the fact that it will always serve the static one FIRST, and then if it sees that the client—or who is accessing this information—whose browser is capable of doing JavaScript on the client side, it will then serve the JavaScript enabled and interactive. There is no difference to the user, they don't see the change at all. But I really like it because the response time's a little bit faster, because there isn't that amount of logic that needs to happen when the when the request is made, like there is in Dynamic. Remember, in Dynamic, the request is made, the server then says "Oh, who is this?," and then sends—you know, determines what to send. In Isomorphic, the same thing is always sent, and THEN it does the query, and then it'll change content, if it needs to.
Isomorphic is something that's becoming very popular among JavaScript frameworks. React already has it with this plugin called "Isomorphic Render". Angular Universal for Version 2 already does it. Vue, however, does not support any isomorphic rendering options that I've heard of so far. And in order to do it, you have to use server-side rendering, and then at that point, you're actually introducing new technologies to make it happen.
Lastly, we have static site generators. So this is where—if you're familiar with how WordPress works, you basically have your template, the PHP goes to the server, and pulls the applicable content that should be rendered within that template. So essentially, what a static site generator does is—it does the same exact thing, but it's going to go ahead and take all that HTML, create the raw HTML files for you, and then actually, at the end of the day, you're serving these HTML files.
Gatsby is one of those popular ones for React. But there's also Hugo and Jekyll which are making big waves. For Angular Scully is pretty much the only one that can be found, and it's not very popular. So static—so using static site generation is not very popular for Angular users. And then for Vue, you've got Vuepress and Gridsome. So those are some of the biggest ones. But definitely, you're going to—when it comes to static site generators, you're going to see React a lot more than anything else. There's a lot of information—questions?
Speaker 2:
Yes. You—could you say a little bit more about the Dynamic rendering in terms of—it used to be really recommended by Google. And now I sort of hear that it's kind of not recommended anymore, or it's considered like, "Well, we used to recommend that, but now, I don't know." I hear names like hydration—and all kinds of stuff I don't understand—are somehow better than the dynamic rendering? Could you—do you have any opinion you could offer there? Or like, what's the development going in—in what direction is this going?
Sam Torres:
Sure. So everything that's within Google's documentation, because I was doing making sure I had my, my all my ducks in a row, I'll make this presentation. Um, everything in Google's documentation is still supporting Dynamic. However, the development community is definitely supporting more Isomorphic, which uses that Hydration layer. And so that's where you're going to hear a lot about it is usually having to do with Isomorphic.
Note: As of August 2022, Google officially no longer recommends Dynamic Rendering as a permanent (3-5+ year) solution; it should only be used as a temporary workaround.
There's a couple other rendering versions that use Hydration, but they are much less popular than Isomorphic, because it is becoming something that's so—almost natural, to React and Angular. Like it's becoming almost part of like, the default set that anybody would use. So the other reason for that is just, you know, to me, and this is a personal opinion—the reason I like Isomorphic is because it looks at the user, at the time—as they're engaging—but the response has already been made. So they're already seeing something, whether it's a user, like an actual person, or a crawler.
Whereas Dynamic, I personally really worry about, you know, that request, and then the calculation of what to serve. Like, we already know that Google has crawlers and other search engines that don't actually identify themselves as Googlebot, right. So "what are they getting served" is kind of hard to tell. The Isomorphic is identifying whether a user has the capability of client-side rendering versus just who is making this request. So to me Isomophic's a little bit more reliable. Dynamic as well—with edge computing—so basically, right before the requests—I'm sorry—the response leaves the server. Edge computing is becoming bigger, but that's also becoming a little bit of a security risk, meaning somebody could—could potentially screw with that response. Um, so it's just a bit of a security gap that has to be taken into consideration when using Dynamic. Now Prerender and Rendertron, they already have those security measures in place, so you don't really have to worry about that. It's just if you were doing it kind of "all by hand" or custom on your own servers, those are considerations that you'd have to—have to think about. Does that answer your question?
Speaker 2:
Um, yeah, thanks. Um, and really short—if a team is using Vue, and they're already familiar with Nuxt, can you think would there be any reason to change to Dynamic? Or should they stay with Nuxt?
Sam Torres:
So with server side, the biggest thing is the configuration—so if it's already running, I would keep it that way. Because also on server-side, everybody's getting the same kind of content.
Speaker 2:
Yeah, I like that, too.
Sam Torres:
Yeah. So yeah, it's just the setup is where it gets—for... it's probably the most difficult to set up. But if you have it, there's no reason to change it. In my opinion.
Thanks. Any other questions about all the rendering options? Okay, cool beans.
Um, so the other option you have for rendering is also Router, and it's not—I should say—this isn't really an either/or—both should be used. So routers are essentially how a Single Page Apps kind of guide what content should be loaded where.
So there are some router plugins that then provide actual URIs or URLs on so that there are unique locations, unique—like I said—URIs where that content can be accessed. So React uses React Router v4. Pretty much any React project that you have—and if they don't have that, what are they doing? I don't know.
And it also can be used with Redux, which is a state management—we're not really going to get into that, but just know Redux is a pretty popular thing that you may run into. And you can tell if it's there by using Wappalyzer.
Angular, as I said, Angular Universal—so this actually provides that same kind of routing, and "creating URL" functionality. And then in Vue, if it's a newer version of Vue, then you can use History mode instead of Hash mode. And the way to know whether it's using hash or history mode is: first you look at the view version, and if it's greater than two, then it has this configuration option. But basically, when you're changing the content that you're looking at, if you're seeing that there is a parameter changing, or basically you just have the pound sign with the content, that's what changes, then it's in hash mode. If you from that point, it just needs to be put in history mode, and then URLs will automatically be generated.
If it's an older version, so any of the ones—one dots—you can use a plug in called Vue Router. So one of the really nice things about Wappalyzer is it will tell you which version of Vue or Angular is being used.
Slide #1 Text:
Slide #2 Text:
Routers are plugins for JS libraries that help with the URI/URL creation and overall user pathing
React: React Router v4 plugin, Redux
Angular: Angular Universal
Vue: Use “History” mode instead of “Hash” mode when configuring the app, Vue Router
Video Transcript:
Another reason why sections of your site may not be getting crawled is because there's no XML sitemap. So thankfully, for all of these, whether it's React, Angular, Vue, or Gatsby, there are plugins to produce XML sitemaps. Now, sometimes there may be a lot of customization that needs to be done—at that point, you're just going to have to document and work with your developers.
So documentation! Documentation all the time! Um, and then these are the sitemap plugins that I recommend and have used, so I can vouch for their functionality. And also, we're going to talk a little bit later about some of the problems. But basically, whenever a query is run to pull the proper content, you do want the query to filter the content, and not the module that's been rendering how it looks.
And we're gonna talk a lot more about that. So if it doesn't make sense yet—it's okay. So those are the three things that could be happening when it comes to sections of your site aren't being crawled and indexed. Remember, that was (1) you're not linking properly, (2) you have rendering or router issues, or (3) there's no XML sitemap.
Slide Text:
Possible Culprit: No XML Sitemap
Thankfully, plugins exist for each of the major frameworks to automatically create an XML sitemap.
Customizations regarding sitemap exclusions, generating multiple for larger sitemaps, etc. will require some customization from the developers.
Let's see if we can do this in less than 1,000 words.
Video Transcript (22:33):
Okay, so content is being crawled, your images show up on the page when you're looking at it, but your images aren't being indexed. Alright. So the first thing could be and this would be true for any site, not just JavaScript sites. But if your images are being served in the CSS, instead of HTML, they're not going to get indexed. So also note that can be a great thing to have in your tool belt on the other side, because maybe you just have these backgrounds, or these button gradients that you don't want to be indexed: put them in the CSS, you don't have to worry about that!
Um, there's also documentation. So again, I pulled this from Google's documentation, there's a link to it, um, so that you can see how all of that works. Um, yeah, that one's pretty easy.
Slide Text:
Possible Culprit: Images are in CSS
It's likely that images are being served in CSS, rather than in HTML
Solution? Provide documentation. Here's Google's.
Don't worry fam, I got you.
Video Transcript (23:26):
Um, so the next one, all right: but your images aren't showing it all! So remember, the last one was "if you're seeing your images show up when you're looking in GSC Inspect", and this is your images AREN'T showing in the rendering at all. So first off, you need to make sure that you are not blocking crawlers to your asset folders. This is something that's pretty common with JavaScript frameworks—is that there will inherently be some measures taken to reduce the number of requests on assets so that you don't get pinged by nonsensical traffic, or people trying to overload your server. So you just have to make sure that your folder where the assets are stored has the permissions to allow crawlers. The way you can do this is test it in GSC. It will probably tell you whether it can grab it or not—you're gonna have to check your robots. txt and your redirect rules.
Video Transcript:
So are you redirecting that request to the assets folder? That is pretty common. You could—and that would be something where you're redirecting really just other agents than your own is essentially: if your code is making a request, then it's allowed because it's coming from internal. Other—if it's coming from external, then it won't let it render. So that's pretty common. You also have to look at your security protocols and headers, and that one gets pretty advanced and even some of it is out of my own purview. I Google a lot with that. So don't be afraid to Google. Questions on that?
Speaker 3 25:04
I do have a question. Although it's not really related to necessarily only JavaScript-rich websites, but I have been seeing an issue in Search Console, when you check a website. Actually, we have this problem. So on each first hit, it tells me that it cannot "access any resource". So anything on our CDN, for example, so it cannot access. Not everything is on our CDN, but it cannot access CSS files, JavaScript files, you know. And then when I check it again, everything's fair with Search Console. So and then when I check again, it doesn't show again, so you know, so on each, first and third and fifth hit, I get this problem. Whereas it doesn't seem that Google has any problem with our sites at all. I don't have that issue in crawling at all. Because I was at first suspecting we have a problem with caching, we use CloudFlare. But that—it doesn't seem to be a problem. So I was wondering if you had seen this in the past? Happening anywhere, because it's really annoying, because that way, I never know, can I? How much can I rely on what Search Console says about my pages? And why is that happening?
Sam Torres 26:24
Yeah. Um, so I've not seen that. I would—my first thought definitely went to caching as well. Um, I also would wonder about security headers.
Speaker 3 26:37
Yeah, I checked them, but they look okay. So it's really weird.
Sam Torres 26:49
I wonder—it's in the server configuration and whitelisting. That might be something to check out as well.
Speaker 3 26:58
Okay, yeah.
Sam Torres 27:03
Because I've definitely seen security measures where it's like, it doesn't take the first time it recognizes an IP address, but it'll take it the second time—to try to help avoid DDoS attacks.
Speaker 3 27:17
Yes, we do have measures like that in place, although this problem existed even before we had those measures.
Sam Torres 27:23
Well, that is—CloudFlare has some of that too?
Speaker 3 27:26
Yes, exactly, exactly. And before we started using that—we already had that issue. So it did not increase as an issue after we started using protection against DDoS attacks. And the weird thing is—so at first, I thought it's a caching problem, because when the first hit, you don't get this—the resources and you get them on the second hit. But then, on the third hit, you don't get the resources again. So that's just utterly weird.
Sam Torres 27:53
Okay, um,
Speaker 3 27:57
But then it should have been cached. So I collect this.
Sam Torres 28:02
Yeah, that. So it definitely sounds like a server response issue. So, um, let me ask my husband about that. He used to be a server infrastructure engineer. So let me see if he has any ideas where that can happen.
Speaker 3 28:21
Okay, thank you.
Sam Torres 28:22
Yeah. Having him as a husband is awesome, like cheating—at life!
Speaker 3 28:28
I imagine!
Sam Torres 28:31
Yeah, I know, he's a full stack developer. So just all of the things, he answers all of my questions. Um, okay. But great question. If you do find an answer, let us know. And then like I said, I'll update after I talk to Arthur. And we can try to continue diagnosing from there because that's, that's very interesting.
Speaker 3 28:47
I will. I will I have been trying to because we have had this issue for a year, maybe. And I've been trying to get the developers on it. But it was always like—prioritized away.
Sam Torres 28:59
Because we broke it to them.
Speaker 3 29:02
So—exactly. Because they say it seems that this is a Search Console issue. And someone claimed that it's also something an issue that someone else outside of our company is seen as well, but I have never heard of anyone having a problem.
Sam Torres 29:17
Yeah, that's a new one for me. Oh, yeah.
Speaker 3 29:22
I feel suspicious. I still think it deserves looking into.
Sam Torres 29:25
I think so because it also means but—you know, how much—so not getting a response the first time, it could mean that you're not getting that content isn't getting crawled as often as it warrants being crawled?
Speaker 3 29:38
Yeah. Exactly.
Sam Torres 29:38
Um, so, for sure, it could be effects. I would think it's definitely making an effect on the budget and efficiency of that budget. So yeah.
Speaker 3 29:51
Thank you. Awesome.
Slide Text:
Possible Culprit: Blocking Crawlers
Well, that won't do at all.
Video Transcript (29:55):
Sam Torres 29:55
Oh, let's see. Okay. Um, so metadata for pages are missing! So page titles, meta descriptions, robots, directives canonicals. All of that, which really is really about 70% of the issues I run into with JavaScript frameworks. It just means somebody put in the plugin in, and it's not configured properly.
So all of the JavaScript frameworks have these plugins for supporting SEO. Unfortunately, this is also one of those cases where really, you just have to go to your developer, there's nothing that really YOU can do as an SEO, because obviously, something is wrong with the code.
Video Transcript:
So the first thing to do is confirm that the SEO plugin is there, and I have a list of the recommended ones in a second. And then creating documentation for how each tag should be rendered. What type of—for each type of content that exists on your site. So think of all the different post types, whether it's a page or a blog post, or a resource, or like a team—person, whatever, all of that —you're going to have to create all of that. And then the most important part is also mapping to wherever that content is housed—or, so your CMS—what field in the CMS should match to that? It's really painful to write the first time. I know, I've done it. But I will also say, once you write it the first time, you never have to write it again, you just have to make changes as you work on different frameworks or different types of websites, different companies to match obviously, their types of content, but you'll never have to write again, like this is what a title tag should look like. This is where it should go, it should be appended with the brand—you know, all of that good stuff, you never have to do it again.
And then that is also something that I say—regardless of where you are, always take it with you, that is your IP. Yeah, just keep a copy in your own personal Dropbox or Google Drive, or whatever you have—take it with you. Unless, of course, you have contracts that are supposed to say you can't then don't, because I don't want anybody to get legally in trouble. But it's all in your brain. So according to IP law, if you could write it again, you can take it with you.
Um, so if there is no plugin, the plugins that are recommended for each of these—Helmet for React. I'm going to be perfectly honest with the functionality that Helmet brings to React apps: if your developer does not have Helmet installed, and I were the engineering manager, I would fire that developer. Helmet is too useful, it should ALWAYS be used. That might be very close-minded, and maybe there are things I don't understand. But from everything I know, everybody uses it, you're kind of missing the boat if you don't.
Angular, again, it's an Angular Universal. So you're pretty much always going to have that in every Angular app as well. And then in Vue, there are a few different ones out there, the most used and most recommended one, and the one I've used is Vue-meta. Now, when talking about these, this is also a great time to start talking to your developers about how to implement structured data that would actually scale with the post types on.
So I haven't done that with Angular and Vue, but I've done that quite a few times with React, and also with Gatsby—as far as being able to just kind of "tie down" that functionality within Helmet. It's really easy, there are a ton of articles out there on how to do it that you can send to your developer, and it just makes life that much easier. it renders the JSON in the page, so that you can get that structured data yumminess just already baked in.
Slide #1 Text:
Possible Culprit: No SEO/Metadata Vehicle
Slide #2 Additional Text:
She doesn't even go here.
Video Transcript (33:47):
Alright, now we're gonna go into some of the more framework-specific. And then just: a complete disclaimer—when it comes to coding my own projects. I am a React developer. I can muddle my way through Angular and Vue, but React is definitely my chosen stack. So I have a lot more experience in those. So as we go into the specifics, you'll see it's basically just React and Gatsby. That's because that's what I use. It's what I recommend when clients are wanting to build things. So that's, that's what I like. So the React site is using a Default 404 page. So you're seeing that horribly ugly page. We've all seen it, where it's the red text, and a gray box at the top.
Video Transcript:
There's a missing config file in your React app. If you're using the React-Router-4, which of course we recommended earlier—to use that for creating the URIs and URLs that you want, so that there are specific, or very depicted locations, for content. You just need to check with your developers that they've created a RouteJS and ServerJS file. Make sure that there is code in there, and for the design of a 404 page—that's where it needs to go. It's really considered not best practice to go ahead and try to use like a page template, you should actually have like a separate file for the 404 page.
Also, I found that this works extremely well, whenever developers are not getting me an answer on this one, I then asked for read only access to the Git repo, so I can just look at it myself, and then all of a sudden, I "magically" have an answer. So just know, developers tend to not want anybody looking at stuff. Why—I'm not really sure, but there you go.
So like I said, those are the 2 files that you need to make sure on those exist, and then also that the design for the 404 page lives there. I should say that can apply to both React and Gatsby—so it's not just react, it would also happen in a Gatsby site.
Learn how to fix 404 page issues on SPAs for SEO.
Slide Text:
Possible Culprit: Missing Config Files
If working with Gatsby, expect this to crash the party.
Video Transcript (35:55):
Um, so with Gatsby, if you look at if you do Wappalyzer and find that you're working with a Gatsby site, pretty much count on—there's going to be an issue with canonical tags, probably every Gatsby site that I've come into—after the fact, they were never working, right.
Video Transcript:
So chances are, it's because the developers are like, Oh, we need to have canonical tags, and they use this canonical plugin called "Gatsby Plugin Canonical URLs." Everything sounds great, right? Except not, because—that plugin actually builds a canonical on every content piece and it's always self-referencing. So if you actually need canonical tags for the uses of it—of preventing duplicate content, then it actually needs to be hard coded.
This is another great extension for adding to Helmet. There's an article written by another developer who wrote up how to do this far more eloquently than I ever could. So that's linked here. So it's super easy to implement, it is something that should be done custom when it comes to React, because that plugin is just not reliable. And then just make sure that whatever CMS you're using with it, that you actually have a place where you can declare that canonical.
Also, I'd recommend—unless you like doing really mundane, monotonous things—place a fallback for the developers that it would just use the default self referencing, if nothing is there. So if the value is null, or empty in the CMS, it would just reference to itself. So it's just just an idea.
Slide Text:
Possible Culprit: Plugin Dependency
Many developers just install gatsby-plugin-canonical-urls for handling canonicals. Unfortunately this ALWAYS builds with each URL self-referencing.
Solution? The developer will need to hardcode the canonical tag into the template header. There’s even the code that they can use to do that. Make sure they properly connect it to whatever field you have in the CMS for setting the canonical.
It's all scientific, I read it in a book.
Video Transcript (37:42):
Um, so another Gatsby thing, I've got orphaned pages, pages that shouldn't be live, what's happening? So there's a few different things—first off Gatsby, so if you remember static site generators, what they do is they actually go through and render the site as if someone were going through it making all those requests. So while it's doing that, while it's going through the build process, it creates a "build folder." Now there is a build folder for every time it's done. So what you actually see is there's a build folder, and then there's all these sub folders with like 20 character strings, which note when which—which build it was, once a build is finished, then that content is taken over to the public HTML file, which is essentially your website, that's accessible by normal users.
So if the build gets interrupted, there will still—that porting doesn't happen. So there are pages in this build file that never went live, except that folder is still accessible. So you may end up with pages that are like maybe—drafts, or, you know, copies of pages that are actually live on your site. But they're orphaned, because there's going to be no links directly to that build version of it. Those would just stay there, and still be crawlable. Less than ideal! Because if it got interrupted, you're going to have to run the build again.
And then you also need to go ahead and have the build folders kind of cleaned up every now and then—like a routine task. I actually set all of mine up to do a chron job on the server, to clean those up every day. That's something that should be relatively easy for your developers to implement. There could also be—maybe the site gets built, and so you have all the files in the build folder—they get placed over to the public HTML, maybe you're saying that draft content or content that shouldn't be posted yet should be published—is getting—is live!
So an example of this would be like—maybe you have a blog post that's in Draft. And so when you look at the blog roll—or the blog index—that post doesn't show up there. But you can still actually access that blog detail page. Well, probably what's happening here, and I referenced this earlier, is that upon the build, everything's getting built, and only some of the content is moving over to the public HTML.
So your developer may have built some rules between the build folder and the public HTML port—that some of the content shouldn't be moved over to public HTML. So that's why you could have these "in draft" posts that are rendering—know you, the build process actually made them, but they don't actually get posted as "live". So what's happening there is your developer should—could add the Gatsby-plugin-exclude. And essentially what that does is that moves it to—instead of building it, and then not moving it over to the port—Gatsby is just never going to build it. So that's much more important. The Gatsby-plugin-exclude, you can set up all kinds of rules. So is it based on status? Is it based on URL? Yeah, just lots of different qualifications that you can have of why content shouldn't be built.
So that plugin makes it really easy, and prevents kind of all those random pages just showing up that—you will pull your hair out until you figure out what's wrong. And I know that one from personal experience with my own website.
Slide Text:
Possible Culprit: The Build Folder
When given the command to “build” the site, Gatsby creates them all within a /build folder within the app. These are then ported over to the public_html folder upon build completion.
If a build gets interrupted, or there are limitations on what goes into the public_html folder, then files can end up left in the build folder.
Solution? If a build got interrupted, it just needs to be run again. If it’s the latter, ask your developers to add the gatsby-plugin-exclude so that those files are never even built.
Video Transcript:
I think—Oh, no, we got one more. Okay, so the SRC/pages folder. So in every Gatsby project, there is a SRC/pages folder. That is just natural to the folder structure when a Gatsby app is created. So if the project hasn't gone through cleanup, or the developers haven't refactored the code to make sure it's as elegant as possible, that folder could still end up existing—which means pages that are in it, are still rendering. So you just want to make sure that that's been cleaned up. Again, if they're not really giving you a response, just ask if you can have Read access to the repo, so you can go diagnose yourself, um, and then all of a sudden, you'll be—you'll get really helpful answers.
So anything that's in there, you probably just want it to be deleted. So that's some Gatsby specific fun. If anybody ever wants to talk about Gatsby, I love it. I think it's awesome. I think they're an awesome company, highly recommend.
Slide Text:
Possible Culprit: The Src/Pages Folder
In every Gatsby project, there exists a src/pages folder. If the project hasn’t gone through re-factoring or clean up, there could still be files in that folder that are causing them to be rendered even though they shouldn’t be.
Solution? Talk with your developer about what’s in those folders. If you have read access to the repo, you should be able to locate those yourself. Anything that’s in the folder that shouldn’t be, just ask for it to be deleted.
Video Transcript:
Alright, the last thing could be—it's happening on the query. So when—so Gatsby uses a query language called GraphQL. You've heard of query languages before, the most popular one is SQL, or SQL. So GraphQL is a new one, and basically, this is how the data is pulled from the CMS and populated into the static HTML files. Often times developers like to build their query to pull everything. But then when it comes to the module that's actually rendering the content, we'll put the filters there. The solution is that you should actually be putting the filter on the query itself, and not on the Rendering module. Not when it's mounted. So, um, that is a big one. And there's also a lot of resources.
So if you're ever in a situation; you're getting a lot of pushback from your developers—I have a library of resources available, saying that doing it at the query level is best practice. Mostly because you reduce the server response, and so anybody who's Head of Engineering that hears "less stress on your server," is pretty much always gonna say yes, because that's always a win.
Slide Text:
Possible Culprit: Draft Mode and GraphQL
Gatsby uses a query language called GraphQL to interact with your CMS/database and populate the dynamic data into static HTML files.
Developers often pull ALL of the content entries, and then use the rendering components to filter what should be live and what shouldn’t. Unfortunately, this means that Gatsby will build the page for a dedicated resource.
Solution? Ask your developer to adjust the query to filter then, and not at build/render time.
Example: a blog post URL is being created and rendered, but the blog post is NOT showing on the blog index page.
Video Transcript:
So those are the major questions and problems that I found. This has my tool set what I use, so of course, Wappalyzer, that we talked about quite a bit. There's also a plugin called—extension for Chrome—called HTTP Headers, which actually shows you all of the requests, and responses, and their codes.
And then if you noticed consistently, the—the solutions for this for these problems is that you have to create documentation, and work with your developers. So I linked this—that is the Google SEO Developer Guidelines. It's their own documentation—use it as much as you want, there's no need to rework the work that they've already done.
Slide Text:
Video Transcript:
And then here, I have some sample sites for each of the frameworks, just in case you ever want to see it.
Slide Text:
So let's for the last two minutes, I'll show you guys. Let's see, just like using Wappalyzer—everybody good with that? Cool.
Sure.
So I'm going to look in here for my—okay, so like, everybody's favorite topic right now. So if I pull this on Wappalyzer—this is what it looks like. You can see it's Gatsby. It's using React for the JavaScript library. It's got another JavaScript framework called Emotion. So you can kind of just—already see everything that's in there. Um, let's see, what's another fun one. Airbnb uses React.
Speaker 3 46:10
Interesting. It looks like it looks different from when I use Wappalyzer. But you're on a Mac, right?
Sam Torres 46:17
I am.
Speaker 3 46:18
Okay, because what I see is quite different. Like they have a specific area for HT—checking for HTML issues as well—HTML5 issues as well, and accessibility and quite interesting.
Sam Torres 46:31
Dude, that sounds awesome.
Speaker 3 46:33
It is awesome. It is even better, I would say.
Sam Torres 46:37
I feel... gypped. I feel gypped right now. Yes, you can see there's React. They think there might be Ruby on Rails on the backend and can confirm. I'm—which Ruby on Rails—just everybody knows—is a backend language. You can use some front end, but you wouldn't—wouldn't normally. But I imagine that all of this "holding the inventory" of available at Airbnb—that's all using Ruby.
So do you guys remember when I said that Google uses Angular and actually, like, helped create Angular. So really funny to me, that their Google Careers {site} —actually uses Vue. So with it being like—which React was something that Facebook developed. Angular was developed by Google and their parties, so it really cracks me up whenever I find Google properties that are using React or Vue instead of Angular—so I'm like—who pissed this person off, right?
Now, I also imagine it usually comes from acquisitions for them. So maybe they acquired the technology that they use for their Careers and just haven't changed it yet. But I still—I get a nice little chuckle out of that. It's like—you're not using your own product, why should I?
Alright, so we are at time, but any other questions from that presentation? Also feedback—if you thought it was horrible, and a waste of your life? I will make it accessible to everybody in the Women in Tech SEO {group} as well as this recording. So, got any questions or comments?
Thanks again for joining us for AHA! JavaScript SEO moments. If you need help, The Gray Dot Company can help bridge gaps with JS SEO by providing flexible solutions that adapt to the ever-changing needs of your team. Reach out today!
Home Search DataDigital Market IntelligenceOpinionated SEO Opinions Social Impact Careers Products Testimonials FeaturedIndustriesResources
