Dealing with Rendered issues?

Crawl your website with Sitebulb for 300+ tech SEO checks

Try for Free
High This Hint is very important, and definitely warrants attention. Issue This Hint represents an error or problem that needs to be fixed.

Nofollow only in the HTTP response HTML

This means that the URL in question contains a nofollow in the response HTML, and not in the rendered HTML.

Why is this important?

Rendering webpages is a resource intensive task, and it takes significantly longer than simply grabbing source (response) HTML content.

This is why Google essentially crawl URLs in a two-stage process: their 'first look' is of the HTML response, then they render the page and have a 'second look' at the rendered HTML, then they update the index based on what they found in the rendered HTML.

Google rendering

The 'Processing' stage is the most complex to get your head around, because things are happening in parallel. It is also necessarily more complicated, because Google are making decisions on which pages to render based on the result of the meta robots check.

During this stage, they are parsing the response HTML, looking for followed links, and then passing these URLs back into the crawl queue - this occurs before rendering has happened.

So if links are nofollow in the response HTML, this may slow down how quickly Google finds and indexes the linked URLs.

Additionally, this mismatch means it is unclear whether the page should contain nofollow or not - is it intended that the nofollow should actually also be present in the rendered HTML?

What does the Hint check?

This Hint will trigger for any internal URL that contains a nofollow robots directive in the response HTML but not in the rendered HTML.

Examples that trigger this Hint

This Hint will trigger for any URL that has meta robots=nofollow in the <head> of the response HTML, that is not present in the rendered HTML.

For example, if the response HTML looked like this:

<!doctype html>
<html lang="en">
<head>
<meta name="robots" content="index,nofollow">
...
</head>
<body>...</body>
</html>

And the rendered HTML looked like this:

<!doctype html>
<html lang="en">
<head>
<meta name="robots" content="index,follow">
...
</head>
<body>...</body>
</html>

How do you resolve this issue?

You will first need to establish the desired state for these pages - if they should be nofollow or not. Once this has been clarified, ensure that whatever robots directives appear in the HTML response are also the same after JavaScript rendering.

Further Reading

Sitebulb Desktop

Ideal for SEO professionals, consultants and digital marketing agencies.

Try our fully featured 14 day trial. No credit card required.

Try Sitebulb for Free

Sitebulb Cloud

Perfect for collaboration, remote teams and extreme scale.

Cloud crawling with no project limits and very generous crawl limits.

Explore Sitebulb Cloud