web crawlers: Does Google revisit a page with a 404 error?


Good question. The short answer is yes. While the 404 URL error has internal / external links pointing to them, Google will eventually visit the page. We can verify this through:

  • Google search console: we can see errors from several months ago.
  • Analyzing the server logs: we can see the search engines that visit 404 and 410 error pages after those URLs were de-indexed.

Maybe they could have an end date to verify an error, but if that were true, once a URL is marked as passed, it would never appear again in the search engines (unless you get new links). In my opinion, this form is much more complex than checking all the URLs and classifying the good ones and those that generate an error.

Reality is more complex than a yes / no answer. In this experiment, they tried several methods to de-index the URLs and in the last column, you can see a yes / no option, whether the URL was visited after the de-indexing or not.