There are several reasons why your content may not appear in the search
engine results, however, it is important to keep in mind that
a search engine
index may contain pages that are not displayed in your
How to know if your content is really indexed
It can be difficult to know if your content is indexed.
In some cases, it may appear that documents are not indexed through one of
These methods, but documents can be found in the index using others
For example, webmaster tools can report that few documents
are indexed even when you can look up your sentences and find the
Documents in the search engine. In such a case, the documents are
How content is indexed
Before search engines index content, they must find it using a
You should check the logs of your web server to see if
search engine crawlers (identified by your user agent, for example,
Bing / MSNbot)
They are visiting your site.
Larger search engines like Google and Bing usually crawl sites
frequently, but the tracker may not know about the new site. You can
Notify search engines of the existence of your site through
registering as your webmaster (Google Webmaster
Tools, Bing Webmaster
Tools) or, if the search
engine does not provide this facility, presenting a link to its
crawlers (for example, Yahoo).
How long has your site / content been online?
Search engines can index very content
Quickly after it has been found, however, these updates are occasionally delayed.
Smaller search engines can also be much less responsive and take weeks to index
If your content has only been online for several days and does not have
any link from other sites (or their links come from sites that
crawlers do not visit frequently) is probably not indexed.
If your site has not been active for more than a few months, the search engines
You may not trust enough to index much content from it yet.
Has the content been excluded by the webmaster?
This step is especially important if you are taking control of a site from
someone else and there is a problem with a specific page or directory:
Exclusions and delete them if you want the crawlers to index the content.
Is there a technical problem that prevents your content from being indexed?
If you have an established site but the specific content is not being
indexed (there are no results of the web crawler in the URLs where the content is found).
resides) the webmaster tools provided by Google and Bing can provide
Useful diagnostic information.
Of Google Crawl
The documentation provides extensive experience in common problems for the web
crawlers that prevent content from being indexed and, if you use
Google Webmaster Tools, you will receive an alert if any of these
Problems are detected on your site.
Correct errors and misconfigurations as quickly as possible to ensure
that all the content of your site is indexed.
Is the content of low quality?
Search engines do not index most of the pages they crawl. They only index content of the highest quality. Search engines will not index content if:
- It's spam, gibberish, or nonsense.
- It is found elsewhere. When search engines find duplicate content, they choose only one of the duplicates to index. Usually that is the original that has more reputation and links.
- It's thin. It needs more than a couple of lines of original text. Preferably much more. It is unlikely that automatically created pages with little content, such as a page for each of your users, will be indexed.
- It does not have enough reputation or links. A page may be buried too deep in your site to classify it. It is unlikely that any page will be indexed without external links and more than a few clicks from the homepage.
Are some of your content indexed, but not all?
If your site has hundreds of pages, Google will almost never choose to index each page. If your site has tens of thousands of pages, it is very common for Google to choose to index only a small part of those pages.
Google chooses the number of pages to index from a site based on the overall reputation of the site and the quality of the content. Typically, Google indexes a larger percentage of a site over time as the reputation of the site grows.