seo – How can a search engine crawl a dynamically generated website?

Short answer: That PHP code is run on the server before sending the response to the crawler, so by the time the page reaches the crawler, all that info is already populated.

For sites written using server-side languages such as your example, here’s the full lifecycle when a user visits a page:

  1. The user’s browser sends an HTTP request to the server for a certain path (such as /an/example/page/).

  2. The server receives the request and determines the appropriate server-side code to run to generate the page. It executes this code, if any (or none if it’s a static site).

  3. The server sends the final generated, by that point static HTML page back to the user’s browser.

Note that all the code is finished running on the server before the server actually sends any information back to the user’s browser (or a web crawler).

Things are a little different when the page is generated in part by client-side code (JavaScript) instead, which is a topic for a different discussion.

Regarding waiting for a user to log in or take action, generally search engine crawlers are cookieless and take no user actions, so anything hidden behind a login won’t get crawled. Stuff hidden from crawlers behind logins like this is called the deep web, which is a cool term if you ask me.