Neverending Story (decadence1) wrote in lj_userdoc,
Neverending Story
decadence1
lj_userdoc

What can I do to *make* my journal be [more likely to be] indexed by search engines?

This is in the "working with other sites" FAQ category - this FAQ (http://www.livejournal.com/support/faqbrowse.bml?faqid=50)
Posting this as a new top level entry, following advice; I'd previously commented in an old entry.

I'd like to raise the issue of something perhaps being added for those rare support requests where someone DOES want their journal to be indexed by search engines. Since it's a rare request, I think a new FAQ would be overkill but perhaps some inclusion in #50? Example support request: http://www.livejournal.com/support/see_request.bml?id=271264

On the same issue (I have no idea where to otherwise post this), the FAQ states:
Additionally, if you have a paid account, a robots.txt file will be added to your paid URL (such as http://exampleusername.livejournal.com/). Not all robots respect the rules, but most of the popular search sites' robots do.

For user who might want their sites to be indexed, suggest we add some blurb about whether or not the robots.txt file is user-configurable? In the same manner as META tags may be switched on or off (2nd para.)

E.g., http://exampleusername.livejournal.com/robots.txt will show (where example used is a Paid Account) that all search robots are disallowed from all sections of the domain (syntax explanation at http://www.searchengineworld.com/robots/robots_tutorial.htm) so the site cannot be indexed via robots (not all robots obey these rules as the FAQ explains, and there are search methods other than robots).
I am assuming the robots.txt file is not user-configurable but I do not know for certain.
Subscribe
  • Post a new comment

    Error

    Comments allowed for members only

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 3 comments