msn bot

Status
Not open for further replies.

bigguy

Retired
Messages
10,984
Reaction score
10
Points
38
I found an msn bot in my forum today. Do you guys here at x10 use msn bots to advertise our pages or what???? I don`t care if you do just asking :cool: :teeth_smi
 

Corey

I Break Things
Staff member
Messages
34,553
Reaction score
204
Points
63
It's just a spider, they index pages. I'm sure Google will drop in sometime too :) You can shut them out by using a robots.txt file.

-Corey
 

bigguy

Retired
Messages
10,984
Reaction score
10
Points
38
Well if there good for my site I don`t mind them at all. I just wanted to know thats all :)
 

Spartan Erik

Retired
Messages
6,764
Reaction score
0
Points
0
Corey said:
It's just a spider, they index pages. I'm sure Google will drop in sometime too :) You can shut them out by using a robots.txt file.

-Corey

seeing as this thread appeals to me, and i dont want to create a new one, i have one question

i want to delist my site from all search engines, and google says to paste this inside robots.txt :

User-agent: *
Disallow: /

will just having that inside a robots.txt file work? also, where do i load the robots.txt? in public_html or in the root (top level of file manager)?
 
Last edited:

Corey

I Break Things
Staff member
Messages
34,553
Reaction score
204
Points
63
Yes, and public_html

-Corey
 
Status
Not open for further replies.
Top