Who's online? Mostly spiders.

[kerry_nf]

Hey guys, I was always impressed by the number of guests browsing the forum until I clicked "Quick Links" and "Who's Online"...the majority of the guests are indexing spiders, mostly from Yahoo.

I know spiders need to crawl website content to keep their indexes up to date but their has to be a way for forum software logic to exclude these "guests" from the "Who's Online" list.

...a simple robots.txt file with the appropriate parameters limits their presense.

[PuzzledMoose]

LOL - just looked and apart from the Slurp Spiders - what a great name! - there are also now a lot of people reading Quick Links and whose online - now look what you've done, Keri ...

[kerry_nf]

LOL! Oh no, I have created a loop...we'll all perish at the mercy of the "slurpies" Aaaarrrgh!!! lol

[DukeNukem]

LOL all I can say .Jerry

[kyscroller]

I like the google spider.

[JimSawyer]

So what exactly are spiders and what do they do

[BobD]

The spiders are sent out by search engines such as Google and Yahoo to check the content of different sites. That way if someone googles "drilling holes for scroll sawing" it will bring up the site.


I'll check with Rob about the tweak you suggested, Kerry, but I know he's swamped right now...it might be a while before he can implement it!


Bob

[Mackem]

I like the "bot" spider, it's got my imagination working overtime.lol.