tag:blogger.com,1999:blog-19248375.post116422311979239833..comments2023-10-18T05:54:12.748-07:00Comments on IncrediBILL's Random Rants: Exalead Preview Violating Webmasters ContentIncrediBILLhttp://www.blogger.com/profile/14244934627308399202noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-19248375.post-1164738654529323312006-11-28T10:30:00.000-08:002006-11-28T10:30:00.000-08:00Looks like they think using robots.txt would be to...Looks like they think using robots.txt would be to confusing based on the reply they posted<BR/><A HREF="http://web-robot-abuse.blogspot.com/2006/11/wwwexaleadcom-violates-robotstxt.html" REL="nofollow">Here on my site</A>.<BR/><BR/><I>If the usage of the thumbnails tend to generalize in search engines, the webmasters will certainly take it into account in the writing of their robots.txt file andtmasterhttps://www.blogger.com/profile/16370669314050985245noreply@blogger.comtag:blogger.com,1999:blog-19248375.post-1164411445138481612006-11-24T15:37:00.000-08:002006-11-24T15:37:00.000-08:00ExaleadGuy,First, your crawler should tell us wher...ExaleadGuy,<BR/><BR/>First, your crawler should tell us where to find information about your bot including the exact user agent names you look for, if it's "Exabot" or "NG" or whatever, I'd prefer the preview at least be "Exabot NG" so people would know they're related.<BR/><BR/>I would suggest a user agent string like:<BR/><BR/>"Exabot/3.0 http://www.exalead.com/bot.html"<BR/><BR/>I would preferIncrediBILLhttps://www.blogger.com/profile/14244934627308399202noreply@blogger.comtag:blogger.com,1999:blog-19248375.post-1164394751981605172006-11-24T10:59:00.000-08:002006-11-24T10:59:00.000-08:00No the robot does not comply with simple basic rob...No the robot does not comply with simple basic robots.txt commands to not load images.<BR/><BR/>User-agent: *<BR/>Disallow: /images/tmasterhttps://www.blogger.com/profile/16370669314050985245noreply@blogger.comtag:blogger.com,1999:blog-19248375.post-1164368289840029612006-11-24T03:38:00.000-08:002006-11-24T03:38:00.000-08:00exaleadguy said:"Would that be ok for you ?"Certai...exaleadguy said:<BR/>"Would that be ok for you ?"<BR/><BR/>Certainly wouldn't be OK for me!<BR/><BR/>I manage over 15 sites, and having to retrospectively hand edit every one of hundreds of pages is a huge task. <BR/><BR/>Your method also adds an extra line of code that valuable search engines will have to wade through before they get to the all important content.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-19248375.post-1164361888717142212006-11-24T01:51:00.000-08:002006-11-24T01:51:00.000-08:00Hello,Since then we had not had any bad feedback f...Hello,<BR/><BR/>Since then we had not had any bad feedback from webmasters for our preview. But I understand your point, we will add the ability for the webmasters to refuse their pages to be previewed and thumbnailed by the use of these two meta:<BR/>meta name="robots" content="nopreview"<BR/>or <BR/>meta name="robots" content="nothumbnail"<BR/>It should be online very soon.<BR/>Would that be okAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-19248375.post-1164357179996280452006-11-24T00:32:00.000-08:002006-11-24T00:32:00.000-08:00My traffic has been increasing, not decreasing, si...My traffic has been increasing, not decreasing, since I started taking control of my content - and it's not this blog either.<BR/><BR/>If they want to index me in Exalead, then they'll have to do something about the preview mode thing, it's over the top.IncrediBILLhttps://www.blogger.com/profile/14244934627308399202noreply@blogger.comtag:blogger.com,1999:blog-19248375.post-1164347890466078692006-11-23T21:58:00.000-08:002006-11-23T21:58:00.000-08:00Hello.I understand that it's your content on the w...Hello.<BR/><BR/>I understand that it's your content on the website and you can do whatever you want with it, but isn't this a little overzealous? If you insist that only humans should request your pages you (a) will loose some visitors who use some funky setup or are from the wrong IP block and (b) you'll kill of the chance for your content to be exposed on other pagesCd-MaNhttps://www.blogger.com/profile/05030326541176171725noreply@blogger.com