Found this FyberSpider thing that used to crawl from a Comcast address and has apparently grown up and is crawling from a real dedicated server now.
The ip was 184.108.40.206 and the reverse DNS claims to be server.fybersearch.net and sure enough there something called FyberSearch with what appears to be a functional search page. The results actually appear to be populated with data collected from their crawl, trade secret, don't ask.
220.127.116.11 "GET /robots.txt HTTP/1.0" "Python-urllib/1.15"Here's the data center info if you want to block it:
18.104.22.168 "GET / HTTP/1.0" "FyberSpider"
OrgName: JTL Networks Inc.The search page has issues finding words in the one page I allowed to be indexed so I'm not terribly impressed, NEXT!
NetRange: 22.214.171.124 - 126.96.36.199