Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> So fuzzycanary also checks user agents

I wouldn't be so surprised if they often fake user agents to be honest. Sure, it 'll stop the "more honest" ones (but then, actual honest scrapers would respect robots.txt)

Cool idea though!



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: