Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I think it was a rhetorical question.


Yes it was. But I should have been clearer for the type of literal argument starters that would tend to hang around here ;)

I wonder if google always honour robots.txt


I hope so because otherwise they could get stuck in one of those:

http://en.wikipedia.org/wiki/Spider_trap




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: