Corentin Chary 14971584af euscan: robots.txt, timeout, user-agent, ... ... - Add a blacklist for robots.txt, we *want* to scan sourceforge - Set a user-agent that doesn't looks like a browser - Handle timeouts more carefully - If brute force detect too much versions, avoid infinite loops - Handle redirections more carefully Signed-off-by: Corentin Chary <corentincj@iksaif.net>		2011-09-21 10:09:50 +02:00
..
__init__.py	euscan: shake the code	2011-08-31 15:38:32 +02:00
generic.py	euscan: robots.txt, timeout, user-agent, ...	2011-09-21 10:09:50 +02:00
php.py	euscan: respect robots.txt	2011-09-06 16:32:29 +02:00
pypi.py	euscan: shake the code	2011-08-31 15:38:32 +02:00
rubygem.py	euscan: respect robots.txt	2011-09-06 16:32:29 +02:00