src_prepare/euscan-ng
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

euscan: don't respect robots.txt for mono download site

Browse Source 
Signed-off-by: Corentin Chary <corentin.chary@gmail.com>
This commit is contained in:
Corentin Chary
2012-06-30 15:57:56 +02:00
parent 9e6f885f89
commit 728770088c
1 changed files with 1 additions and 0 deletions

1
pym/euscan/__init__.py

@ -65,6 +65,7 @@ ROBOTS_TXT_BLACKLIST_DOMAINS = [
'(.*)qt.nokia.com(.*)', '(.*)qt.nokia.com(.*)',
'(.*)chromium.org(.*)', '(.*)chromium.org(.*)',
'(.*)nodejs.org(.*)', '(.*)nodejs.org(.*)',
'(.*)download.mono-project.com(.*)',
] ]
from out import EuscanOutput from out import EuscanOutput