src_prepare/euscan-ng
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
580 Commits 1 Branch 4 Tags 13 MiB
70f88ed37d
Commit Graph

580 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Corentin Chary
57ed9251e8 euscan: add README, TODO and some scripts 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-11-10 14:07:53 +01:00
Corentin Chary
8be69f2350 euscan: show a trace when blocked by robots.txt 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-10-08 08:33:48 +02:00
Corentin Chary
739358cf7c euscan: really skip all dev versions 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-10-08 08:33:25 +02:00
Corentin Chary
277fb4ebe6 euscan: add new robots.txt exceptions 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-10-08 08:33:03 +02:00
Corentin Chary
d7f655cdde euscan: add an optional persistent cache 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-10-02 10:04:44 +02:00
Corentin Chary
7f56dd4ac4 euscan: fix a stupid bug 
finally is always called, so the list was always empty.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-10-02 10:03:49 +02:00
Corentin Chary
0ad1eecca7 euscanwww: chmod +x manage.py 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-30 09:13:47 +02:00
Corentin Chary
96775215fa euscanwww/scripts: sync with prod 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-30 09:13:15 +02:00
Corentin Chary
1d8e87482f euscanwww: remove progress stuff 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-30 09:10:56 +02:00
Corentin Chary
7fb4e6edff euscan: better output 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-21 10:58:41 +02:00
Corentin Chary
a13d593d8d euscanwww: remove unused gchartwrapper 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-21 10:48:32 +02:00
Corentin Chary
1bd42f303d euscanwww: add some json 
This is not a real API, I will break it !

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-21 10:12:01 +02:00
Corentin Chary
14971584af euscan: robots.txt, timeout, user-agent, ... 
- Add a blacklist for robots.txt, we *want* to scan sourceforge
- Set a user-agent that doesn't looks like a browser
- Handle timeouts more carefully
- If brute force detect too much versions, avoid infinite loops
- Handle redirections more carefully

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-21 10:09:50 +02:00
Corentin Chary
8c40a1795c euscan: blacklist art.gnome.org 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-10 08:25:39 +02:00
Corentin Chary
9da62b211b euscan: fix some robots.txt issues 
- disable checks for ftp
- fail silently
- use einfo and not eerror

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-10 08:23:46 +02:00
Corentin Chary
c5af0e1937 euscan: don't mix spaces and tabs 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 17:35:17 +02:00
Corentin Chary
2210b2610d euscan: don't get robots.txt on ftp 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 17:34:50 +02:00
Corentin Chary
a137ef60e3 euscan: respect robots.txt 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 16:32:29 +02:00
Corentin Chary
bd75e1af4e euscan/helpers: use HEAD in tryurl 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 15:47:54 +02:00
Corentin Chary
454d369ced euscan/handlers: fix resursive brute force in generic handler 
component was modified by the function since it's a reference,
do an explicit copy to fix that.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 15:47:00 +02:00
Corentin Chary
1fae084566 euscanwwww: delete packaded versions, not upstream 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 11:31:27 +02:00
Corentin Chary
6a10041300 euscanwww/templates: save datatable state 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 09:18:20 +02:00
Corentin Chary
8dc19b9856 euscan: fix some errors 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 09:17:08 +02:00
Corentin Chary
833f1253a1 euscanwww/templates: add appropriate copyrights 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-09-06 09:16:21 +02:00
Corentin Chary
752fb04425 euscan: shake the code 
- add custom site handlers
- use a custom user agent
- fix some bugs in management commands

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-31 15:38:32 +02:00
Corentin Chary
5634c59944 euscan: better blacklists 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-28 13:20:40 +02:00
Corentin Chary
a5cf338905 euscanwww/templates: add anchors to versions 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-28 11:22:17 +02:00
Corentin Chary
70ac94474e euscanwww/views: remove old FIXME 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-28 11:22:04 +02:00
Corentin Chary
9507d5ac4b euscanwww: enable caching 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-28 11:21:37 +02:00
Corentin Chary
4b25e2744b euscan: update TODO 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-28 11:21:11 +02:00
Corentin Chary
45ca837908 euscanwww/feed: tweak feeds 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-28 10:45:46 +02:00
Corentin Chary
60bc61275e euscanwww: fix urls 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-26 13:27:46 +02:00
Corentin Chary
42a2b61f55 templates: fix feed link 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-25 16:22:20 +02:00
Corentin Chary
ac4cdcf036 media: add missing feed icon 2011-08-25 16:17:33 +02:00
Corentin Chary
d0fa19bc1d euscanwww: try to keep trace of versions change 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-25 15:39:54 +02:00
Corentin Chary
7570453bc2 euscan: fix a bug and add another exception 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-22 10:47:21 +02:00
Corentin Chary
c9f9331aaf euscanwww/commands: don't keep all logs 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-22 10:46:56 +02:00
Corentin Chary
03a7a0a277 euscan: remove debug traces 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-03 11:55:33 +02:00
Corentin Chary
5474d6f03e euscan: clean quirks and fix htop 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-03 11:54:45 +02:00
Corentin Chary
142cfe0924 euscanwww/css: change the layout 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-03 10:55:33 +02:00
Corentin Chary
2d224279f8 euscanwww/charts: add a transparent background 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-08-03 10:55:13 +02:00
Corentin Chary
421dbefdcf euscanwww: datetime is not indexed, id is 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-07-21 15:01:14 +02:00
Corentin Chary
91e928371d euscanwww: update theme 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-07-21 15:00:59 +02:00
Corentin Chary
1a482acde1 euscanwww: add weekly charts 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-05-14 23:09:00 +02:00
Corentin Chary
917a290e3a euscanwww: finish charts 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-05-05 18:10:47 +02:00
Corentin Chary
54f7771b43 euscanwww: add new charts 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-05-03 08:19:01 +02:00
Corentin Chary
120ae425af euscanwww: add --purge to scan-portage 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-04-29 09:24:47 +02:00
Corentin Chary
244e7d64fd scritps: handle local trees 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-04-27 10:14:09 +02:00
Corentin Chary
0aba96f66f euscanwww: charts, about, etc... 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-04-25 22:27:32 +02:00
Corentin Chary
13dd433996 euscan: remove some false positives 
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
 2011-04-25 22:24:52 +02:00