Corentin Chary
a1bdde6a95
euscanwww: import local_settings and add postgres example
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-11-15 16:43:18 +01:00
Corentin Chary
f3363c52f9
euscan: update TODO
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-11-10 17:46:40 +01:00
Corentin Chary
57ed9251e8
euscan: add README, TODO and some scripts
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-11-10 14:07:53 +01:00
Corentin Chary
8be69f2350
euscan: show a trace when blocked by robots.txt
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-10-08 08:33:48 +02:00
Corentin Chary
739358cf7c
euscan: really skip all dev versions
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-10-08 08:33:25 +02:00
Corentin Chary
277fb4ebe6
euscan: add new robots.txt exceptions
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-10-08 08:33:03 +02:00
Corentin Chary
d7f655cdde
euscan: add an optional persistent cache
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-10-02 10:04:44 +02:00
Corentin Chary
7f56dd4ac4
euscan: fix a stupid bug
...
finally is always called, so the list was always empty.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-10-02 10:03:49 +02:00
Corentin Chary
0ad1eecca7
euscanwww: chmod +x manage.py
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-30 09:13:47 +02:00
Corentin Chary
96775215fa
euscanwww/scripts: sync with prod
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-30 09:13:15 +02:00
Corentin Chary
1d8e87482f
euscanwww: remove progress stuff
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-30 09:10:56 +02:00
Corentin Chary
7fb4e6edff
euscan: better output
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-21 10:58:41 +02:00
Corentin Chary
a13d593d8d
euscanwww: remove unused gchartwrapper
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-21 10:48:32 +02:00
Corentin Chary
1bd42f303d
euscanwww: add some json
...
This is not a real API, I will break it !
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-21 10:12:01 +02:00
Corentin Chary
14971584af
euscan: robots.txt, timeout, user-agent, ...
...
- Add a blacklist for robots.txt, we *want* to scan sourceforge
- Set a user-agent that doesn't looks like a browser
- Handle timeouts more carefully
- If brute force detect too much versions, avoid infinite loops
- Handle redirections more carefully
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-21 10:09:50 +02:00
Corentin Chary
8c40a1795c
euscan: blacklist art.gnome.org
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-10 08:25:39 +02:00
Corentin Chary
9da62b211b
euscan: fix some robots.txt issues
...
- disable checks for ftp
- fail silently
- use einfo and not eerror
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-10 08:23:46 +02:00
Corentin Chary
c5af0e1937
euscan: don't mix spaces and tabs
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 17:35:17 +02:00
Corentin Chary
2210b2610d
euscan: don't get robots.txt on ftp
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 17:34:50 +02:00
Corentin Chary
a137ef60e3
euscan: respect robots.txt
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 16:32:29 +02:00
Corentin Chary
bd75e1af4e
euscan/helpers: use HEAD in tryurl
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 15:47:54 +02:00
Corentin Chary
454d369ced
euscan/handlers: fix resursive brute force in generic handler
...
component was modified by the function since it's a reference,
do an explicit copy to fix that.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 15:47:00 +02:00
Corentin Chary
1fae084566
euscanwwww: delete packaded versions, not upstream
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 11:31:27 +02:00
Corentin Chary
6a10041300
euscanwww/templates: save datatable state
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 09:18:20 +02:00
Corentin Chary
8dc19b9856
euscan: fix some errors
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 09:17:08 +02:00
Corentin Chary
833f1253a1
euscanwww/templates: add appropriate copyrights
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-09-06 09:16:21 +02:00
Corentin Chary
752fb04425
euscan: shake the code
...
- add custom site handlers
- use a custom user agent
- fix some bugs in management commands
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-31 15:38:32 +02:00
Corentin Chary
5634c59944
euscan: better blacklists
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-28 13:20:40 +02:00
Corentin Chary
a5cf338905
euscanwww/templates: add anchors to versions
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-28 11:22:17 +02:00
Corentin Chary
70ac94474e
euscanwww/views: remove old FIXME
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-28 11:22:04 +02:00
Corentin Chary
9507d5ac4b
euscanwww: enable caching
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-28 11:21:37 +02:00
Corentin Chary
4b25e2744b
euscan: update TODO
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-28 11:21:11 +02:00
Corentin Chary
45ca837908
euscanwww/feed: tweak feeds
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-28 10:45:46 +02:00
Corentin Chary
60bc61275e
euscanwww: fix urls
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-26 13:27:46 +02:00
Corentin Chary
42a2b61f55
templates: fix feed link
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-25 16:22:20 +02:00
Corentin Chary
ac4cdcf036
media: add missing feed icon
2011-08-25 16:17:33 +02:00
Corentin Chary
d0fa19bc1d
euscanwww: try to keep trace of versions change
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-25 15:39:54 +02:00
Corentin Chary
7570453bc2
euscan: fix a bug and add another exception
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-22 10:47:21 +02:00
Corentin Chary
c9f9331aaf
euscanwww/commands: don't keep all logs
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-22 10:46:56 +02:00
Corentin Chary
03a7a0a277
euscan: remove debug traces
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-03 11:55:33 +02:00
Corentin Chary
5474d6f03e
euscan: clean quirks and fix htop
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-03 11:54:45 +02:00
Corentin Chary
142cfe0924
euscanwww/css: change the layout
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-03 10:55:33 +02:00
Corentin Chary
2d224279f8
euscanwww/charts: add a transparent background
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-08-03 10:55:13 +02:00
Corentin Chary
421dbefdcf
euscanwww: datetime is not indexed, id is
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-07-21 15:01:14 +02:00
Corentin Chary
91e928371d
euscanwww: update theme
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-07-21 15:00:59 +02:00
Corentin Chary
1a482acde1
euscanwww: add weekly charts
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-05-14 23:09:00 +02:00
Corentin Chary
917a290e3a
euscanwww: finish charts
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-05-05 18:10:47 +02:00
Corentin Chary
54f7771b43
euscanwww: add new charts
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-05-03 08:19:01 +02:00
Corentin Chary
120ae425af
euscanwww: add --purge to scan-portage
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-04-29 09:24:47 +02:00
Corentin Chary
244e7d64fd
scritps: handle local trees
...
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
2011-04-27 10:14:09 +02:00