swaits f3152b6f44 broken links now recorded as such ...

* Broken field in Page{} now,
* anything that fails to fetch (with http 200) or parse (should not happen) is considered broken
* tests updated appropriately
* discovered my "image.png" file was in the wrong directory because it showed up as broken. fixed!
* bug fix: only fetch if type=text/html (after HEAD response)

2015-02-15 22:25:50 -08:00

.vendor

add coverage to Makefile (make cover)

2015-02-15 12:40:56 -08:00

docs

misc. minor cleanup

2015-02-15 00:40:36 -08:00

src/docrawler

broken links now recorded as such

2015-02-15 22:25:50 -08:00

util

misc. minor cleanup

2015-02-15 00:40:36 -08:00

.flow

flow initialization: Added configuration file.

2015-02-14 23:50:10 -08:00

.hgignore

add coverage to Makefile (make cover)

2015-02-15 12:40:56 -08:00

Makefile

minor test cleanup; add "make stats" target to run cloc against project

2015-02-15 16:19:58 -08:00

S

Description

Simple web crawler written in Go.

MIT

5.1 MiB

0 Stars 1 Watchers 0 Forks

Languages

Go 96.5%

JavaScript 1.3%

HTML 0.8%

CSS 0.6%

Emacs Lisp 0.3%

Other 0.3%