docrawler

0 Fork 0

T

swaits 0456723f0b minimally functional crawler - crawls single page, finds links, stores in data structure

* refined Page struct. now need to refactor everything else to use this
* wrote docrawl() function. NOT intended to be anything like the final design. just sketching it out to get a better idea of needs, design, etc.
* replaced fmt.Print* with t.Log* in tests (oops!)
* added a header fetcher function to get the file type (since we only want to parse html (with tests)
* added small test for the URL resolver where the base URL is empty
* added placeholder files in testsite/ for about.html and image.png

2015-02-15 12:12:30 -08:00

.vendor

initial vendor commit

2015-02-14 21:53:22 -08:00

docs

misc. minor cleanup

2015-02-15 00:40:36 -08:00

src/docrawler

minimally functional crawler - crawls single page, finds links, stores in data structure

2015-02-15 12:12:30 -08:00

util

misc. minor cleanup