add a TODO list

This commit is contained in:
swaits committed 2015-02-16 06:13:59 -08:00
1 parent 77b50a22b4
commit fe46e7af15
1 file changed
+22
+22
View File
@@ -0,0 +1,22 @@
# To Do
## Bugs
* deal with anchor issue (repeatedly crawling index.html#about, index.html#contact, etc.)
* should only crawl these pages once
* but should be included in the links part
* when sorting links, "uniq" them as well
## Cleanup
* rename Page -> crawlerState
* helper function to do the whole crawl process
* document code better
* test with godoc
## Features
* some sort of status during crawl?
* command line options, flag parsing
* check RFCs on URL character sets, compliance, etc.
* packagize parts of app