diff --git a/docs/TODO.md b/docs/TODO.md new file mode 100644 index 0000000..3e03d44 --- /dev/null +++ b/docs/TODO.md @@ -0,0 +1,22 @@ +# To Do + +## Bugs + +* deal with anchor issue (repeatedly crawling index.html#about, index.html#contact, etc.) + * should only crawl these pages once + * but should be included in the links part +* when sorting links, "uniq" them as well + +## Cleanup + +* rename Page -> crawlerState +* helper function to do the whole crawl process +* document code better + * test with godoc + +## Features + +* some sort of status during crawl? +* command line options, flag parsing +* check RFCs on URL character sets, compliance, etc. +* packagize parts of app