From fe46e7af15c9d3002a3e2bb1e58413e5e748e77d Mon Sep 17 00:00:00 2001 From: Stephen Waits Date: Mon, 16 Feb 2015 06:13:59 -0800 Subject: [PATCH] add a TODO list --- docs/TODO.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 docs/TODO.md diff --git a/docs/TODO.md b/docs/TODO.md new file mode 100644 index 0000000..3e03d44 --- /dev/null +++ b/docs/TODO.md @@ -0,0 +1,22 @@ +# To Do + +## Bugs + +* deal with anchor issue (repeatedly crawling index.html#about, index.html#contact, etc.) + * should only crawl these pages once + * but should be included in the links part +* when sorting links, "uniq" them as well + +## Cleanup + +* rename Page -> crawlerState +* helper function to do the whole crawl process +* document code better + * test with godoc + +## Features + +* some sort of status during crawl? +* command line options, flag parsing +* check RFCs on URL character sets, compliance, etc. +* packagize parts of app