* so now http://a.com/index.html and http://a.com/ are understood equivalents
* as are http://a.com/x.html#blog and http://a.com/x.html (etc.)
* this meant a little more bookkeeping in docrawler, but not horrible
* stripAnchorFromURL because stripURL because it now handles anchor and path suffix stripping
* and url tests were written to support this
* outputs status
* crawlStatus() function created (split out of docrawl())
* changed from using bool flags to linkType in single httpItem enum, tests updated to match this
* unified where we set httpItem.linkType (inside fetchType), appropriate errors and returns added
* renamed a lot of things from 'page' to 'item'
* include refurl in httpItem
* []*Location result is now sorted
* moved $/testfilter.sh to $/util/testfilter.sh (updated Makefile accordingly)
* updated idea in Design.md
* renamed TestParse/TestFetch to TestSimpleParse/TestSimpleFetch.. more tests coming
* parse now sorts its results. test updated.
* moved regexp compilation to init instead of every time parse() is called
* misc comments and cleanup
swaits