swaits
7498b648d7
* added Skipped field to Page to remember this state * marshaled into "Remote" field in JSON * bug fix: skipped (etc.) sites weren't getting added to the main page list * tests had to be reworked. because the crawl is non-deterministic, the tests started breaking. tests made a bit more robust, but are slightly weaker now. * the lists of URLs in the output are now sorted * ... and the parseLinks is no longer doing any sorting (no point in it) * parser tests updated accordingly * added a "broken" link to index.html - not currently supported, will fix next!