swaits/docrawler
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

flow: Created branch 'feature/remove_error_from_parselinks'.

Browse Source
This commit is contained in:
swaits committed 2015-02-16 23:06:45 -08:00
1 parent 6665729e3f
commit f16be33dbd
1 file changed
-1
Download Patch File Download Diff File Expand all files Collapse all files
docs/TODO.md
-1
View File
@@ -5,7 +5,6 @@
* deal with anchor issue (repeatedly crawling index.html#about, index.html#contact, etc.) * deal with anchor issue (repeatedly crawling index.html#about, index.html#contact, etc.)
* should only crawl these pages once * should only crawl these pages once
* but should be included in the links part * but should be included in the links part
* when sorting links, "uniq" them as well
* be smart about index.html? * be smart about index.html?
## Cleanup ## Cleanup