hi... this question is for the wget guru! i'm testing wget on a test site.. i'm using the recursive function of wget to crawl through a portion of the site... it appears that wget is hitting a link within the crawl that's causing it to begin to crawl through the section of the site again... i know wget isn't as robust as nutch, but can someone tell me if wget keeps a track of the URLs that it's bben through so it doesn't repeat/get stuck in a never ending processs... i haven't run across anything in the docs that seems to speak to this point.. thanks -bruce
