Re: Web crawling

Top Page
Attachments:
Message as email
+ (text/plain)
Delete this message
Reply to this message
Author: Jay
Date:  
To: plug-discuss
Subject: Re: Web crawling


MirrorDot <http://www.mirrordot.org> uses wget to do all of our mirroring.
We had to tweak the options quite a bit, but it does work very well for
us.

~Jay


On Mon, 28 Feb 2005, Craig Brooksby wrote:

> I like HTTrack. http://www.httrack.com/ I've used wget too... used
> both, on both Win32 and Linux. HTTrack is very configurable -- I
> spider a lot of large sites, and it has never let me down.
> ---------------------------------------------------
> PLUG-discuss mailing list -
> To subscribe, unsubscribe, or to change you mail settings:
> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>


--
~Jay


---------------------------------------------------
PLUG-discuss mailing list -
To subscribe, unsubscribe, or to change you mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss