I asked this question, because I can only get wget to use spider if I provide 
a file for it to follow. Otherwise it only reports back with the index.html 
then exits... I just can't get it to work.


On Monday 28 February 2005 08:12 am, Craig White wrote:
> On Mon, 2005-02-28 at 07:51 -0700, Nathan England wrote:
> > I am looking for a way to create a list of every file on a web site.
> > Similar to the way wget will check your bookmarks, but I want something
> > that will scan a website and create a list of everything found.
> >
> > Anyone know of anything like this?
>
> ----
> wget has a '--spider' option
>
> man wget
>
> Craig
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
> To subscribe, unsubscribe, or to change  you mail settings:
> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
---------------------------------------------------
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change  you mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss