Re: Web crawling

Página superior
Adjuntos:
Obtener este mensaje como un correo
+ (text/plain)
Eliminar este mensaje
Responder a este mensaje
Autor: Craig White
Fecha:  
A: plug-discuss
Asunto: Re: Web crawling
On Mon, 2005-02-28 at 07:51 -0700, Nathan England wrote:
> I am looking for a way to create a list of every file on a web site.
> Similar to the way wget will check your bookmarks, but I want something that
> will scan a website and create a list of everything found.
>
> Anyone know of anything like this?

----
wget has a '--spider' option

man wget

Craig

---------------------------------------------------
PLUG-discuss mailing list -
To subscribe, unsubscribe, or to change you mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss