I would use "wget".  It has a recursive option, so that
it can download everything on a site to a specified number
of levels.  It can even load from multiple sites.

I have used it often under Linux and Solaris, and recently
pulled the whole PLUG web site onto my PC using wget
under Cygwin in my Win98 environment.

Vic