automation of collecting web site size

[Eric Johnson]

On Fri, 13 Oct 2000, Wes Bateman wrote:
: Hey Guys:
: 
: I've been playing around and trying to see how large a given website is
: remotely.  I've been using wget and du.  It can take a long time, and I
: really don't want to mirror the site (I erase the data right after I do
: it).  Seems a shame to waste so much time and bandwidth, when all I really
: want is the total space (size in bytes) that a website occupies.  I've
: been following infinite depth of links, but only remaining in the original
: domain.
: 
: There must be a way to get this info, without either sucking the whole
: site down, or having access to the webserver?
: 
: Anyone have any ideas, suggestions, etc.?

lynx -dump -head $URL | grep Content-Length | cut -d: -f 2

---
Eric Johnson (ej@netasm.com)                                net.assembly
http://netasm.com/                      12629 North Tatum Boulevard #191
602 996-9682                                      Phoenix, Arizona 85032