Scripting a mirror test

[der.hans]

Am 31. Aug, 2007 schwätzte Joey Prestia so:

> Problem: I need to come up a script that I can use to test if between 30
> to 50 mirrors are carrying the content of a particular page in its
> unchanged form. I would guess to create a script with maybe a for loop
> and a file with the web addresses of these mirrors and run a diff on the
> pages then send any errors to me so if the site is failing to maintain
> its content I can be notified. I am still quite new to this but am sure
> it can be done with a script that I could run as a cron job instead of
> manually testing each of the pages several times a week. Any suggestions
> on how I could load the site content to do this and what commands would
> be best suited?

Write a script and run it via cron :).

Here's a sample that won't work, but it should give you the primary parts
you need.

###

#!/bin/path/to/bourne/derivative/shell

repos=/some/dir/
datum=`date +%Y%h%d`

# today's repos dir
trd=${repos}/${datum}

mkdir $trd

wget -O $trd/golden.$datum.html http://goldenhost/page.html
goldenhash=`md5sum $trd/golden.$datum.html | cut -d' ' -f1`
for i in `seq 1 50`; do
 	wget -O $trd/host${i}.$datum.html http://host${i}/page.html
 	iterhash=`md5sum $trd/host${i}.$datum.html | cut -d' ' -f1`
 	if [ $goldenhash != $iterhash ] ; then
 		borken="$borken $i"
 	fi
done

( for i in $borken; do
 	echo host${i}
 	diff $trd/golden.$datum.html $trd/host${i}.$datum.html
done ) | mail -s "Borken hosts $datum" $youraddy

###

That last piece doing the diff to mail probably formats horribly.

If you have questions about how parts of the code work please join the
devel list and ask there :).

There are definite improvements I'd make before putting code like this
live, but I think this takes care of your stated needs.

ciao,

der.hans
-- 
#  https://www.LuftHans.com/        http://www.CiscoLearning.org/
#  "Dinosaurs return. Want their oil back." -- David Brin