Re: way to search the internet automatically say once a day …

Top Page
Attachments:
Message as email
+ (text/plain)
Delete this message
Reply to this message
Author: Victor Odhner
Date:  
To: plug-discuss
Subject: Re: way to search the internet automatically say once a day with no human intervention and store the results
Jim B: What I want is way to create a list that will be used to search
the internet and then have the results stored not the actual web pages
just the urls and then have a way for the urls to be reviewed.

VO: If you would like a custom solution, you could write a very simple
Perl program using the 'get' command. That returns the contents of a
page as a string, and you can pattern-match on it or whatever. It could
read through a simple text file with one URL per line, visit that URL,
and spit out any URL matching your criteria. Very, very simple. I
Googled some links for you below ... the first one explains the
LWP::Simple module which I've used. The second one looks interesting
and more general, but I didn't look at it in any detail.

Good luck,

Vic

<http://www.perl.com/pub/a/2002/08/20/perlandlwp.html>
<http://www.devshed.com/c/a/Perl/Web-Mining-with-Perl>



---------------------------------------------------
PLUG-discuss mailing list -
To subscribe, unsubscribe, or to change you mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss