Am 23. May, 2005 schwätzte Josef Lowder so:
> Anyone know how I could execute 'lynx {url.address]' in a shell script
> and capture the output to a file as a background task?
>
> For example, let's say I wanted to collect the latest listings of some
> specific item from the Arizona Republic classified section and send
> the results to a text file. I have tried various scripts to do this, but
> haven't been able to get one to work yet.
As pointed out, use -dump to get only the presented text of the web page.
Anyone know if it handles javascript created text?
Use -source to get the HTML.
Redirect to file using '>'.
wget or curl might be the tools you want to use, though.
I like --force-directories for wget to keep things in a nice tree.
Anyone know of an option to wget to get it to report the filename(s) that
it'll use?
Use -O for wget to specify an output file. Normally wget uses the same
filename as the file you're downloading. That results in index.html
getting overwritten a lot if you're not careful :).
BTW, elinks is a better, IMO, text-based web browser.
If you're needing to parse a bunch of pages in order to find links to
follow use a screen scraper. wget and curl have options for that. There
are also libs in various languages. curl is plugged into PHP. Perl has
WWW::Mechanize.
Anyone know of a screen scraper that can handle javascript?
ciao,
der.hans
--
# https://www.LuftHans.com/ http://www.AZOTO.org/
# HERE LIES LESTER MOORE
# SHOT 4 TIMES WITH A .44
# NO LES
# NO MOORE
# -- tombstone, in Tombstone, AZ
---------------------------------------------------
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change you mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
(text/plain)