Updated October 04, 2004
Created June 27, 2003

Here's the results from searching the ftp.compaq.com site after using one of these two wget site index methods:

[riblack@box1 1]$ fgrep -i bootdisk * | grep linux | grep 9 | grep RedHat
index.html.1396: 2003 Jun 05 15:17 File bootdisk.rhl9.readme (2,013 bytes)
index.html.1396: 2003 Jun 05 15:17 File bootdisk.rhl9dl.img (1,474,560 bytes)
index.html.1396: 2003 Jun 05 15:17 File bootdisk.rhl9us.img (1,474,560 bytes)
[riblack@box1 1]$





Short version that requires you to do the first 2 indexing:
wget ftp://ftp.compaq.com
wget ftp://ftp.compaq.com/pub
COUNT=1; while :; do FILES=`cat index.html.$COUNT | grep "a href" | sed -e \
's,^.*=",,' -e 's,".*$,,' | grep /$`; if [ -n "$FILES" ]; then wget $FILES; \
fi; COUNT=`expr $COUNT + 1`; if [ ! -e index.html.$COUNT ]; then break; fi; done




the cleanup for this long version took a few more minutes:

Long version that takes care of itself and gives progress report during download:

ls index.html* >/dev/null 2>&1 && { echo you should either remove these index.html \
files or make another directory and change to it; } || { URL=ftp://ftp.compaq.com; \
COUNT=; while :; do if [ -z "$COUNT" ]; then echo; echo Just one moment while I get \
ready to index $URL; touch index.html; wget $URL >/dev/null 2>&1; COUNT=1; \
echo Downloading files now\:; fi; FILES=`cat index.html.$COUNT | grep "a href" \
| sed -e 's,^.*=",,' -e 's,".*$,,' | grep /$`; if [ -n "$FILES" ]; then for X in \
$FILES; do wget $X >/dev/null 2>&1; echo -ne "\033[80DProgress \
Meter: `expr \( $COUNT \* 100 / \`ls -tr | tail -1 | sed -e 's,^.*index\.html\.,,'\` \) `%\033[K"; \
done; else echo -ne '.'; fi; COUNT=`expr $COUNT + 1`; if [ ! -e index.html.$COUNT ]; \
then break; fi; done; } 

ls index.html* >/dev/null 2>&1 && {
echo you should either remove these index.html files or make another directory and change to it; 
} || {
URL=ftp://ftp.compaq.com;
COUNT=;
while :; do 
if [ -z "$COUNT" ]; then 
	echo; 
	echo Just one moment while I get ready to index $URL; 
	touch index.html; 
	wget $URL >/dev/null 2>&1; COUNT=1; 
	echo Downloading files now\:; 
fi; 
FILES=`cat index.html.$COUNT | grep "a href" | sed -e 's,^.*=",,' -e 's,".*$,,' | grep /$`; 
if [ -n "$FILES" ]; then 
	for X in $FILES; do 
		wget $X >/dev/null 2>&1; 
		echo -ne "\033[80DProgress Meter: `expr \( $COUNT \* 100 / \`ls -tr | tail -1 | sed -e 's,^.*index\.html\.,,'\` \) `%\033[K"; 
	done; 
else 
	echo -ne '.'; 
fi; 
COUNT=`expr $COUNT + 1`; 
if [ ! -e index.html.$COUNT ]; then break; fi; done; 
} 



You can test the above block of code by making a new directory, cd to that directory,
then copy (highlight) paste this block of code to the bash prompt.  You may have to 
press enter on the last line.  I successfully copy/pasted the above block and it worked.

Just keep in mind that the progress meter will start out close to 100%, then 
will go lower, then will bounce around on it's way back to 100%.  When it
gets back to 100% it should be done.  Sorry, but it doesn't travel smoothly
from 0% to 100% like some other status bars do (like any of them could 
actually give accurate progress results...).