Unattended Batch ftp

[mattmorrow]

Want to upload or download via ftp in an unattended batch mode?

When running sar (performance measurements) on several remote workstations, I needed to grab the results each morning in the wee hours for post-processing. I used a simple shell script and ftp in a batch mode. Here's the ftp batch file:

debug
open
200.200.175.1 #IP to ftp to
user yourname yourpassword
case
prompt off
ascii # or binary
cd /dir/on/remote_machine
lcd /dir/on/local_machine # avoid this by doing a "cd" before executing
get myfile.text # or mget, put, whatever...
bye

Have your shell script call it via:
ftp -n < batch.ftp

(-n makes it so you don't need .netrc)

[bdg1983]

Do you know of a way to rig an ftp program that would resume a download as soon as it breaks, and would allow you to execute a command before resuming the download (such as restarting the dhcpc daemon). This would be of measureless value to me...

[This message has been edited by bdg1983 (edited 29 December 2000).]

[Sterling]

bdg - I'm surprised there isn't already at least one FTP program that does this... It sounds interesting, and very useful, so you should have no trouble hooking a programmer or two. http://www.linuxnewbie.org/ubb/wink.gif I'd volunteer, but I've already got enough drains on my time as it is.

------------------
-Sterling
-This post made with the Lizard! (http://www.mozilla.org)

[Craig McPherson]

You could shorten your script down to this, assuming you're just downloading things rather than uploading.

cd /dir/on/local/machine && wget -t3 -g ftp://yourname:yourpassword@200.220.175.1/dir/on/remote_machine/myfile.txt

wget could do roughly what you describe, bdg1983. Use the -t0 argument to specify an unlimited number of retries, -c for resuming, and --execute to execute your command. See the man page for more details.

You know the drill, apt-get install wget if you don't have it.

If you want a more flexible way of scipting interactive programs like FTP, use expect. The weakness of shell scripting is that it's basically useless when it comes to interactive programs. expect can, and it gets around unfriendly programs that have to be run from a "real" terminal.

Here's an expect script to log into a FTP server, upload AND download a few things, based on an example in the Red Book (well, Purple Book now).

spawn /usr/bin/ftp 200.200.175.1
while 1 { expect {
"Name*: " {send "yourname\r"}
"Password: " {send "yourpassword\r"}
"ftp> " {break}
"failed" {send_user "Can't log in.\r"; exit 1}
timeout {send_user "Timeout problem.\r"; exit 2}
}}
send "lcd /yourdir\r"
expect "ftp> " {send "cd /theirdir\r"}
expect "ftp> " {send "asc\r"}
expect "ftp> " {send "get theirfile\r"}
expect "ftp> " {send "put yourfile\r"}
expect "ftp> " {send "quit\r"; send_user "\r"}
exit 0

Of course, you can put in all kinds of loops and conditionals and error traps to make a "smart" automatic FTP script, and encorporate any other kind of system command, but you'll need to know some Tcl for that.

[This message has been edited by Craig McPherson (edited 03 January 2001).]

[Sterling]

You might also be able to do something with scp, although the requirement of passwords there could pose a problem...

------------------
-Sterling
-This post made with the Lizard! (http://www.mozilla.org)