Mirroring With Mirrordir

[QAPete]

Hello, all, I'm hoping someone can help me out with a script I'd like to write to do some automatic mirroring with mirrordir (http://mirrordir.sourceforge.net).

Here's the scenario:

I want to log into a site (ftp://username :password@ftpsite.com), cd to the appropriate directory (let's call it sourcedir1) copy all new files to my /www/files/localdir1, then proceed to do the other 7 folders that are on the same directory tree level as sourcedir1. There are some dirs on the remote machine that I do not wish to mirror.

Copydir (part of mirrordir) can handle the copying and has the appropriate switches to not download corrupted files, etc., but I don't know how to get the sequence of events right in a script that I can later cron.

Appreciate any help you guys can give me, even other methodologies for doing this. This is my first post in this forum, so I'm anxious to see what kind of help is really available here.

Pete

[bdg1983]

There's all kinds of help available here at LNO, but if we are not familiar with whatever you are using (such as I), then you will most likely not get an answer.

We don't know the solution to everything that get's asked.

Are you trying to write a shell script or modify something you got from mirrordir?

[QAPete]

My end goal is to have an automated way of 'freshening' a set of mirrored folders, with the source being on an ftp server and the mirror being on my new Linux box.

I have 99.9% of it as of a couple of days ago (done using wget), but in looking at mirrordir (http://mirrordir.sourceforge.net), I noted a number of desirable command line switches that would, among other things, prevent the mirroring of incomplete files, force the downloading of only files that have changed/been added since the last mirror job, etc.

I'm open to using wget for this task as well, but could use some help getting a proper command line to do the above.

On a side note, I did get mirrordir (actually, copydir) to log in and start doing its job, but it started downloading ALL files again, rather than simply freshen the mirror, so now I'm at a standstill with that command for that reason.

Pete

[bdg1983]

Of what I quickly read about mirrordir and copydir, it looks like it uses 'cp' in the scripts. cp accepts -u or --update to only copy new or changed files.

I got that from the cp man pages that are referenced on their website, so they must be valid parameters you can use for their products.

[dvdnut]

what u could do is write a script in which all the commands you need to obtain this remote dir, using this mirroridir app.

then using cron

run it as often as you wish