...making Linux just a little more fun!
J.Bakshi [j.bakshi at icmail.net]
Hello,
Hope you all are well.
I have designed a script which compress some files and store those files in a folder named by date like 10-11-2008 ( for 10th Nov 2008 ) and uploads it in a remote ftp server through lftp. the script runs every day. As the FTP space is little after every 15 or so days the FTP space qouata is full. I like to include a section in the script which will check the folders in the ftp server through lftp and deletes the folders which already 10 days matured.
Can any one suggest me how to do this throug lftp ?
thanks a lot
Neil Youngman [ny at youngman.org.uk]
On Saturday 22 November 2008 17:12:10 J.Bakshi wrote:
> Hello, > > Hope you all are well. > > I have designed a script which compress some files and store those files in > a folder named by date like 10-11-2008 ( for 10th Nov 2008 ) and uploads it > in a remote ftp server through lftp. the script runs every day. As the FTP > space is little after every 15 or so days the FTP space qouata is full. I > like to include a section in the script which will check the folders in the > ftp server through lftp and deletes the folders which already 10 days > matured. > > Can any one suggest me how to do this throug lftp ?
Lftp has an rm -r command, which is the obvious option to use.
You can probably work out the directory names to remove with a little bit of perl/python to do the date calculations.
HTH
Neil
J.Bakshi [j.bakshi at icmail.net]
On Sunday 23 Nov 2008 3:22:16 pm Neil Youngman wrote:
> On Saturday 22 November 2008 17:12:10 J.Bakshi wrote: > > Hello, > > > > Hope you all are well. > > > > I have designed a script which compress some files and store those files > > in a folder named by date like 10-11-2008 ( for 10th Nov 2008 ) and > > uploads it in a remote ftp server through lftp. the script runs every > > day. As the FTP space is little after every 15 or so days the FTP space > > qouata is full. I like to include a section in the script which will > > check the folders in the ftp server through lftp and deletes the folders > > which already 10 days matured. > > > > Can any one suggest me how to do this throug lftp ? > > Lftp has an rm -r command, which is the obvious option to use. > > You can probably work out the directory names to remove with a little bit > of perl/python to do the date calculations. >
Hello,
I stuck exactly here. The folder names are like 10-11-2008 11-11-2008
12-11-2008 and so on. lftp is not suppose to support pattern matching; even
it is not possible to calculate the date with in lftp
Any suggestion is most welcome.
thanks
Neil Youngman [ny at youngman.org.uk]
On Sunday 23 November 2008 10:03:48 J.Bakshi wrote:
> On Sunday 23 Nov 2008 3:22:16 pm Neil Youngman wrote: > > You can probably work out the directory names to remove with a little bit > > of perl/python to do the date calculations. > > Hello, > > I stuck exactly here. The folder names are like 10-11-2008 11-11-2008 > 12-11-2008 and so on. lftp is not suppose to support pattern matching; even > it is not possible to calculate the date with in lftp> Any suggestion is most welcome.
Obviously you need to do the calculation externally. I don't use lftp, but I think you should try something like this:
$ DATE=`perl -we "date calculation"` $ lftp -c rm -r $DATE
HTH
Neil
Ben Okopnik [ben at linuxgazette.net]
On Sun, Nov 23, 2008 at 03:33:48PM +0530, J.Bakshi wrote:
> On Sunday 23 Nov 2008 3:22:16 pm Neil Youngman wrote: > > > > Lftp has an rm -r command, which is the obvious option to use. > > > > You can probably work out the directory names to remove with a little bit > > of perl/python to do the date calculations. > > I stuck exactly here. The folder names are like 10-11-2008 11-11-2008 > 12-11-2008 and so on. lftp is not suppose to support pattern matching; even > it is not possible to calculate the date with in lftp
The first problem is the format of the date you're using. Yes, it's possible to calculate using those - but you have to do some really heavy-duty processing (leap years, etc.) before you can get a stable base for calculating anything. You'd be much better off using either the RF3339 format ('date --rfc-3339=date') or the epoch (time in seconds since 1/1/1970 12:00:00; e.g., "perl -we'print time'.) The latter is especially useful for the kind of thing you're trying to do.
The other approach would be to ignore whatever you used for the name and to use the file's timestamp - just as the script that I pubbed here last month did.
savedir=/my/backup/directory keeptime=30 # Delete files older than $keeptime days /usr/bin/find $savedir -ctime +$keeptime -delete
-- * Ben Okopnik * Editor-in-Chief, Linux Gazette * https://LinuxGazette.NET *
J.Bakshi [j.bakshi at icmail.net]
On Sunday 23 Nov 2008 8:24:52 pm Ben Okopnik wrote:
> On Sun, Nov 23, 2008 at 03:33:48PM +0530, J.Bakshi wrote: > > On Sunday 23 Nov 2008 3:22:16 pm Neil Youngman wrote: > > > Lftp has an rm -r command, which is the obvious option to use. > > > > > > You can probably work out the directory names to remove with a little > > > bit of perl/python to do the date calculations. > > > > I stuck exactly here. The folder names are like 10-11-2008 11-11-2008 > > 12-11-2008 and so on. lftp is not suppose to support pattern matching; > > even it is not possible to calculate the date with in lftp
Thanks for your response.
I can access the remote ftp space through FTP, hence I have to check if the script can run internally via lftp.
Thjanks
Ben Okopnik [ben at linuxgazette.net]
On Sun, Nov 23, 2008 at 08:34:59PM +0530, J.Bakshi wrote:
> > I can access the remote ftp space through FTP, hence I have to check if the > script can run internally via lftp.
Well, nothing can "run internally" via FTP - that's not how FTP works. You can execute commands on your local machine via the "!command" syntax, but not on the remote machine. Given that FTP passwords and such go across the wire as plain text, it wouldn't make much sense to allow an FTP client to exec a command, right?
For what you describe, I suggest you consider SSH. It should work just fine.
-- * Ben Okopnik * Editor-in-Chief, Linux Gazette * https://LinuxGazette.NET *
J.Bakshi [j.bakshi at icmail.net]
On Monday 24 Nov 2008 4:25:55 am Ben Okopnik wrote:
> On Sun, Nov 23, 2008 at 08:34:59PM +0530, J.Bakshi wrote: > > I can access the remote ftp space through FTP, hence I have to check if > > the script can run internally via lftp. > > Well, nothing can "run internally" via FTP - that's not how FTP works. > You can execute commands on your local machine via the "!command" > syntax, but not on the remote machine. Given that FTP passwords and such > go across the wire as plain text, it wouldn't make much sense to allow > an FTP client to exec a command, right? > > For what you describe, I suggest you consider SSH. It should work just > fine.
Yes, with ssh all these could work fine. Unfortunately I the remote ftp space provides ony ftp access.
Thanks
Francis Daly [francis at daoine.org]
On Sun, Nov 23, 2008 at 03:33:48PM +0530, J.Bakshi wrote:
> On Sunday 23 Nov 2008 3:22:16 pm Neil Youngman wrote: > > On Saturday 22 November 2008 17:12:10 J.Bakshi wrote:
> > > I have designed a script which compress some files and store those files > > > in a folder named by date like 10-11-2008 ( for 10th Nov 2008 ) and > > > uploads it in a remote ftp server through lftp. the script runs every > > > day. As the FTP space is little after every 15 or so days the FTP space > > > qouata is full. I like to include a section in the script which will > > > check the folders in the ftp server through lftp and deletes the folders > > > which already 10 days matured.
> > Lftp has an rm -r command, which is the obvious option to use. > > > > You can probably work out the directory names to remove with a little bit > > of perl/python to do the date calculations.
> I stuck exactly here. The folder names are like 10-11-2008 11-11-2008 > 12-11-2008 and so on. lftp is not suppose to support pattern matching; even > it is not possible to calculate the date with in lftp
Your script runs every day. So just delete the folder from 10 days ago.
If you once, manually, delete anything older than that, then as the script keeps running, it will never build up again.
Your script currently does something like
new_folder=$(date +%d-%m-%Y) lftp_upload_stuff_to $new_folder
where lftp_upload_stuff_to presumably includes "mkdir" and "mput".
So add in
old_folder=$(date -d '10 days ago' +%d-%m-%Y) lftp_delete $old_folder
where lftp_delete will presumably include "rm -r".
All the calculation should happen in the script that drives the ftp client, not in the client itself.
Good luck,
f
-- Francis Daly francis@daoine.org
J.Bakshi [j.bakshi at icmail.net]
On Monday 24 Nov 2008 2:45:05 pm Francis Daly wrote:
> On Sun, Nov 23, 2008 at 03:33:48PM +0530, J.Bakshi wrote: > > On Sunday 23 Nov 2008 3:22:16 pm Neil Youngman wrote: > > > On Saturday 22 November 2008 17:12:10 J.Bakshi wrote: > > > > I have designed a script which compress some files and store those > > > > files in a folder named by date like 10-11-2008 ( for 10th Nov 2008 ) > > > > and uploads it in a remote ftp server through lftp. the script runs > > > > every day. As the FTP space is little after every 15 or so days the > > > > FTP space qouata is full. I like to include a section in the script > > > > which will check the folders in the ftp server through lftp and > > > > deletes the folders which already 10 days matured. > > > > > > Lftp has an rm -r command, which is the obvious option to use. > > > > > > You can probably work out the directory names to remove with a little > > > bit of perl/python to do the date calculations. > > > > I stuck exactly here. The folder names are like 10-11-2008 11-11-2008 > > 12-11-2008 and so on. lftp is not suppose to support pattern matching; > > even it is not possible to calculate the date with in lftp
Great !!
It is really what I am looking for. Millions of thanks