Is there an easy way to do this (Perl)?

[Energon]

I have an array of a bunch of file names... and right now I go through the entire array making sure each file exists and aborting if one doesn't... I'd like to set it up instead so that it removes the file from the list (they're basically just string values) and continues... the array gets passed through a system call to tar so they can be archived, so the array has to be cleaned up (or tar will bail on me)... I can't think of an easy way to do this other than creating a new array each time a file has to be removed from the list, and even that is kind of not working in my head (I can't quite get the idea into Perl speak)...

So is there an easier way to do this?

[EyesWideOpen]

I recently had to do something like this. I think splice (http://www.perl.com/pub/doc/manual/html/pod/perlfunc/splice.html) is what you're looking for.


splice(@filenames_array, $element_to_be_deleted, 1);


This will delete the value at position $element in array @filenames_array.

;)

[feve]

if you want to remove them from the current array look into the splice function (from a command line type perldoc -f splice).


for (0..5) {
$arr[$_] = $_;
}

print "My array is @arr.\n";

splice(@arr,1,2);

print "My array is @arr.\n";


This should fill up the array with elements 0 through 5. In this example the splice function removes 2 elements starting at position 1. Therefore the second print statement shows the array having only the elements 0 3 4 5.

[Energon]

so I could do this:

foreach(@files) {
unless(-e) {
splice @files, $_, 1;
}
}

and that should remove the element from the list? Wow... cool... that function isn't in Learning Perl... time to pick up the next book i think...

[Energon]

nope... that won't work... what about this:


my $count = 0;
foreach(@files) {
$count++;
unless(-e) {
splice @files, $count, 1;
$count--;
}
}


that would be the right solution, wouldn't it?

[EyesWideOpen]

That should work. I don't think you want to decrement $count though. You will need to keep it in sync with the next element in the array so you will have that element to pass to splice as an argument if/when you want to delete the next file from the list.


my $count = 0;

foreach(@files) {
$count++;

unless(-e) {
splice(@files, $count, 1);
}
}


[ 09 May 2001: Message edited by: EyesWideOpen ]

[feve]

I hope i'm not missing something, but i think there might be a problem with incrementing count and then using it in the splice function.

For example. The first time in the loop $count is incremented to 1. If the first file is not found, the element at position 1 is removed. But doesn't the first element have an index of 0?

I was thinking of something like this . . .


for (0..$#files) {
unless (-e $files[$_]) {
print "$files[$_] does not exist\n";
splice (@files,$_,1);
}
}


---- edit -----

Glad you got the count method to work, because i just played around with my version here and it doesn't work in some circumstances (e.g. good file, bad file, bad file). There is a problem with the fact that $#files changes in some iterations (and so the last bad file isn't checked) Sorry if i added confusion.

[ 09 May 2001: Message edited by: feve ]

[Energon]

actually, the solution would be this:


my $count = 0;
foreach(@files) {
$count++;
unless(-e) {
splice @files, --$count, 1;
}
}


that way, assuming it's a one element array, then it puts count at 1, and if that element doesn't exist, it decrements count back to 0 and deletes the 0th element of the array (or the 1st element as they say in English)... and then on the next pass, it starts back at 0 if there's more than one element...

or does splice work in the non-programmers counting system (ie, the first element to delete is 1 instead of 0)?

[EyesWideOpen]

for (0..$#files) {
unless (-e $files[$_]) {
print "$files[$_] does not exist\n";
splice (@files,$_,1);
}
}


This does seem like the best solution. ;)

My last example should have been this:


my $count = 0;
foreach(@files) {
unless(-e) {
splice @files, $count, 1;
}
$count++;
}


so that $count starts at 0 and increments after the first loop. I tested this however and it behaves funky. Not the incrementing part but the looping through the array elements part (foreach(@files)) so you're better off going with feve's example.

[ 09 May 2001: Message edited by: EyesWideOpen ]

[Energon]

a for(0...$#files) is the same as a foreach... except that you get the count already... but aren't you guys forgetting that when you splice out a member of the array, the number of elements is reduced and the next value becomes the current one... kinda like if you have n things, and you delete the k element, then the k+1 element gets the k spot (ie, moved down the line)...

I may have that wrong, but it's how it works out in my head... if you have 10 items and delete the 5th, then the 6th item has to become the 5th or you have a big gaping hole there... which is something I have to avoid in order to pass the array to tar...

[feve]

EyesWideOpen,

thanks for the credit but i just had to edit my post. There are some cases that i didn't see right away where $#files is changed and then the last element isn't checked. So i can't come up with anything other than keeping count of the elements in a seperate variable.

Energon,

I definately see you point, but i tried your code with the same test data in the files array (valid, invalid1, invalid2) and the last file isn't being checked. When invalid2 is "pulled" into invalid1's spot, the foreach is done because there's now no longer an element in position 2. Maybe if we got the array size first?

[ 09 May 2001: Message edited by: feve ]

[Energon]

so maybe something like this:


my $elements = $#files;
my $count = 0;
while($elements > 0) {
$count++;
unless(-e) {
splice @files, --$count, 1;
$elements -= 2;
} else {
$elements--;
}
}
}


I haven't tested it, but it keeps track of the full # of elements I think, and it keeps track of which element we're working with, even with deleting some of them...

EDIT - stupid stuff needed fixed...

[ 09 May 2001: Message edited by: Energon ]

[feve]

I tried the last "while elements > 0" and it still gets stuck on one good then two bad cases. I think after finding and removing the first invalid one $elements drops down to -1.

i think this does it . . .


for ($i = 0; $i <= $#files; $i++) {
unless (-e $files[$i]) {
print "$files[$i] does not exist\n";
splice @files, $i--, 1;
}
}


Not a very perl like for loop but keeping the element to be deleted in a seperate variable seems to help. I'm still kind of wary about the <= $#files.

Sorry again about the confusion, i thought i had simple answer for you and i feel as though i've led you all over the place.

[Energon]

it's all good... it's been an interesting discussion... I'm also a little concerned about the <$#files, but I can't really say whether it should work... my reasoning says you'll overbound the array if you're not careful, but I can't say for sure...

[Mikey123]

Haven't read every post in detail but anyone suggest this

for(@array_of_files){
push @files_that_exist,$_ if -e
}
# do stuff here
# with @files_that_exist

[Mikey123]

Or how bout:


my @files_that_exist = grep{ -e }@list_of_files;

[Energon]

well, that would work except I think it would break the stuff I need inside of the code... I have to print which files don't exist to a log, and in the end, I also have to eliminate files older than 2 days if the backup type is incremental (-M < 2)...

And I'm not sure yet which is better, using cpu or memory... the former seems to use more cpu, but the one you suggested seems to use more memory... but I don't know for sure...

[YaRness]

maybe i'm just missing part of the problem, but i think it's fairly simple....


my @list = (1,2,3,4,5); #or whatever
#to get rid of an element at index $index:
delete $list[$index];


and, perl has a tar module if you are interested:


#from the activeperl docs
use Archive::Tar;

$tar = Archive::Tar->new();
$tar->read("origin.tar.gz",1);
$tar->add_files("file/foo.c", "file/bar.c");
$tar->add_data("file/baz.c","This is the file contents");
$tar->write("files.tar");

no doubt you can do somethin like "$tar->add_files(@list)"

try "perldoc Archive::Tar" or look for it on CPAN (www.perl.com/CPAN/)

[EyesWideOpen]

my @list = (1,2,3,4,5); #or whatever
#to get rid of an element at index $index:
delete $list[$index];
[/qb]<HR></blockquote>

:eek: I didn't even realize you could use delete on an array because I've only ever used it (and seen it used) with a hash.

:(

[ 10 May 2001: Message edited by: EyesWideOpen ]

[Mikey123]

I believe grep is the proper one to use to select items from a list. The following will do what you want I think.


@files_that_dont_exist = grep{ !-e }@list_of_files;
# Remove files that don't exist and (-M < 2)
@list_of_files = grep{ -e and (-M < 2)}@list_of_files;

then you can log your @files_that_dont_exist and do whatever you want with the rest. @list_of_files will now only contain files that exist and match (-M < 2)

[YaRness]

Originally posted by EyesWideOpen:
<STRONG>I didn't even realize you could use delete on an array because I've only ever used it (and seen it used) with a hash.</STRONG>
do {&lt;read the documentation&gt;} while (1)

i'm constantly reading the docs, because i never remember how to do anything.

[Mikey123]

Yar,

It is probably important to point out that 'delete' on an array element is not the same as splice'ing it. delete simply removes the value from the element but does not remove the element. Splice removes the element.


my @list = qw(one two three four);
print "elements before delete=".scalar(@list)."\n"
delete $list[1];
print "elements after delete=".scalar(@list)."\n"

@list = qw(one two three four);
print "elements before splice=".scalar(@list)."\n"
splice @list,1,1;
print "elements after splice=".scalar(@list)."\n"

[Energon]

Originally posted by Mikey123:
<STRONG>Yar,

It is probably important to point out that 'delete' on an array element is not the same as splice'ing it. delete simply removes the value from the element but does not remove the element. Splice removes the element.
</STRONG>

this is probably my biggest thing right now... I need the element totally gone... :\

But I'll look into the tar module... I've never used any modules before so it's probably a good time for me to start... just have to see if I have it installed or not...

[YaRness]

Originally posted by Mikey123:
<STRONG>Yar,

It is probably important to point out that 'delete' on an array element is not the same as splice'ing it. delete simply removes the value from the element but does not remove the element. </STRONG>
i'll be damned, so it doesn't.


Deleting an array element effectively returns that position of
the array to its initial, uninitialized state. Subsequently
testing for the same element with exists() will return false.
Note that deleting array elements in the middle of an array will
not shift the index of the ones after them down--use splice()
for that. See the exists entry elsewhere in this document.

i'm just full of stupid things to say about perl today.

[Mikey123]

my earlier post will give you a list with just the filenames you want;


@files_that_dont_exist = grep{ !-e }@list_of_files;
# Remove files that don't exist and (-M &lt; 2)
@list_of_files = grep{ -e and (-M &lt; 2)}@list_of_files;

[YaRness]

how many times do you need to use the list? if it's just once, then i'd check for existence as you are doing &lt;whatever&gt; to each file:


my @noexist;
foreach (@list)
{
if (-e)
{
&lt;do stuff&gt;
}
else
{
#here you can save nonexistent names for later
#or you could just do something with them now
push @noexist, $_;
#or whatever
}
}

[Energon]

As it stands now, I can't do it that way because the array is passed to tar... it's pretty simple... though once I read up about the tar module, I may end up going that route instead...