Scripting using bash commands: Dividing #'s in Different Columns

[evac-q8r]

Hello Everyone,

If I have a file with two columns and n lines or records of data what command can I use to divide value in the first column by value in the second column if there is any.

Thanks a Million,

EVAC

[ph34r]

cat cut and xargs

[dkeav]

that would work, i would probably use awk over cut and expr to do the math

[evac-q8r]

Got it. Great!

[dkeav]

you might post some examples of your script, and problem for future users who search google/forums for a such a task

[voidinit]

Bash is capable of arithmatec expansion without using any external programs. Much faster and more effiecient. If you require floating point precision, then it's trickier to do internally but it's still possible.

If given the following file:


cat > numbers.txt << EOF
20 10
38 8
31 12
57 6
23 8
16 9
EOF


Remainder division is easiest:


#!/bin/bash

#remainder division
echo "Processing numbers.txt with remainder division."
while read num den
do
quot=$(( $num / $den ))
rem=$(( $num % $den ))
echo "$num divided by $den is $quot remainder $rem."
done < numbers.txt


For floating point divison, multiply your numerator by 10 ^ $decimal_places, and then do your division. Then inject a decimal point $decimal_places from the end of your number string.


#floating point division with double precision
echo ""
echo "Processing numbers.txt with floating point division"
while read num den
do
num=$(( $num * 100 ))
quot=$(( $num / $den ))
#inject the decimal point where appropriate
if [ ${#quot} -gt 2 ]; then
final="${quot:0:$(( ${#quot} - 2))}.${quot:$(( ${#quot} - 2 )):${#quot}}"
else
final=".$quot"
fi
echo "$(( num / 100 )) divided by $den is $final"
unset final
done < numbers.txt



The output of this is

Processing numbers.txt with remainder division.
20 divided by 10 is 2 remainder 0.
38 divided by 8 is 4 remainder 6.
31 divided by 12 is 2 remainder 7.
57 divided by 6 is 9 remainder 3.
23 divided by 8 is 2 remainder 7.
16 divided by 9 is 1 remainder 7.

Processing numbers.txt with floating point division
20 divided by 10 is 2.00
38 divided by 8 is 4.75
31 divided by 12 is 2.58
57 divided by 6 is 9.50
23 divided by 8 is 2.87
16 divided by 9 is 1.77

[evac-q8r]

Thanks SO Much for that voidinit. Very Comprehensive and beautifully done!

EVAC

[Icarus]

kudo voidinit! Always nice to see posts with a real answers other then man/rtfm :D

[dkeav]

if needed yes, otherwise if we give you the answer to just copy and paste did you learn anything? no

and now we have just reinforced a bad habit, if we dont know we ask and they will just give us the answer without me having to lift a finger YAY!

[evac-q8r]

That's a very good point dkeav.

After having looked at the solution for some time voidinit and as elegant and as it appears, there is much more at work here than meets the eye which is why I'm sure you regarded the solution as being 'tricky'. With that said, there are actually a question or two I may have. I cannot seem to figure out what the colons are doing in the conditional. I also cannot understand the hash # (normally a comment symbol) which you have inserted in a few instances in front of the quot variable. Could you clarify these two things as my references don't seem to explain these symbols...

Thanks a Million

[voidinit]

It will make more sense if I answer the second question first.


I also cannot understand the hash # (normally a comment symbol) which you have inserted in a few instances in front of the quot variable. Could you clarify these two things as my references don't seem to explain these symbols...


Well the '#' is the length operator.


#Lenght of strings.
foo="This is a string."
echo "${#foo}" #output 17
unset foo

#Length of Arrays.
foo=( "one" "two" "three" "four" ) #declare an array
echo "${#foo}" #output is 4

#Length of Array members
echo ${#foo[0]} #output is 3




I cannot seem to figure out what the colons are doing in the conditional.


The colons are for trailing substring extraction. substring=${oldstring:index:length} Basically they make a substring of the string starting at index up through length.


foo="This is a string"
bar=${foo:0:4}
echo "$bar" #Out is This



final="${quot:0:$(( ${#quot} - 2))}.${quot:$(( ${#quot} - 2 )):${#quot}}"


The above basically means:

Final equals a substing of $quot from index 0 through the length of $quot minus 2. Then append a period, then append a substring of $quot from index lenght of $quot minus 2 through length of $quot.

Have a gander at the arrays section of the Advanced Bash Scripting Guide from tldp: http://www.tldp.org/LDP/abs/html/arrays.html

[Icarus]

I fail to see what is wrong with voidinit's examples, he's not saying "Here's the answer" he's saying "It's something like this". People can read man pages all day long and understand the concept of things, but without examples it's mostly a jumble of words. And explaining the examples is what he's really doing right ;)

[voidinit]

if needed yes, otherwise if we give you the answer to just copy and paste did you learn anything? no

and now we have just reinforced a bad habit, if we dont know we ask and they will just give us the answer without me having to lift a finger YAY!

Rarely do I say "go RTFM" or "go google for it" where bash is concerned. Usually the examples and answers you get are just plain wrong! Unless they come from reputable sources that have been using bash for quite a while, like the author of TLDP's Advanced Bash Scripting Guide, then there is a good chance that the document you get is a good example of how not to do something. If someone is asking for bash help, then they probably aren't experienced enough with bash to know the difference.

Most unix users see bash as a sort of "glue" that holds together strings of external commands. Their examples and such on the web greatly reflect this. Left to google, evac-q8r might have ended up with some sort of 'for cat file do bc pipe sed pipe awk' kludged together monstrosity. While these sorts of commands glued together might be great for quick hacks and one liners, they are not what linux users, especially new ones, should be learning to do with bash.

Also, there isn't really a good "cook book" for bash like there are for perl and other scripting languages. Most of what you get on the web are either the kludged together examples of how bash launches external programs or text book style presentation of bash's many faucets. It's hard for someone learning bash to see how all these peices work together.

[ghostdog74]

if you have Python:


>>> from __future import division
>>> print "processing floating point numbers"
processing floating point numbers
>>> for lines in open("numbers.txt"):
... a,b = lines.split()
... c = int(a) / int(b)
... print "%s divided by %s is %s" %(a,b,str(c))
...
20 divided by 10 is 2.0
38 divided by 8 is 4.75
31 divided by 12 is 2.58333333333
57 divided by 6 is 9.5
23 divided by 8 is 2.875
16 divided by 9 is 1.77777777778



>>> print "Processing with remainder division"
Processing with remainder division
>>> for lines in open("numbers.txt"):
... a,b = lines.split()
... q = int(a) / int(b)
... r = int(a) % int(b)
... print "%s divided by %s is %s, remainder is %s" %(a,b,str(q),str(r))
...
20 divided by 10 is 2, remainder is 0
38 divided by 8 is 4, remainder is 6
31 divided by 12 is 2, remainder is 7
57 divided by 6 is 9, remainder is 3
23 divided by 8 is 2, remainder is 7
16 divided by 9 is 1, remainder is 7


more readable.

[evac-q8r]

That's a very nice contribution as well ghostdog. It saves time especially when you want to do floating point division. I have never used python, but that example will definitely get me to consider it more as a scripting/programming language. I'm somewhat baffled the neither bash nor expr would do floating point division for you. It seems somewhat strange to me that this basic calculator function has been left out. Can anyone explain this? Is anyone aware of an executable that will perform floating point division. I suppose at the end of the day I can always code up a 10-liner in C and use that as an executable within the script. Not asking anyone to do that for me though... Don't want anyone to get angry :rolleyes: (Just Kidding)

EVAC

[ghostdog74]

I'm somewhat baffled the neither bash nor expr would do floating point division for you.


if you find that they don't provide this facility , then use another tool.
after all, there are plenty to choose from, esp in linux env :) . bottomline is, use the right tool for the problem to solve.
regarding ur qns about an executable that does floating point, you can try the bc command.

[voidinit]

The most useful thing about bash is that any program any on the system can be a "plugin" for you, if you want to pay the price of calling an external program. Usually, calling an external program is just fine, it depends on the performance/memory requirements of your script.

There are a number of external programs that can do floating poing arithmatic for you. bc is one of them, I think there is one called calc or something like that.

Another option is to bash function. I have a standard library of bash functions on my box that use all the time. All you have to do is keep them in a file, and source them if you want to use them.