[CoLoCo] bash question

[Neal McBurnett]

I started doing systems-level stuff in Unix with the shell, awk etc
also (that being all there was back then besides C).  But it is pretty
grungy from a modern perspective.  A good investment for your future
would be learning a real language.  Now I know this is a controversial
topic, and I don't want to see a flame war or anything, but I suggest
you look at the options and pick one.

My suggestion is python, since it is widely used for system-level
stuff (e.g. in Ubuntu) as well as the web, and it is clean and
powerful.  Alternatives might be Ruby or even Javascript, which has
its share of server-related applications.  I'd stay away from PHP -
just too much of a history of insecure and patchy design there.  Perl
is old-school cool, but inscruitable.

Cheers,

Neal McBurnett                 http://neal.mcburnett.org/

On Fri, Aug 19, 2011 at 03:17:38PM -0600, Jim Hutchinson wrote:
> Thanks for the options. My problem with python or php is that I don't know
> those at all and it's hard for me to decipher the scripts to understand what
> they are doing or to modify them if needed. Of course, I could just run them on
> faith and will give it a try.
> 
> Joey, for the big2.log reference, is that the file that would have all the
> files names I want to process? If so, full path in there or can I use sort path
> and run the scrip from the same folder?
> 
> Using awk was easier
> 
> awk '{sum=sum+$3} END {print sum}' file1.txt
> 
> Worked fine but only after manually taking out the header row. Still not sure
> how to just ignore it. Also not sure how to read in multiple files and output
> to one file.
> 
> I'll give this a try (David's idea too) and see what happens.
> 
> Thanks,
> Jim
> 
> On Fri, Aug 19, 2011 at 3:08 PM, Joey Stanford <joey at canonical.com> wrote:
> 
>     I think the easier way is going to be with awk ... but here's a python
>     program that's roughly equivalent...just not looking for the 3rd field
> 
>     #! /usr/bin/env python
> 
>     data = open('big2.log')
>     totals = {}
>     for line in data:
>        line = line.strip()
>        if line:
>            pageid = line.split()[1]
>            pagecount = int(line.split()[0])
>            if pageid in totals:
>               totals[pageid] += pagecount
>            else:
>               totals[pageid] = pagecount
> 
>     for key in totals:
>            print totals[key], key
> 
> 
>     On Fri, Aug 19, 2011 at 14:45, Jim Hutchinson <jim at ubuntu-rocks.org> wrote:
>     > Wondering if any of you script gurus can help with a small problem. I
>     have
>     > several text files containing 3 columns. I was to count the number of
>     > occurrences of the text in column 2 (or just count the lines) and sum
>     column
>     > 3 which is a number. I know how to do the latter with something like
>     >
>     > #!/bin/bash
>     >
>     > file="/home/test/file1.txt"
>     > cat ${file} | \
>     > while read name article count
>     > do
>     > sum=$(($sum + $count ))
>     > echo "$sum"
>     > done
>     >
>     > Although that prints each sum as it goes rather than just the final sum.
>     > I'm not sure how to count text (basically counting the lines that contain
>     > the numbers would work the same). Also, because each file has a header
>     row
>     > it's giving errors so I need to tell it to skip row 1.
>     > Finally, I want to automate the input of each file so having it read the
>     > list of text files from somewhere, process the file, output to a new file
>     > amending each time, and then repeat with the next one until all files are
>     > done.
>     > Any ideas?
>     > Thanks.
>     > --
>     > Jim (Ubuntu geek extraordinaire)
>     > ----
>     > Please avoid sending me Word or PowerPoint attachments.
>     > See http://www.gnu.org/philosophy/no-word-attachments.html
>     >
>     > --
>     > Ubuntu-us-co mailing list
>     > Ubuntu-us-co at lists.ubuntu.com
>     > Modify settings or unsubscribe at:
>     > https://lists.ubuntu.com/mailman/listinfo/ubuntu-us-co
>     >
>     >
> 
>     --
>     Ubuntu-us-co mailing list
>     Ubuntu-us-co at lists.ubuntu.com
>     Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/
>     listinfo/ubuntu-us-co
> 
> 
> 
> 
> --
> Jim (Ubuntu geek extraordinaire)
> ----
> Please avoid sending me Word or PowerPoint attachments.
> See http://www.gnu.org/philosophy/no-word-attachments.html

> -- 
> Ubuntu-us-co mailing list
> Ubuntu-us-co at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-us-co