Prev: Press conference
Next: 'You are not connected to the internet'
From: Justin C on 15 Jul 2010 08:38

I've added a new command to the Text bundle on my system. I use it to
massage some copy/pasted output from another system. First it uses 'cut'
to remove the stuff that I don't need, then it uses 'sed' to rearrange
the data, I end up with three columns, the third of which I'd like to
add up. Awk can do this, and I have a command for awk that does this,
the problem is that awk just overwrites my data.

I'm looking to add a line to the bottom of the file, print the string
"Total: " followed by the output of my awk code that sums the column -
without awk erasing the data.

I would have gone to a Linux newsgroup for this, but this isn't editing
a file that's saved to disk, it's all pipes and STDIN.

Any suggestions?


Here's what I've got so far:
cut -c1-11 -c24-35 -c55- |
sed 's_^\([0-9]\{4\}/[0-9]\{5\}\) \([-A-Za-z0-9]\{11\}\) \(.\{8\}\)_\2 \1 \3_'

And here's a sample of my data:
1006/01398 1005/012411 10-Jun-2010 95.07
1006/01399 1005/013821 10-Jun-2010 38.73
1006/01400 1005/013851 10-Jun-2010 193.66
1006/01401 1006/019511 10-Jun-2010 318.15
1006/01402 1005/015001 10-Jun-2010 56.34

The code above works as intended with the above data. Any suggestions of
commands that will help complete this will be gratefully received. I'm
not looking for someone to complete this, I'd rather do it myself, but
I don't know where to start looking, so just pointers in the right
direction would be most appreciated.


Justin.
From: Bruce Horrocks on 15 Jul 2010 19:31
On 15/07/2010 13:38, Justin C wrote:
>
> I've added a new command to the Text bundle on my system. I use it to
> massage some copy/pasted output from another system. First it uses 'cut'
> to remove the stuff that I don't need, then it uses 'sed' to rearrange
> the data, I end up with three columns, the third of which I'd like to
> add up. Awk can do this, and I have a command for awk that does this,
> the problem is that awk just overwrites my data.
>
> I'm looking to add a line to the bottom of the file, print the string
> "Total: " followed by the output of my awk code that sums the column -
> without awk erasing the data.
>
> I would have gone to a Linux newsgroup for this, but this isn't editing
> a file that's saved to disk, it's all pipes and STDIN.
>
> Any suggestions?

AWK doesn't pass through data like cut and sed. If you want something to
appear on the output then you have to print it explicitly. So if you
wanted to display all lines starting '1006' then your pattern and action
would be:

/^1006/ { print }

To get a total just add up as you go and use the END pattern to cause it
to be printed at the end. There's an example of summing at
<http://www.gnu.org/manual/gawk/gawk.html#More-Complex>

AWK can do the cut and sed bits as well. So if the purpose of the cut is
to ignore the second column (starting 1005) then

{ print $1, $3, $4 }

will do it, and summing $4 will give you your total.

Use the sub or gsub functions to do the work of sed.

>
>
> Here's what I've got so far:
> cut -c1-11 -c24-35 -c55- |
> sed 's_^\([0-9]\{4\}/[0-9]\{5\}\) \([-A-Za-z0-9]\{11\}\) \(.\{8\}\)_\2 \1 \3_'
>
> And here's a sample of my data:
> 1006/01398 1005/012411 10-Jun-2010 95.07
> 1006/01399 1005/013821 10-Jun-2010 38.73
> 1006/01400 1005/013851 10-Jun-2010 193.66
> 1006/01401 1006/019511 10-Jun-2010 318.15
> 1006/01402 1005/015001 10-Jun-2010 56.34
>
> The code above works as intended with the above data. Any suggestions of
> commands that will help complete this will be gratefully received. I'm
> not looking for someone to complete this, I'd rather do it myself, but

I've tried to resist the temptation of doing it all. :-)

> I don't know where to start looking, so just pointers in the right
> direction would be most appreciated.
>
>
> Justin.


--
Bruce Horrocks
Surrey
England
(bruce at scorecrow dot com)
From: Justin C on 16 Jul 2010 05:32
On 2010-07-15, Bruce Horrocks <07.013(a)scorecrow.com> wrote:
> On 15/07/2010 13:38, Justin C wrote:
>>
>> I've added a new command to the Text bundle on my system. I use it to
>> massage some copy/pasted output from another system. First it uses 'cut'
>> to remove the stuff that I don't need, then it uses 'sed' to rearrange
>> the data, I end up with three columns, the third of which I'd like to
>> add up. Awk can do this, and I have a command for awk that does this,
>> the problem is that awk just overwrites my data.
>>
>> I'm looking to add a line to the bottom of the file, print the string
>> "Total: " followed by the output of my awk code that sums the column -
>> without awk erasing the data.
>>
>> I would have gone to a Linux newsgroup for this, but this isn't editing
>> a file that's saved to disk, it's all pipes and STDIN.
>>
>> Any suggestions?
>
> AWK doesn't pass through data like cut and sed. If you want something to
> appear on the output then you have to print it explicitly. So if you
> wanted to display all lines starting '1006' then your pattern and action
> would be:
>
> /^1006/ { print }

Blimey. I really should have investigated sed and awk much, *much*
earlier in my Linux life. It's only taken me about ten minutes and I've
dumped what I'd already written, and am almost done with this 100% awk
replacement.

Yay!!!

In less than the time I spent typing up my post to the newsgroup I've
written it.

Thank you very much for the clue (well, mostly the web-page) Bruce, very
helpful indeed.


> I've tried to resist the temptation of doing it all. :-)

Much appreciated. I learn better by doing.

Justin.

--
Justin C, by the sea.
 | 
Pages: 1
Prev: Press conference
Next: 'You are not connected to the internet'