Prev: disk space usage
Next: Shell script to open browser and get response back
From: Yuriy Skripchenko on 26 May 2010 13:06
Hello,

I would like to add an arbitrary number of files together line by
line. So if I have 3 csv files, each with 2 columns I would like to
create 1 file with 6 columns. I have tried paste but paste has a limit
on the length of the combined string. I cannot use join as I do not
have any similar fields. Is there another solution out there?
From: Janis Papanagnou on 26 May 2010 13:25
Yuriy Skripchenko wrote:
> Hello,
>
> I would like to add an arbitrary number of files together line by
> line. So if I have 3 csv files, each with 2 columns I would like to
> create 1 file with 6 columns. I have tried paste but paste has a limit
> on the length of the combined string. I cannot use join as I do not
> have any similar fields. Is there another solution out there?

Your paste has a limit that you exceed? How large is that limit?

What OS are you using? How long is the resulting line WRT your data?

Can you install paste from the GNU project to circumvent any limit?

What other limits and restrictions do you have and not yet told us?

Janis
From: pg on 30 May 2010 08:02
On May 26, 1:06 pm, Yuriy Skripchenko <yskripche...(a)gmail.com> wrote:
> Hello,
>
> I would like to add an arbitrary number of files together line by
> line. So if I have 3 csv files, each with 2 columns I would like to
> create 1 file with 6 columns. I have tried paste but paste has a limit
> on the length of the combined string. I cannot use join as I do not
> have any similar fields. Is there another solution out there?

I'm sure there is a more elegant solution but something like this wold
work:

mypaste() {
d=$(mktemp -d)

for f in "$@"; do
nl=$(wc -l < "$f")
w=$(awk "BEGIN { print int(log($nl) / log(26)) + 1 }")
split -l 1-a $w "$f" "$d/$f"
done

for h in $d/"$1"*; do
tr -d '\n' < "$h" >> outf # Assuming the output file is outf
s=${$f##file1}
for g in $(ls $d/*$s | tail -n +2); do
echo -n , >> outf # Assuming , is the delimiter
tr -d '\n' < "$g" >> outf
done
echo >> outf
done
}

mysplit bigfile1 bigfile2 bigfile3
From: pg on 30 May 2010 18:36
Of course I meant
mypaste bigfile1 bigfile2 bigfile3
From: Lew Pitcher on 30 May 2010 18:47

Warning:

Lew Pitcher, who posts to this newsgroup, is a domain thief.

Read the full story at http://www.lewpitcher.ca

 |  Next  |  Last
Pages: 1 2
Prev: disk space usage
Next: Shell script to open browser and get response back