Prev: disk space usage
Next: Shell script to open browser and get response back
From: pg on 30 May 2010 18:53
Also, in the first "for" loop,

   split -l 1-a $w "$f" "$d/$f"

should have a space before before the -a

split -l 1 -a $w "$f" "$d/$f"

And it would be good to remove the temporary directory, especially
because you say your files are large:

rm -r $d


Another approach is to split the file into chunks small enough to
"paste" on your system:

#!/bin/sh

mypaste() {
d=$(mktemp -d)
touch outf
npieces=0

for f in "$@"; do
split --lines 10000 --suffix-length 3 "$f" "$d/$f"
n=$(ls "$d/$f"* | wc -l)

done

for h in $d/"$1"*; do
s=${$f##file1}
paste *"$s" >> outf
done
rm -r $d
}
From: pg on 30 May 2010 19:01
Sorry, I hit Send by mistake:


#!/bin/sh

mypaste() {
d=$(mktemp -d)
touch outf
nparts=0
biggestf=

for f in "$@"; do
split --lines 10000 --suffix-length 3 "$f" "$d/$f"
n=$(ls "$d/$f"* | wc -l)
test $n -gt $nparts && nparts=$n && biggest="$f"
done

for h in $d/"$biggestf"*; do
s=${$f##file1}
paste *"$s" >> outf
done
rm -r $d
}
First  |  Prev  | 
Pages: 1 2
Prev: disk space usage
Next: Shell script to open browser and get response back