Prev: Plotting in batch with no display
Next: Python treats non-breaking space wrong?
From: Steve on 5 Jun 2010 03:53
I am new to Python and am wanting to replace characters in a very
large text file.....6 GB
In plain language what I wish to do is:

Remove all comma's
Replace all @ with comma's
Save as a new file.

Any of you clever people know the best way to do this......idiot guide
please.

Thanks

Steve
From: Steven D'Aprano on 5 Jun 2010 04:06
On Sat, 05 Jun 2010 00:53:23 -0700, Steve wrote:

> I am new to Python and am wanting to replace characters in a very large
> text file.....6 GB
> In plain language what I wish to do is:
>
> Remove all comma's
> Replace all @ with comma's
> Save as a new file.


input_file = open("some_huge_file.txt", "r")
output_file = open("newfilename.txt", "w")
for line in input_file:
line = line.replace(",", "")
line = line.replace("@", ",")
output_file.write(line)
output_file.close()
input_file.close()


--
Steve
From: Paul Rubin on 5 Jun 2010 10:46
Steve <vvw25w(a)googlemail.com> writes:
> Remove all comma's
> Replace all @ with comma's
> Save as a new file.

The simplest way is just copy the file one character at a time, making
replacements to commas and @'s as stated. That will be a bit slow
(especially in Python) but if you only have to do it once, just wait it
out.
From: MRAB on 5 Jun 2010 11:35
Steven D'Aprano wrote:
> On Sat, 05 Jun 2010 00:53:23 -0700, Steve wrote:
>
>> I am new to Python and am wanting to replace characters in a very large
>> text file.....6 GB
>> In plain language what I wish to do is:
>>
>> Remove all comma's
>> Replace all @ with comma's
>> Save as a new file.
>
>
> input_file = open("some_huge_file.txt", "r")
> output_file = open("newfilename.txt", "w")
> for line in input_file:
> line = line.replace(",", "")
> line = line.replace("@", ",")
> output_file.write(line)
> output_file.close()
> input_file.close()
>
I'd probably process it in larger chunks:

CHUNK_SIZE = 1024 ** 2 # 1MB at a time
input_file = open("some_huge_file.txt", "r")
output_file = open("newfilename.txt", "w")
while True:
chunk = input_file.read(CHUNK_SIZE)
if not chunk:
break
chunk = chunk.replace(",", "")
chunk = chunk.replace("@", ",")
output_file.write(chunk)
output_file.close()
input_file.close()

From: Steve on 5 Jun 2010 22:27
On 5 June, 08:53, Steve <vvw...(a)googlemail.com> wrote:
> I am new to Python and am wanting  to replace characters in a very
> large text file.....6 GB
> In plain language what I wish to do is:
>
> Remove all comma's
> Replace all @ with comma's
> Save as a new file.
>
> Any of you clever people know the best way to do this......idiot guide
> please.
>
> Thanks
>
> Steve

Many thanks for your suggestions.

sed -i 's/Hello/hello/g' file

Run twice on the CL..with the hello's changed for my needs did it in a
few minutes ,

Again thanks

Steve
 |  Next  |  Last
Pages: 1 2
Prev: Plotting in batch with no display
Next: Python treats non-breaking space wrong?