Sort Big File Help
in [Python]
|
From: Jonathan Gardner on 3 Mar 2010 17:16 On Wed, Mar 3, 2010 at 8:19 AM, John Filben <johnfilben(a)yahoo.com> wrote: > I am new to Python but have used many other (mostly dead) languages in the > past. I want to be able to process *.txt and *.csv files. I can now read > that and then change them as needed mostly just take a column and do some > if-then to create a new variable. My problem is sorting these files: > > 1.) How do I sort file1.txt by position and write out file1_sorted.txt; > for example, if all the records are 100 bytes long and there is a three > digit id in the position 0-2; here would be some sample data: > > a. 001JohnFilben > > b. 002Joe Smith .. > > 2.) How do I sort file1.csv by column name; for example, if all the > records have three column headings, id, first_name, last_name; here > would be some sample data: > > a. Id, first_name,last_name > > b. 001,John,Filben > > c. 002,Joe, Smith > > 3.) What about if I have millions of records and I am processing on a > laptop with a large external drive basically, are there space > considerations? What are the work arounds. > > Any help would be appreciated. Thank you. > You may also want to look at the GNU tools "sort" and "cut". If your job is to process files, I'd recommend tools designed to process files for the task. -- Jonathan Gardner jgardner(a)jonathangardner.net |