Script to count duplicate lines in a text file
in [VbScript]
|
Prev: Calling a C++ dll
Next: users import
From: Gadgetman on 30 Sep 2010 02:07 We have a requirement to count the number of duplicate entries in a text file and then list the line and the number of duplicates. For example, we have a file call test.txt that has the following entries: A1 A1 B1 B2 B2 B2 B2 This would return the following: A1,2 B1,1 B2,4 Any assistance would be greatly appreciated
From: ekkehard.horner on 30 Sep 2010 06:51 Gadgetman schrieb: > We have a requirement to count the number of duplicate entries in a text file > and then list the line and the number of duplicates. For example, we have a > file call test.txt that has the following entries: > A1 > A1 > B1 > B2 > B2 > B2 > B2 > This would return the following: > A1,2 > B1,1 > B2,4 > > Any assistance would be greatly appreciated Create a Dictionary to hold the lines (key) und their frequencies (values) Open the file ForReading and loop until AtEndOfStream put each line into the Dictionary dicLines( sLine ) = dicLines( sLine ) + 1 Display the Dictionary For Each sLine In dicLines.Keys WScript.Echo sLine, dicLines( sKey )
From: Mayayana on 30 Sep 2010 21:57 The scripting help files are here: http://www.microsoft.com/downloads/details.aspx?FamilyId=01592C48-207D-4BE1-8A76-1C4099D7BBB9 FileSystemObject deals with file I/O. Textstream is the object for reading/writing files. Lots of people are happy to help but it sounds like you're asking for someone to write your script. People don't just hang around here waiting for a chance to do other peoples' work for free. If you come up wit a script then you can ask here for help. | We have a requirement to count the number of duplicate entries in a text file | and then list the line and the number of duplicates. For example, we have a | file call test.txt that has the following entries: | A1 | A1 | B1 | B2 | B2 | B2 | B2 | This would return the following: | A1,2 | B1,1 | B2,4 | | Any assistance would be greatly appreciated
From: contrex on 3 Oct 2010 04:35 On 1 Oct, 02:57, "Mayayana" <mayay...(a)invalid.nospam> wrote: > People don't just hang around here waiting for a chance to do > other peoples' work for free. Ignore the top posting unhelpful troll. I had 20 minutes to spare so I knocked out this little example '*****Script start***** 'Create a Dictionary to hold the lines (key) und their frequencies (values) Set diclines = CreateObject("Scripting.Dictionary") 'Invoke the File System Object Set FSO = CreateObject("Scripting.FileSystemObject") 'Filename passed as argument Strfilename = wscript.arguments(0) 'Show Filename wscript.echo Strfilename 'Open the file for reading Const ForReading = 1 Set objFile = FSO.OpenTextFile(strFileName, ForReading) 'Loop until end of stream Do Until objFile.AtEndOfStream 'Read each line Fline = objFile.ReadLine 'If line is not already in dictionary, add Item 'and set key value to 1 'If line is already in dictionary, increment key value If Not diclines.Exists(Fline) Then diclines.Add Fline , 1 Else dicLines( FLine ) = dicLines( FLine ) + 1 End If Loop objFile.Close 'Show keys and items For Each element In diclines wscript.echo element & "," & diclines(element) Next '*****Script end***** *****test.txt***** Beijing Beijing Beijing Beijing Beijing Beijing Beijing Beijing Beijing Beijing Berlin London London London London London London London London London London London London London London New York New York New York New York New York New York Tokyo Tokyo Tokyo Tokyo Tokyo Tokyo ****Output test.txt Beijing,10 Berlin,1 London,14 New York,6 Tokyo,6
From: mbyerley on 3 Oct 2010 10:11
"contrex" <mike.j.harvey(a)gmail.com> wrote in message news:de11c212-caa7-4432-b43c-339a06814264(a)l20g2000yqm.googlegroups.com... > On 1 Oct, 02:57, "Mayayana" <mayay...(a)invalid.nospam> wrote: > >> People don't just hang around here waiting for a chance to do >> other peoples' work for free. > > Ignore the top posting unhelpful troll. I had 20 minutes to spare so I > knocked out this little example I guessed you missed the part of the OP saying "We have a requirement" which implies something beyond an individual needing assistance and more likely the company or orgainzation the OP is a part. That the OP asked for assistance is inaccurate, lacking the foundation the OP had already made an unsuccessful attempt on his own, you (and the rest of the planet) could conclude the OP wanted someone to do the work for him ( or the organization that employs him, how is that helpful?). So Mayayana's response was maybe curt, but exactly right and your comments to the contrary would be dismissed out of hand by most vampire hunters/haters. http://slash7.com/2006/12/22/vampires/ > '*****Script start***** > 'Create a Dictionary to hold the lines (key) und their frequencies > (values) > Set diclines = CreateObject("Scripting.Dictionary") > > 'Invoke the File System Object > Set FSO = CreateObject("Scripting.FileSystemObject") > > 'Filename passed as argument > Strfilename = wscript.arguments(0) > > 'Show Filename > wscript.echo Strfilename > > 'Open the file for reading > Const ForReading = 1 > Set objFile = FSO.OpenTextFile(strFileName, ForReading) > > 'Loop until end of stream > Do Until objFile.AtEndOfStream > 'Read each line > Fline = objFile.ReadLine > 'If line is not already in dictionary, add Item > 'and set key value to 1 > 'If line is already in dictionary, increment key value > If Not diclines.Exists(Fline) Then > diclines.Add Fline , 1 > Else > dicLines( FLine ) = dicLines( FLine ) + 1 > End If > Loop > objFile.Close > > 'Show keys and items > For Each element In diclines > wscript.echo element & "," & diclines(element) > Next > '*****Script end***** > > *****test.txt***** > Beijing > Beijing > Beijing > Beijing > Beijing > Beijing > Beijing > Beijing > Beijing > Beijing > Berlin > London > London > London > London > London > London > London > London > London > London > London > London > London > London > New York > New York > New York > New York > New York > New York > Tokyo > Tokyo > Tokyo > Tokyo > Tokyo > Tokyo > > ****Output > test.txt > Beijing,10 > Berlin,1 > London,14 > New York,6 > Tokyo,6 > > > > > |