Prev: Manifest question!
Next: IViewObject:Draw
From: jjoohhnn on 5 Feb 2009 00:18 Nathan, > The ideal/optimum chunk size (which you seem to be asking for) > probably depends on your HD, what other applications are running at > the same time, memory, etc. Write your code with a tuning value that > adjusts the chunk size. Then, experiment. Chunk size should be constant for our applicaiton. So, I am sorry. I cannot go with this. We had given constant value for that. It is running fine without cpu & memory issues. >I don't know what you mean by "unique value" here. You'll have to be >clearer. Much clearer. Checksum value for each chunk like MD5 >You've not managed to get your code running for huge files. I can manage for huge files. I just want to know VC-Experts views. Just for that. I would like to know the difference between FAT & NTFS. Why Volume Shadow Copies will not support FAT, Is it true? Regards, John "Nathan Mates" <nathan(a)visi.com> wrote in message news:c7-dnbkDZcjk7RfUnZ2dnUVZ_gidnZ2d(a)posted.visi... > In article <OGhWmY0hJHA.1172(a)TK2MSFTNGP04.phx.gbl>, > jjoohhnn <jjoohhnn(a)microsoft.discussions.com> wrote: >>> Read in small (1-4MB chunks), and process the chunks in order. > >> I would like to know about this chunk size while processing huge >>data (from files). To get better(high like cpu, memory) performance on win >>32 applications, which is the chunk size? > > The ideal/optimum chunk size (which you seem to be asking for) > probably depends on your HD, what other applications are running at > the same time, memory, etc. Write your code with a tuning value that > adjusts the chunk size. Then, experiment. > >> How can we identify the each chunk with unique value? But >>performance should be very high. > > I don't know what you mean by "unique value" here. You'll have to > be clearer. Much clearer. If you want performance, the best thing to > do is probably some form of overlapped I/O. Look at CreateFile -- > http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx , > especially the FILE_FLAG_NO_BUFFERING & FILE_FLAG_SEQUENTIAL_SCAN > flags. (And, pay attention to all the comments as to the special work > you must do when using the first!) Then call ReadFile -- > http://msdn.microsoft.com/en-us/library/aa365467(VS.85).aspx with an > overlapped handle. Going double-buffered where you request reads into > a second buffer when you're processing the first might help as well. > >> what is the difference between fat & ntfs file systems? Is there >> any >>other file systems are there? > > Get it running, first. Then make it fast. You've not managed to get > your code running for huge files. Caring about FAT vs NTFS is > irrelevant at this point. You're committing the error of premature > optimization, or premature "what if" nonsense. Working code that's not > stupidly slow is far better than thoughts on paper that care about > every detail under the sun. > > Nathan Mates > -- > <*> Nathan Mates - personal webpage http://www.visi.com/~nathan/ > # Programmer at Pandemic Studios -- http://www.pandemicstudios.com/ > # NOT speaking for Pandemic Studios. "Care not what the neighbors > # think. What are the facts, and to how many decimal places?" -R.A. > Heinlein |