VirtualAlloc,GlobalAlloc,HeapAlloc,..what to use in my case,...
in [Visual C]
|
Prev: Warnings about deprecated / insecure function usage
Next: App built by VS2008 causes "side-by-side configuration"-error in Vista
From: Kerem G�mr�kc� on 8 Mar 2008 11:20 Hi Alex, >The real question here is why do you need `VirtualAlloc' at all? Why not and i also want the MEM_WRITE_WATCH Option for some special purpose. I must watch some special areas of the memory. There is a bit more in going on my application, nit just read and write some files and streams. >You always call it with `MEM_COMMIT' and `PAGE_READWRITE' flags making it >eventually indistinguishable form regular `operator new' or `malloc'. the C++ "new" Operator is no option for me, because it is a plain C Application and i dont want to mix it up with C++ heap memory stuff. >Actually you're making it worse since you miss all optimizations that CRT >may enable (like Low Fragmentation Heap, for example). I can still use HeapCompact(...) if i need to... >I suggest you at least to try your code with regular `new' and `delete'. No C++,... Thanks,... Regards K. -- ----------------------- Beste Gr�sse / Best regards / Votre bien devoue Kerem G�mr�kc� Microsoft Live Space: http://kerem-g.spaces.live.com/ Latest Open-Source Projects: http://entwicklung.junetz.de ----------------------- "This reply is provided as is, without warranty express or implied." "Alex Blekhman" <tkfx.REMOVE(a)yahoo.com> schrieb im Newsbeitrag news:u%23B7IxPgIHA.1132(a)TK2MSFTNGP06.phx.gbl... > "Kerem G�mr�kc�" wrote: >> I have several vrapper functions arround the VirtualAlloc and VirtualFree >> and i was passing accidentally a MEM_DECOMMIT to a VirtualFree, not a >> MEM_RELEASE and also not a zero as second parameter. This was not clear >> because of my own flags and complex wrappers,....dow,... > > The real question here is why do you need `VirtualAlloc' at all? You > always call it with `MEM_COMMIT' and `PAGE_READWRITE' flags making it > eventually indistinguishable form regular `operator new' or `malloc'. > Actually you're making it worse since you miss all optimizations that CRT > may enable (like Low Fragmentation Heap, for example). > > I suggest you at least to try your code with regular `new' and `delete'. > Also, if you want to avoid buffer allocations altogether, then you can use > memory mapping to read the files. > > Alex >
From: Nathan Mates on 8 Mar 2008 18:05 In article <eGHCLiTgIHA.6092(a)TK2MSFTNGP06.phx.gbl>, Kerem G�mr�kc� <kareem114(a)hotmail.com> wrote: >>You always call it with `MEM_COMMIT' and `PAGE_READWRITE' flags making it >>eventually indistinguishable form regular `operator new' or `malloc'. >the C++ "new" Operator is no option for me, because it is a plain >C Application and i dont want to mix it up with C++ heap memory >stuff. new and malloc both use the same heap, provided by the C Runtime library, which is used by C and/or C++ code. They can coexist just fine, no matter which language requested the memory, and nothing will be 'mixed up.' Plus, your code will be far more portable to other (i.e. non-Win32) systems. Nathan Mates -- <*> Nathan Mates - personal webpage http://www.visi.com/~nathan/ # Programmer at Pandemic Studios -- http://www.pandemicstudios.com/ # NOT speaking for Pandemic Studios. "Care not what the neighbors # think. What are the facts, and to how many decimal places?" -R.A. Heinlein
From: Joseph M. Newcomer on 8 Mar 2008 20:44 Why use VirtualAlloc at all? Doesn't make much sense to me to see this low-level call in use; it is used in rare situations (I've used it twice in over 15 years of Win32 programming). There is a horrid fascination with people reporting the amount of physical memory on their machines; this is almost totally irrelevant to the question of running out of memory, because you are not running out of physical memory, you are running out of virtual memory. Use 'malloc' if you are programming in C or 'new' if you are programming in C++. I would not consider VirtualAlloc, GlobalAlloc or HeapAlloc for any of these situations. For that matter, one of the fastest ways to do this would be to scan the directory, find the largest file, allocate one buffer of that size, and use it for all the other files. Each time you open a file, you double-check that the memory you allocated is still large enough (the file might have grown, but once you open it exclusive, it isn't going to get any larger). If it isn't, you reallocate at that point, otherwise you retain your allocation. This avoids issues of memory fragmentation (if you're not sure what this means, read my essay on storage allocation on my MVP Tips site) joe On Sat, 8 Mar 2008 01:54:10 +0100, "Kerem G�mr�kc�" <kareem114(a)hotmail.com> wrote: >Hi, > >i have a application that reads a unknown bunch of >files to a block of memory, handles them and finally >frees the memory. For now, i work with VirtualAlloc, >but i get after processing ~250 Files a GetLastError=8 >for any memory request, even i know that memory has >been released with VirtualFree=TRUE. Why is that >the case? The System has 1GB of real physical memory. >Sure i know the limitations of user memory space and >process space. VirtualAlloc will always be called with >MEM_COMMIT and PAGE_READWRITE in my code. >I have to read the complete file, not a piece of it, so >some files are 205 byte others about 20MB. So what >and how can i asure, that my application frees the >allocated memory after ist usage and what are the >differences on processing impact if i use these functions, >better to say: what are the differences between VirtualAlloc, >GlobalAlloc and HeapAlloc? The Application is a single >threaded app and expects a folder path as input, then it >processes the files: > >FOR EACH FILE IN FOLDER > IsFileReadable > ReadFileSize > VirtualAlloc > ReadFileContent > DoAnalyzeFileContent > WriteToOwnAppFile > FlushBuffers > VirtualFree >NEXT FILE >ExitApplication > >How can i prevent to run into a GetLastError()=8 >Subsequent Memory Request fail after a GetLastError=8 > > >Regards > >K. Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 8 Mar 2008 20:57 See my previous post. What made you think that VirtualAlloc was a reasonable strategy in the first place? Why use HeapAlloc when 'new' or 'malloc' work just fine? From the essay ====Although the GlobalAlloc, LocalAlloc, and HeapAlloc functions ultimately allocate ====memory from the same heap, each provides a slightly different set of functionality. ====For example, ====HeapAlloc can be instructed to raise an exception if memory could not be ====allocated, a capability not available with LocalAlloc. **** Nobody cares about raising exceptions, because these are SEH exceptions which don't play well with MFC, so this feature is irrelevant. HeapAlloc is used if you have to allocate storage in DllMain, and almost nowhere else in user-level code (the low-level allocators use it) **** ====LocalAlloc supports allocation ====of handles which permit the underlying memory to be moved by a reallocation without ====changing the handle value, a capability not available with HeapAlloc. **** Since there is no reallocation mechanism worth using, the use of LocalAlloc is essentially irrelevant ***** ====The VirtualAlloc function allows you to specify additional options for memory ====allocation. However, its allocations use a page granularity, so using VirtualAlloc can ====result in higher memory usage. **** Largely irrelevant; the concept of page granularity does not even enter the discussion at the level you are working, because losing a fraction of a page on a 20MB file is such a tiny percentage (average: 0.01%, that's right, %, or 0.0001) that there is no reason to ever consider it as an issue. For a 205byte file, yes, if you were going to keep that storage for any duration, but you don't, you free it immediately ***** ====The malloc function has the disadvantage of being run-time dependent. ***** But if you're writing in MFC, you don't care in the slightest about this. It uses the same runtime as the rest of your program. It would be an unaesthetic choice, but not an incorrect one. ***** ====The new operator has the disadvantage of being compiler dependent and ====language dependent. ***** And what is the point? You're writing MFC code in C++, so it does't matter if it is language dependent! ***** ====The CoTaskMemAlloc function has the advantage of working well in either C, C++, or ====Visual Basic. It is also the only way to share memory in a COM-based application, ====since MIDL uses CoTaskMemAlloc and CoTaskMemFree to marshal memory. ***** But you're not using COM, so this is inoperative It sounds to me like you are badly fragmenting storage. I would avoid this by doing as I outlined earlier: one allocation of max size with a fallback strategy of a realloc if necessary, but only if necessary. joe **** On Sat, 8 Mar 2008 02:13:22 +0100, "Kerem G�mr�kc�" <kareem114(a)hotmail.com> wrote: >Hi, > >i had a look at its resource uttilization/usage with process >explorer and it grows exponentially expecially its "Virtual Size" >for Virtual Memory, sure for VirtualAlloc and its CPU Usage. >At a certail level it slows down the complete system and os starts >to work with the pagefile until i kill or stop the process,... > >Something is wrong here, I think i should not use VirtualAlloc >or find a way to 100% give back the allocated memory and >then reallocate new memory. How can i force my application >to use real physical memory instead virtual? Dow, this belongs >to the OS and its internals, i know,... > >I found this: >[Comparing Memory Allocation Methods] >http://msdn2.microsoft.com/en-us/library/aa366533.aspx > >which makey my decision go HeapAlloc, because of the page granularity,... > > >Regards > >K. Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Kerem G�mr�kc� on 8 Mar 2008 21:47
Hi Joseph, >Why use VirtualAlloc at all? Doesn't make much sense to me to see this >low-level call in >use; it is used in rare situations (I've used it twice in over 15 years of >Win32 >programming). I think i posted the reason, because i need to keep track of several regions of the memory to get signaled on change/read/write/access... >There is a horrid fascination with people reporting the amount of physical >memory on their >machines; this is almost totally irrelevant to the question of running out >of memory, >because you are not running out of physical memory, you are running out of >virtual memory. I think i said that i know what realy physical memory and the virtual adress space means and thas why i said my application sizes tremendously its "virtual" memory. I can have a application that requests 500MB Ram on a system with 256 MB real physical ram, it would work, but my system paging file would also "work",... >Use 'malloc' if you are programming in C or 'new' if you are programming in >C++. >I would not consider VirtualAlloc, GlobalAlloc or HeapAlloc for any of >these situations. Then let me ask you: Why are these functions available to the users, if i should not use them? VirtualAlloc is interessting to me, because of its "notification" features,... >For that matter, one of the fastest ways to do this would be to scan the >directory, find >the largest file, allocate one buffer of that size, and use it for all the >other file >Each time you open a file, you double-check that the memory you allocated >is still large >enough (the file might have grown, but once you open it exclusive, it isn't >going to get >any larger). If it isn't, you reallocate at that point, otherwise you >retain your >allocation. This avoids issues of memory fragmentation (if you're not sure >what this >means, read my essay on storage allocation on my MVP Tips site) This is a really good idea, at least i would not make my application alloc/free free like hell,... Regards K. -- ----------------------- Beste Gr�sse / Best regards / Votre bien devoue Kerem G�mr�kc� Microsoft Live Space: http://kerem-g.spaces.live.com/ Latest Open-Source Projects: http://entwicklung.junetz.de ----------------------- "This reply is provided as is, without warranty express or implied." |