Prev: WMI compiing problem
Next: linker problem: __iob
From: Alf P. Steinbach on 24 Feb 2006 17:25
* rivenburgh(a)gmail.com:
> Unfortunately no, the data needs to be read from a file and processed a
> bit. All of this data management is actually in an external library
> that I don't want to modify too much. Swapping out "new" for HeapAlloc
> sounds pretty manageable.
>
> Thanks for the idea, though!

Not sure: are your requirements for processing are to allocate and
deallocate a lot of small chunks?

Otherwise, when you call MapViewOfFile you get a memory pointer that you
can pass to your external library.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
From: rivenburgh on 24 Feb 2006 17:31
No, the processing involved doesn't allocate and deallocate small
chunks. It pretty much allocates the one big chunk and reads the data
from the file, processing the input and filling the memory. The
library just wants a file name and handles all of the memory management
itself, so it doesn't really lend itself to those kinds of
modifications....

Thanks,
Reid

From: Scott McPhillips [MVP] on 24 Feb 2006 17:45
rivenburgh(a)gmail.com wrote:
> The ideal thing for me would be if the OS memory manager gave me only
> what I ask for (e.g. 250 MB) and used no more, and did as good a job as
> possible of minimizing memory fragmentation, improving my chances for
> finding a contiguous chunk of memory of that size....
>
> Reid
>

The heap manager can't do anything useful about memory fragmentation in
such a case.

> Does anyone have any suggestions for maximum
> efficiency when allocating huge chunks of memory? The exact amount
> I'll need isn't something I know ahead of time, unfortunately, so I
> can't just do something like grab 1.5 GB when the program starts.
> (Well, I guess I could, but that might be unnecessarily greedy in some
> cases.)

No, it's not greedy. Grabbing everything you can get ahead of time is
stealing nothing except your own app's memory space. It does not impact
other apps until and unless you actually access it. If you do your own
suballocation of the large data sets from a huge buffer like this you
have the power to do something about fragmentation, because you can move
arrays if necessary.

--
Scott McPhillips [VC++ MVP]

From: Reid Rivenburgh on 24 Feb 2006 18:16
"Scott McPhillips [MVP]" <org-dot-mvps-at-scottmcp> writes:

> rivenburgh(a)gmail.com wrote:
>> The ideal thing for me would be if the OS memory manager gave me only
>> what I ask for (e.g. 250 MB) and used no more, and did as good a job as
>> possible of minimizing memory fragmentation, improving my chances for
>> finding a contiguous chunk of memory of that size....
>
> The heap manager can't do anything useful about memory fragmentation
> in such a case.

Well, I just meant in general that it would be nice if fragmentation
could be minimized as much as possible. Put the small chunks
together, for example, freeing up more big spaces.

> > Does anyone have any suggestions for maximum
> > efficiency when allocating huge chunks of memory? The exact amount
> > I'll need isn't something I know ahead of time, unfortunately, so I
> > can't just do something like grab 1.5 GB when the program starts.
> > (Well, I guess I could, but that might be unnecessarily greedy in some
> > cases.)
>
> No, it's not greedy. Grabbing everything you can get ahead of time
> is stealing nothing except your own app's memory space. It does not
> impact other apps until and unless you actually access it. If you
> do your own suballocation of the large data sets from a huge buffer
> like this you have the power to do something about fragmentation,
> because you can move arrays if necessary.

Hm, interesting, I wasn't looking at it like that. It looks to me in
Memory Validator, however, as if HeapCreate tries to grab the chunk of
memory requested and call it "commit", even before trying to do
anything with it. I figured the heap space would be unusable by other
apps. Maybe I'm misunderstanding you; are you talking about using C++
"new" to grab 1.5 GB and managing that space in the app, NOT using
HeapCreate? That I can believe, though trying to modify the library
I'm using to work that way would be a big hassle.

That brings me back to my original question: Even when I used
HeapCreate and HeapAlloc in a dummy program, it looked as if the
HeapAlloc space wasn't being taken from the heap I created. Is that
possible? If I grab 1.5 GB of heap for use by my datasets, it'd be
kind of pointless if the actual memory allocated comes from somewhere
else....

Thanks,
Reid
From: Alf P. Steinbach on 24 Feb 2006 18:23
* rivenburgh(a)gmail.com:
> No, the processing involved doesn't allocate and deallocate small
> chunks. It pretty much allocates the one big chunk and reads the data
> from the file, processing the input and filling the memory. The
> library just wants a file name and handles all of the memory management
> itself, so it doesn't really lend itself to those kinds of
> modifications....

I don't understand what the problem is. If you can replace 'new' with
HeapAlloc, and there is essentially one big chunk of memory that's the
problem, why can't you replace 'new' with MapViewOfFile? They both
yield memory pointers, and the use of those pointers are the same, no
difference.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5 6 7
Prev: WMI compiing problem
Next: linker problem: __iob