From: Dan Polansky on
Following up on the subject title "how to enlarger Java heap size?"
rather than on the recent discussion:

Is there a way to tell Java on the command line to take as much heap
size as the physical memory or the virtual memory of the computer
allows?

I have a launcher written in C for Windows that runs a Java
application. I can tell the launcher to pass "-Xmx1024M" to "javaw ",
but then the launcher won't work on computers that have less memory,
which is why I use "-Xmx256M", to be on the safe side. What I would
really like to tell JRE is: "Look, take as much memory for the heap as
the physical memory or the virtual memory of the computer allows". I
do not know in advance what the physical memory of user's computer
will be.

--Dan
From: Peter Duniho on
On Sat, 19 Sep 2009 09:26:37 -0700, Dan Polansky <dan.polansky(a)gmail.com>
wrote:

> Following up on the subject title "how to enlarger Java heap size?"
> rather than on the recent discussion:
>
> Is there a way to tell Java on the command line to take as much heap
> size as the physical memory or the virtual memory of the computer
> allows?

No, not solely with the command line arguments.

You could write a helper application that detects the installed physical
memory and then passes that along as it starts the Java process. However,
that's not actually a very useful thing to do; physical memory has nothing
at all to do with how much memory is actually available to the process.
At best, it's a potential performance optimization, and in reality because
your Java process has no way to monopolize the physical memory, it's an
optimization that may or may not (and most likely not) have any effect.

As far as "virtual memory" goes, the first issue is what you mean by
"virtual memory". A process has two limits: virtual address space, and
virtual memory. The former is limited in part architecturally, but also
by overhead within the process. The latter is limited both by virtual
address space and by available disk space (whichever is smaller).

On 32-bit architectures, normally (these days) disk space is much larger
than the virtual address space, so it's really just virtual address space
that you are concerned with. But while the actual virtual address space
is known (e.g. 2GB on 32-bit Windows), as with the physical memory,
there's no reliable way to predict how much of that is usable by your
process. Due to various other overhead in the process, on Windows you can
realistically get up to about 1.6GB or so before a memory allocation
fails. Depending on the actual allocation pattern, this can be
significantly lower.

So, as with physical memory, while it is possible to know the theoretical
upper bound (i.e. the architecturally defined limit), that's not a useful
number. There are other variables that are unknown.

Note that all of the above is even worse if you're dealing with a 64-bit
OS. Physical memory is even less relevant to your process, and the
theoretical upper bound of how much you can allocate becomes huge, and
impractical to try to reach.

Which is all a long way of saying, no...not only can you not have Java
determine this automatically, it is not even practical for you to use
external information to configure a Java process in that way. The
maximums you can measure aren't actually useful in terms of controlling
performance of your process.

> I have a launcher written in C for Windows that runs a Java
> application. I can tell the launcher to pass "-Xmx1024M" to "javaw ",
> but then the launcher won't work on computers that have less memory,

"Less" what kind of "memory"? Physical memory? Irrelevant; you can start
with 1GB of Java heap, even if the computer only has 128MB of RAM.
Virtual memory? It's true that if you have a user with less than 1GB of
disk space left, they may not be able to start a process that starts out
by trying to allocate a 1GB block of memory. But how often do you really
run into that these days? Why isn't it more useful to just tell the user
"hey, you need to have at least 1GB of disk space for my process to run".

> which is why I use "-Xmx256M", to be on the safe side.

If your program can run in 256MB of heap, why would allocating more help?

> What I would
> really like to tell JRE is: "Look, take as much memory for the heap as
> the physical memory or the virtual memory of the computer allows". I
> do not know in advance what the physical memory of user's computer
> will be.

Personally, I think it's a bit archaic that you have to specify the upper
bound of allocation up-front for Java programs. Java _ought_ to be able
to adjust the heap size as necessary, within the limits available on the
OS, according to usage. Your process ought to be able to start out with
(for example) a 64MB heap, but have that heap grow as much as is supported
by the OS should the Java program require additional memory.

But, AFAIK that's just not possible right now. So, I think you're pretty
much stuck figuring out what the largest heap your program will need, and
just always asking for that. In theory, you could provide the user with
additional start-up options, but it seems to me that in reality, you're
not going to run into many computers where specifying the max you'll use
won't actually work. Just specify 1GB all the time, and leave it at that.

Pete