Prev: VSS Fails on InitializeForBackup.
Next: Using memory mapped files to quickly transfer data between pro
From: Hugo gleaves on 17 Feb 2010 09:00

Hi

I'm unsure of something so want to pick the brains of anyone who might have
some further knowledge.

In an app that has a memory mapped file loaded, if it updates some mapped
pages and then does nothing else, then if I kill the app (e.g. with task
manager) the modified pages are written to disk, that is the updates are not
lost.

Further, if I run the same test but instead of killing the app, I just pull
power from the machine, the updates are lost and the files contents are
unmodified (or they might be if the OS paged the page out, but this is not
often the case).

Now my question:

If I run that same test, start the app, update some data in some mapped
pages and then (with the app running but idle) I do a system shutdown.

My understanding was that this operation would kill the processes (as I did
above with task manager) and then (eventually) finish the shutdown and power
off.

Because of this I have assumed that modfied pages would be written to disk
when I shutdown, much as they are when I just kipp a running app.

But is this true? I am seeing behaviour reported by a customer that suggests
a shutdown may actually kill processes more "brutally" and therefore might
not flush modfied pages to the mapped file.

I cannot find formal documentation on this anywhere.

If shutdown is more "dirty" in this scenario, then how can I detect shutdown
in my code? I'd ideally like to create either a handler or a dedicated thread
that gets signalled when shutdown occurs (this will be done inside our API
library DLL).

Handling it this was will mean we can handle it whether our library is used
in a windows app, a console app, a service etc and lets us handle it without
the user having to code for it.

Any info greatly appreciated.

H
From: Francois PIETTE on 17 Feb 2010 09:28
> But is this true? I am seeing behaviour reported by a customer that
> suggests
> a shutdown may actually kill processes more "brutally" and therefore might
> not flush modfied pages to the mapped file.
>
> I cannot find formal documentation on this anywhere.

Shutdown is an orderly process. It is clean. No risk to loose data. I would
even say there is much less risk to loose data when properly shutting down
the computer (not by removing the power cord or clicking the hardware reset,
but using the shutdown procedure) than killing a process with task manager.

>
> If shutdown is more "dirty" in this scenario,

It isn't.

> then how can I detect shutdown
> in my code?

Handle the message WM_ENDSESSION:
http://msdn.microsoft.com/en-us/library/aa376884(VS.85).aspx

--
francois.piette(a)overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be



From: Hugo gleaves on 17 Feb 2010 11:26


"Francois PIETTE" wrote:

> > But is this true? I am seeing behaviour reported by a customer that
> > suggests
> > a shutdown may actually kill processes more "brutally" and therefore might
> > not flush modfied pages to the mapped file.
> >
> > I cannot find formal documentation on this anywhere.
>
> Shutdown is an orderly process. It is clean. No risk to loose data. I would
> even say there is much less risk to loose data when properly shutting down
> the computer (not by removing the power cord or clicking the hardware reset,
> but using the shutdown procedure) than killing a process with task manager.
>
> >
> > If shutdown is more "dirty" in this scenario,
>
> It isn't.
>
> > then how can I detect shutdown
> > in my code?
>
> Handle the message WM_ENDSESSION:
> http://msdn.microsoft.com/en-us/library/aa376884(VS.85).aspx
>
> --
> francois.piette(a)overbyte.be
> The author of the freeware multi-tier middleware MidWare
> The author of the freeware Internet Component Suite (ICS)
> http://www.overbyte.be
>
>
>
> .
>

Thanks Francois

That is more or less what I understood too.

Now here is next question, which might explain what is happening.

If the app maps a 2 GIG file, then populates say 1 GIG with data, this
equates to about 260,000 modified memory pages. Assume the box has lots of
memory (eg, 8 or 16 G) and that memory is under very little pressure.

Now when the reboot begins, it will go through the orderly termination and
flushing you described, but it will take X seconds for all of these modified
pages to be written to the disk (the media itself).

So there could be a weakness, in that the shutdown will wait only so many
seconds and could perhaps power off the box, while these pages are being
flkushed, resulting in not all modfied pages being written.

This sounds plausible to me, fixing it is a separate issue, but could this
happen?

H

From: Hector Santos on 17 Feb 2010 12:45
Our server make extensive use of memory mapped files (including 1-2
gigs sizes) and over time, we just decided that it was best to flush
record writes to resolve the occasional support issues where customers
where not gracefully shutting down or were doing machine shutdowns and
the server moved into a exit state or there was a brown out with UPS
failed, etc. In the past, speed was a consideration so always
flushing was not always desired but not as much today allowing us to
afford flushing record writes. The speed of computer and improvements
in Windows itself, make it more feasible to do. Bottom line result for
us: No more support calls.

If you can't afford flushing records with each write, then you might
consider a timer that will do so every X seconds or so.

Hugo gleaves(a)hotmail.com> <hugh wrote:

>
> Thanks Francois
>
> That is more or less what I understood too.
>
> Now here is next question, which might explain what is happening.
>
> If the app maps a 2 GIG file, then populates say 1 GIG with data, this
> equates to about 260,000 modified memory pages. Assume the box has lots of
> memory (eg, 8 or 16 G) and that memory is under very little pressure.
>
> Now when the reboot begins, it will go through the orderly termination and
> flushing you described, but it will take X seconds for all of these modified
> pages to be written to the disk (the media itself).
>
> So there could be a weakness, in that the shutdown will wait only so many
> seconds and could perhaps power off the box, while these pages are being
> flkushed, resulting in not all modfied pages being written.
>
> This sounds plausible to me, fixing it is a separate issue, but could this
> happen?
>
> H
>



--
HLS
From: Francois PIETTE on 17 Feb 2010 15:16
>> Shutdown is an orderly process. It is clean. No risk to loose data. I
>> would
>> even say there is much less risk to loose data when properly shutting
>> down
>> the computer (not by removing the power cord or clicking the hardware
>> reset,
>> but using the shutdown procedure) than killing a process with task
>> manager.

> That is more or less what I understood too.
>
> Now here is next question, which might explain what is happening.
>
> If the app maps a 2 GIG file, then populates say 1 GIG with data, this
> equates to about 260,000 modified memory pages. Assume the box has lots of
> memory (eg, 8 or 16 G) and that memory is under very little pressure.
>
> Now when the reboot begins, it will go through the orderly termination and
> flushing you described, but it will take X seconds for all of these
> modified
> pages to be written to the disk (the media itself).
>
> So there could be a weakness, in that the shutdown will wait only so many
> seconds and could perhaps power off the box, while these pages are being
> flkushed, resulting in not all modfied pages being written.

Not sure I understand what you mean.
No matter how long it will take to write the data to disk, it will be
written in the shutdown process.
But if the user is impatient and turn the power off before the shutdown is
finished, then yes things will go wrong.

--
francois.piette(a)overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be


 |  Next  |  Last
Pages: 1 2
Prev: VSS Fails on InitializeForBackup.
Next: Using memory mapped files to quickly transfer data between pro