Prev: Create a Dialog
Next: partition&format question?
From: ManicQin on 24 Jul 2008 07:52
Hello everybody,
I'm trying to build a very fast logger module (C++).
I found out that not closing and opening the handle of the file
each time I write to the file can achieve fast performances
(much better than the other method)
but it makes other problems, I rather not elaborate for now.

A new idea that I thought of, is to write all the data into a
temporary file
(which will not be closed and reopened every writefile)
and once in a while to flush the temp file into the Log file
(Asynchronously).

Do you think that this method will result good performances?

if you have any thoughts about this method please tell me.
My logger should handle large buffers, frequent calls , he MUST be
reliable
(the only way to analyze the system after a crush)
From: Dean Earley on 24 Jul 2008 11:42
ManicQin wrote:
> Hello everybody,
> I'm trying to build a very fast logger module (C++).
> I found out that not closing and opening the handle of the file
> each time I write to the file can achieve fast performances
> (much better than the other method)
> but it makes other problems, I rather not elaborate for now.
>
> A new idea that I thought of, is to write all the data into a
> temporary file
> (which will not be closed and reopened every writefile)
> and once in a while to flush the temp file into the Log file
> (Asynchronously).
>
> Do you think that this method will result good performances?
>
> if you have any thoughts about this method please tell me.
> My logger should handle large buffers, frequent calls , he MUST be
> reliable
> (the only way to analyze the system after a crush)

I don't see any problem with keeping the main file open that will be
resolved by suign a temp file, and in fact will cause MORE problems (as
data will be split over multiple files.

Maybe if you elaborated on why keeping the main file open is not
suitable, we could help further.

This method is exactly what I use, and the only problem I had was data
not being flushed to disk immediately which was easy to fix.

--
Dean Earley (dean.earley(a)icode.co.uk)
i-Catcher Development Team

iCode Systems
From: ManicQin on 24 Jul 2008 13:57
On 24 יולי, 18:42, Dean Earley <dean.ear...(a)icode.co.uk> wrote:
>
> Maybe if you elaborated on why keeping the main file open is not
> suitable, we could help further.
>

I create the main log file with a FILE_SHARE_DELETE share mode
I need to give my QA\client the option of deleting the file whenever
they want

When you delete a file that still have handle to him,
You can still see a "ghost" file in the Explorer,
the file will be with no share attributes & with the size 0.
until you close the handle to him.
It confuses my clients , My Q.A. and scripts that are running on my
computer
and try to access the logs \ zip them \ delete them

functions like FindFirstChangeNotification dont work too
(Before you say anything try it...).

the problem consists until you release the handle to the file
(and again many people argue with me that I'm wrong an dthe this
behaviour could never be and that maybe I dont know what
I'm talking about but I prove them wrong every time...
It's a windows behaviour and It's perfectly normal)

thanks
From: Sten Westerback (MVP SDK 2005-6 :) on 31 Jul 2008 04:13

"ManicQin" <ManicQin(a)gmail.com> wrote in message
news:c8446173-253b-4230-afe8-6764729d5e3e(a)b30g2000prf.googlegroups.com...
On 24 ????, 18:42, Dean Earley <dean.ear...(a)icode.co.uk> wrote:
>>
>> Maybe if you elaborated on why keeping the main file open is not
>> suitable, we could help further.
>>
>
>I create the main log file with a FILE_SHARE_DELETE share mode
>I need to give my QA\client the option of deleting the file whenever
>they want
>
>When you delete a file that still have handle to him,
>You can still see a "ghost" file in the Explorer,
>the file will be with no share attributes & with the size 0.
>until you close the handle to him.
>It confuses my clients , My Q.A. and scripts that are running on my
>computer
>and try to access the logs \ zip them \ delete them
>
>functions like FindFirstChangeNotification dont work too
>
>the problem consists until you release the handle to the file
>(and again many people argue with me that I'm wrong an dthe this
>behaviour could never be and that maybe I dont know what
>I'm talking about but I prove them wrong every time...
>It's a windows behaviour and It's perfectly normal)

File ghost? I haven't actually check that myself but i guess you mean the
handle to the file in memory is still open (which is natural as its refernce
counter is held up by your handle). I would believe the delete makes the
file disappear from view.

And if you write to the handle it should probably fail and ask you to
recreate the file. I can't see a problem in your code checking if the file
still exists from time to time.

Or if it doesn't fail then you will just have to SetFilePosition() or
GetFileSize() before writing in case you need to write some header or make
it visible again (close + create).

- Sten


 | 
Pages: 1
Prev: Create a Dialog
Next: partition&format question?