Prev: NtDisplayString is not able to display unicode string
Next: Open Handle in one Process and Close in another?
From: Motonari Ito on 25 Feb 2010 19:49
I have a 3rd party library which reads a huge media file (7GB+).
Obviously the library uses ReadFile() Win32 API function without
FILE_FLAG_NO_BUFFERING flag and the size of the file system cache is
increased while the file is being read.

This behavior invalidates any valuable data existing in the file
system cache. After the file reading completes, the whole system is
slowed down for a while. The problem doesn't happen when the media
file is located on remote machine because the file system cache is not
used for remote files.

I don't have an access to the source code of the library (In fact, the
library is Apple Quick Time). Is there any way to disable the use of
file system cache for a particular file? I start to think writing a
file system driver would be the only option, but I want to confirm
before jumping into that.

Environment:
RAM: 8GB
OS: Windows 7 64bits edition

Motonari Ito
From: Alexander Grigoriev on 25 Feb 2010 22:25
Welcome to windows file cache hell. I was hoping the cache bloat problem was
fixed, but apparently I was wrong.

You could use something like Detours (which works in user mode) to intercept
CreateFile call, then add FILE_FLAG_SEQUENTIAL_SCAN to dwFlagsAndAttributes.
I don't recommend setting FILE_FLAG_NO_BUFFERING, because it places
additional constraints on the arguments of ReadFile/SetFilePosition.


"Motonari Ito" <motonari.ito(a)gmail.com> wrote in message
news:5bdd3cf4-c364-453f-9af1-97722e2860b6(a)b36g2000pri.googlegroups.com...
>I have a 3rd party library which reads a huge media file (7GB+).
> Obviously the library uses ReadFile() Win32 API function without
> FILE_FLAG_NO_BUFFERING flag and the size of the file system cache is
> increased while the file is being read.
>
> This behavior invalidates any valuable data existing in the file
> system cache. After the file reading completes, the whole system is
> slowed down for a while. The problem doesn't happen when the media
> file is located on remote machine because the file system cache is not
> used for remote files.
>
> I don't have an access to the source code of the library (In fact, the
> library is Apple Quick Time). Is there any way to disable the use of
> file system cache for a particular file? I start to think writing a
> file system driver would be the only option, but I want to confirm
> before jumping into that.
>
> Environment:
> RAM: 8GB
> OS: Windows 7 64bits edition
>
> Motonari Ito


From: Pavel A. on 26 Feb 2010 09:29
Try to hook ReadFile() in usermode and set FILE_FLAG_NO_BUFFERING, using
something like Detours or VMware ThinApp?
--pa


"Motonari Ito" <motonari.ito(a)gmail.com> wrote in message
news:5bdd3cf4-c364-453f-9af1-97722e2860b6(a)b36g2000pri.googlegroups.com...
> I have a 3rd party library which reads a huge media file (7GB+).
> Obviously the library uses ReadFile() Win32 API function without
> FILE_FLAG_NO_BUFFERING flag and the size of the file system cache is
> increased while the file is being read.
>
> This behavior invalidates any valuable data existing in the file
> system cache. After the file reading completes, the whole system is
> slowed down for a while. The problem doesn't happen when the media
> file is located on remote machine because the file system cache is not
> used for remote files.
>
> I don't have an access to the source code of the library (In fact, the
> library is Apple Quick Time). Is there any way to disable the use of
> file system cache for a particular file? I start to think writing a
> file system driver would be the only option, but I want to confirm
> before jumping into that.
>
> Environment:
> RAM: 8GB
> OS: Windows 7 64bits edition
>
> Motonari Ito

From: Alexander Grigoriev on 26 Feb 2010 09:46
FFNB may not work because of alignment constraints.

"Pavel A." <pavel_a(a)12fastmail34.fm> wrote in message
news:C1EA6D79-992E-4FCF-862B-B9F4B384CFE4(a)microsoft.com...
> Try to hook ReadFile() in usermode and set FILE_FLAG_NO_BUFFERING, using
> something like Detours or VMware ThinApp?
> --pa
>
>
> "Motonari Ito" <motonari.ito(a)gmail.com> wrote in message
> news:5bdd3cf4-c364-453f-9af1-97722e2860b6(a)b36g2000pri.googlegroups.com...
>> I have a 3rd party library which reads a huge media file (7GB+).
>> Obviously the library uses ReadFile() Win32 API function without
>> FILE_FLAG_NO_BUFFERING flag and the size of the file system cache is
>> increased while the file is being read.
>>
>> This behavior invalidates any valuable data existing in the file
>> system cache. After the file reading completes, the whole system is
>> slowed down for a while. The problem doesn't happen when the media
>> file is located on remote machine because the file system cache is not
>> used for remote files.
>>
>> I don't have an access to the source code of the library (In fact, the
>> library is Apple Quick Time). Is there any way to disable the use of
>> file system cache for a particular file? I start to think writing a
>> file system driver would be the only option, but I want to confirm
>> before jumping into that.
>>
>> Environment:
>> RAM: 8GB
>> OS: Windows 7 64bits edition
>>
>> Motonari Ito
>


From: Pavel A. on 26 Feb 2010 09:53
"Alexander Grigoriev" <alegr(a)earthlink.net> wrote in message
news:eomFGKvtKHA.732(a)TK2MSFTNGP06.phx.gbl...
> FFNB may not work because of alignment constraints.

You're right. Then this needs a more sophisticated patch, with
intermediate buffer.

Or... why not just report this problem to Apple and let them sort it out?
--pa

> "Pavel A." <pavel_a(a)12fastmail34.fm> wrote in message
> news:C1EA6D79-992E-4FCF-862B-B9F4B384CFE4(a)microsoft.com...
>> Try to hook ReadFile() in usermode and set FILE_FLAG_NO_BUFFERING, using
>> something like Detours or VMware ThinApp?
>> --pa
>>
>>
>> "Motonari Ito" <motonari.ito(a)gmail.com> wrote in message
>> news:5bdd3cf4-c364-453f-9af1-97722e2860b6(a)b36g2000pri.googlegroups.com...
>>> I have a 3rd party library which reads a huge media file (7GB+).
>>> Obviously the library uses ReadFile() Win32 API function without
>>> FILE_FLAG_NO_BUFFERING flag and the size of the file system cache is
>>> increased while the file is being read.
>>>
>>> This behavior invalidates any valuable data existing in the file
>>> system cache. After the file reading completes, the whole system is
>>> slowed down for a while. The problem doesn't happen when the media
>>> file is located on remote machine because the file system cache is not
>>> used for remote files.
>>>
>>> I don't have an access to the source code of the library (In fact, the
>>> library is Apple Quick Time). Is there any way to disable the use of
>>> file system cache for a particular file? I start to think writing a
>>> file system driver would be the only option, but I want to confirm
>>> before jumping into that.
>>>
>>> Environment:
>>> RAM: 8GB
>>> OS: Windows 7 64bits edition
>>>
>>> Motonari Ito
>>
>
>
 |  Next  |  Last
Pages: 1 2 3 4
Prev: NtDisplayString is not able to display unicode string
Next: Open Handle in one Process and Close in another?