Prev: DevCon - Status Finds, Disable Doesn't.?
Next: Signed catalog file still giving me "can't verify publisher" error
From: eagersh on 19 Aug 2010 12:55
On Aug 19, 3:40 am, pascal <pas...(a)discussions.microsoft.com> wrote:
> if i need to store in all data in memory for example 6 GByte all in memory
> (64bit OS), the question is what memory? Virtual /physical  VirtualAlloc can
> reserve 6 GByte in the Virtual memory page file but will that will be too
> slow? i mean that windows memory manger may could not stand up to the rate of
> data needed to writen to the page file.
> or use physical memory, what is the best way/practice ?
>
> Thanks
>
> "Tim Roberts" wrote:
> > pascal <pas...(a)discussions.microsoft.com> wrote:
>
> > >i have question and looking for help.
>
> > >the situation: the appliction needs to read from the driver/Board (PCIe)
> > >data rate of about ~2GByte/s to the pc first save it to memory then to buffer
> > >it to loacal drive for later process.
>
> > For how long?  Hot drives can save about 40MB/s, so you will need to store
> > it all in memory.
> > --
> > Tim Roberts, t...(a)probo.com
> > Providenza & Boekelheide, Inc.
> > .

The best approach is to allocate virtual memory and map it by using
DIRECT_IO request. In this case you will avoid extra memory copy. You
should figure out how much virtual memory you could lock in physical
in one IOCTL request for x64bit Windows.

Igor Sharovar
From: Tim Roberts on 22 Aug 2010 01:41
pascal <pascal(a)discussions.microsoft.com> wrote:
>
>if i need to store in all data in memory for example 6 GByte all in memory
>(64bit OS), the question is what memory? Virtual /physical VirtualAlloc can
>reserve 6 GByte in the Virtual memory page file but will that will be too
>slow? i mean that windows memory manger may could not stand up to the rate of
>data needed to writen to the page file.

I already told you, disks can't handle it. Doesn't matter whether you're
writing it directly or letting the page manager write it. It's still the
same disk. You will have to allocate virtual memory to hold the whole
thing, grab 2 or 3 seconds worth, then stop gathering for 3 minutes while
you flush it out to the disk.

>> >the situation: the appliction needs to read from the driver/Board (PCIe)
>> >data rate of about ~2GByte/s to the pc first save it to memory then to buffer
>> >it to loacal drive for later process.

You do realize that it's going to require a 16-lane PCIExpress board to
transfer 2GB/s, right? You are designing it as 16-lane from the beginning?
And are you quite sure your memory subsystem can sustain 2GB/s?
--
Tim Roberts, timr(a)probo.com
Providenza & Boekelheide, Inc.
From: pascal on 23 Aug 2010 03:01

Hi thanks for the help:

from the information i gathered my plan is:

1. try to alloc virutal memory about 4GByte for 2sec properly using
VirtualAlloc and VirutalLock API function to lock all memory in to the
physical memory to avoid paging out of memory to the disk. (are VirtualAlloc
and VirutalLock do work? i saw alot poeple had problem with those function
when trying to lock alot of memory?)

2. pass the memory pointer to the driver using IOCTL command or when using
the ReadFile() IOREAD routine.

3. when the transfer and IO is complete, save data to the Hard drive.

4. the platform used is winXP x64 with 6GByte memory and the target board is
the xilinx PCIe core that current support up to 10Gbit/s (1.25GByte) using x8
lanes, but the plan is to support 2GByte later on.

what to you think ? is that plan practicable/feasible?


From: Tim Roberts on 24 Aug 2010 01:08
pascal <pascal(a)discussions.microsoft.com> wrote:
>
>3. when the transfer and IO is complete, save data to the Hard drive.
>
>4. the platform used is winXP x64 with 6GByte memory and the target board is
>the xilinx PCIe core that current support up to 10Gbit/s (1.25GByte) using x8
>lanes, but the plan is to support 2GByte later on.

1.25GB/s is a stretch for an 8-lane device

>what to you think ? is that plan practicable/feasible?

As long as you never need to capture more than you can hold in memory,
sure.
--
Tim Roberts, timr(a)probo.com
Providenza & Boekelheide, Inc.
First  |  Prev  | 
Pages: 1 2
Prev: DevCon - Status Finds, Disable Doesn't.?
Next: Signed catalog file still giving me "can't verify publisher" error