Intel and AMD RDMA implementation
|
Prev: Simple Hack To Get $2500 To Your PayPal Account.
Next: ARM-based desktop computer ? (Hybrid computers ?: Low + High performance ;))
From: nmm1 on 29 Jul 2010 13:54 In article <r5b356900nnc1ubjb74p68ff6kb3fm0nsc(a)4ax.com>, George Neuner <gneuner2(a)comcast.net> wrote: >On Wed, 28 Jul 2010 19:28:54 +0100 (BST), nmm1(a)cam.ac.uk wrote: > >>POSIX asynchronous I/O is the real mess. Not merely does it allow >>the user to specify any accessible location as a buffer, which is >>incompatible with most forms of DMA, it doesn't forbid the program >>from reading the contents of a buffer with data being read into it >>asynchronously. And appending is specified to occur in the order >>of the aio_write calls, whatever that means ... > >It's a sequential model. However, thread execution isn't. >Preventing concurrent access isn't really possible unless the buffer >page is mapped out of the process during DMA. ... You have missed the point. They have specified that something is defined, when it isn't, at a deeper level than POSIX. Reading data while it is in the process of being changed is not a clever idea, at best. >>... which is REALLY bad news for some process/thread structures >>and schedulers. > >Not really following this. Concurrent access again? No. Deadlock, unswappable processes and so on. Regards, Nick Maclaren.
From: George Neuner on 29 Jul 2010 14:34 On Thu, 29 Jul 2010 18:54:47 +0100 (BST), nmm1(a)cam.ac.uk wrote: >In article <r5b356900nnc1ubjb74p68ff6kb3fm0nsc(a)4ax.com>, >George Neuner <gneuner2(a)comcast.net> wrote: >>On Wed, 28 Jul 2010 19:28:54 +0100 (BST), nmm1(a)cam.ac.uk wrote: >> >>>POSIX asynchronous I/O is the real mess. Not merely does it allow >>>the user to specify any accessible location as a buffer, which is >>>incompatible with most forms of DMA, it doesn't forbid the program >>>from reading the contents of a buffer with data being read into it >>>asynchronously. And appending is specified to occur in the order >>>of the aio_write calls, whatever that means ... >> >>It's a sequential model. > >However, thread execution isn't. POSIX is a *process* level sequential execution model which was defined without considering concurrent threads. POSIX threads were an afterthought that had to be designed around the already existing IO model. >>Preventing concurrent access isn't really possible unless the buffer >>page is mapped out of the process during DMA. ... > >You have missed the point. Undoubtedly. >They have specified that something is defined, when it isn't, >at a deeper level than POSIX. Reading data while it is in the >process of being changed is not a clever idea, at best. What I'm missing is what you believe "is defined when it isn't". The POSIX API includes synchronization to prevent concurrent access to the buffer in transit. It is not POSIX's fault if a programmer circumvents the API to do something stupid. The only viable fix for that is hardware enforcement of buffer hand-off. George
From: nmm1 on 29 Jul 2010 15:31 In article <aih356to4nkpn2518a8r80not679f5htol(a)4ax.com>, George Neuner <gneuner2(a)comcast.net> wrote: > >>>> And appending is specified to occur in the order >>>>of the aio_write calls, whatever that means ... >>> >>>It's a sequential model. >> >>However, thread execution isn't. > >POSIX is a *process* level sequential execution model which was >defined without considering concurrent threads. POSIX threads were an >afterthought that had to be designed around the already existing IO >model. We are agreed there. However, (a) asynchronous I/O was introduced later than threading (seriously) and (b) that's no excuse for making a pig's ear of it. >>They have specified that something is defined, when it isn't, >>at a deeper level than POSIX. Reading data while it is in the >>process of being changed is not a clever idea, at best. > >What I'm missing is what you believe "is defined when it isn't". See the specification of aio_read: "For any system action that changes the process memory space while an asynchronous I/O is outstanding to the address range being changed, the result of that action is undefined." Even ignoring the 'minor' detail that this says "system action" and it is reasonable to interpret that as not applying to the program, that says "changes" and not "accesses". So it explicitly implies that a system action may read data with an aio_read outstanding on it. Well, that's not sane. >The POSIX API includes synchronization to prevent concurrent access to >the buffer in transit. It is not POSIX's fault if a programmer >circumvents the API to do something stupid. The only viable fix for >that is hardware enforcement of buffer hand-off. No, it doesn't. It includes synchronisation between two memory accesses in different threads, but has NO form of synchronisation for any of the non-memory actions (like I/O). In any case, the use of asynchronous I/O to which I refer can be witin a SINGLE thread. No, however you cut it, POSIX's specification of asynchronous I/O is a disaster area. Regards, Nick Maclaren.
From: MitchAlsup on 29 Jul 2010 18:58 On Jul 29, 11:50 am, George Neuner <gneun...(a)comcast.net> wrote: > On Wed, 28 Jul 2010 19:28:54 +0100 (BST), n...(a)cam.ac.uk wrote: > >POSIX asynchronous I/O is the real mess. Not merely does it allow > >the user to specify any accessible location as a buffer, which is > >incompatible with most forms of DMA, it doesn't forbid the program > >from reading the contents of a buffer with data being read into it > >asynchronously. And appending is specified to occur in the order > >of the aio_write calls, whatever that means ... > > It's a sequential model. > > Preventing concurrent access isn't really possible unless the buffer > page is mapped out of the process during DMA. In my considered opinion. Any program, that expects to operate under any reasonable definition of correctness, cannot be concurrently accessing any buffer that has I/O scheduled to/from it. Mitch
From: George Neuner on 29 Jul 2010 19:19
On Thu, 29 Jul 2010 20:31:47 +0100 (BST), nmm1(a)cam.ac.uk wrote: >However, (a) asynchronous I/O was introduced later than threading >(seriously) and (b) that's no excuse for making a pig's ear of it. I really thought it was the other way around. Asynch I/O appeared in most Unix(-like) OSes before threads did and I thought the POSIX timeline reflected that as well. Live and learn. >See the specification of aio_read: "For any system action that changes >the process memory space while an asynchronous I/O is outstanding to the >address range being changed, the result of that action is undefined." > >Even ignoring the 'minor' detail that this says "system action" and >it is reasonable to interpret that as not applying to the program, >that says "changes" and not "accesses". So it explicitly implies >that a system action may read data with an aio_read outstanding on >it. Well, that's not sane. Badly written, I agree. I could argue that this is a paraphrasing of the spec and not the spec itself ... but I see your concern. >>The POSIX API includes synchronization to prevent concurrent access to >>the buffer in transit. > >No, it doesn't. It includes synchronisation between two memory >accesses in different threads, but has NO form of synchronisation >for any of the non-memory actions (like I/O). In any case, the use >of asynchronous I/O to which I refer can be witin a SINGLE thread. You and I have different ideas about what constitutes synchronization. From the page for aio.h: "The aio_sigevent member [of struct aiocb] defines the notification method to be used on I/O completion. If aio_sigevent.sigev_notify is SIGEV_NONE, no notification is posted on I/O completion, but the error status for the operation and the return status for the operation shall be appropriately set." It's quite clear that in almost every case the caller is expected to provide an event and then wait for it to be signaled before touching the buffer. As I said before, there isn't any way to keep a program from doing something stupid other than to prevent it with hardware. VMM page granularity makes that an unreasonable step to take. >however you cut it, POSIX's specification of asynchronous I/O >is a disaster area. The interface is not well designed, I agree. It would be made less dangerous simply by severing the program's connection to the control block and providing the signal event (maybe in thread local storage) rather than asking for one. There aren't many legitimate reasons for handing off a chunk of address space and forgetting about it. If a program makes an aio_write call and then gracefully exits, there is an expectation that the write will complete so the process can't terminate until the call completes. Given that, there's no harm to requiring the signal event. George |