TLBs which cache invalid (not present) entries
|
From: Andy Glew "newsgroup at on 3 Aug 2010 20:48 On 8/3/2010 7:59 AM, EricP wrote: > Andy Glew wrote: >> >> If ever you see flakey results, on x86 or elsewhere I would strongly >> suggest that you have your invalid page exception handler rewalk the >> page tables to see if the page is, indeed, invalid. > > In a multi-threaded SMP OS, I think you may, depending on the OS design, > have to always do that. The OS _should_ allow concurrent page faults > from different threads in the same process - no reason not to - > with access to the process page table coordinated by a mutex. > It is therefore possible that between when a fault occurs and when > the table mutex is granted, another thread could patch up the PTE. This is a reason not to have the hardware or microcode rewalk the page tables when reporting a fault. Otherwise, you might end up having walked the page tables 3 times: First, the speculative TLB miss page walk by hardware. Second, the non-speculative TLB miss page walk by hardware (or microcode) when reporting the fault. Third, the page walk inside the OS page fault handler. That's a whole lot of walking, although some of these page table walks may be cut short by interior node caches such as the PDE (and higher) cache.
From: Nick Maclaren on 4 Aug 2010 04:45 In article <UtGdnbKuyMxxJcXRnZ2dnUVZ_g2dnZ2d(a)giganews.com>, Andy Glew <"newsgroup at comp-arch.net"> wrote: >>> >>> If ever you see flakey results, on x86 or elsewhere I would strongly >>> suggest that you have your invalid page exception handler rewalk the >>> page tables to see if the page is, indeed, invalid. >> >> In a multi-threaded SMP OS, I think you may, depending on the OS design, >> have to always do that. The OS _should_ allow concurrent page faults >> from different threads in the same process - no reason not to - >> with access to the process page table coordinated by a mutex. >> It is therefore possible that between when a fault occurs and when >> the table mutex is granted, another thread could patch up the PTE. > >This is a reason not to have the hardware or microcode rewalk the page >tables when reporting a fault. Otherwise, you might end up having >walked the page tables 3 times: > >First, the speculative TLB miss page walk by hardware. > >Second, the non-speculative TLB miss page walk by hardware (or >microcode) when reporting the fault. > >Third, the page walk inside the OS page fault handler. It's an even better reason to abolish page faulting altogether! As posted before, it would be trivial to do at the hardware level, fairly easy to do at the software level, and seriously compromise only a very few, very perverse usages. But it still rocks the boat too much to be considered nowadays :-( Regards, Nick Maclaren.
From: Stephen Fuld on 4 Aug 2010 12:32 On 8/4/2010 1:45 AM, Nick Maclaren wrote: > In article<UtGdnbKuyMxxJcXRnZ2dnUVZ_g2dnZ2d(a)giganews.com>, > Andy Glew<"newsgroup at comp-arch.net"> wrote: snip >> This is a reason not to have the hardware or microcode rewalk the page >> tables when reporting a fault. Otherwise, you might end up having >> walked the page tables 3 times: >> >> First, the speculative TLB miss page walk by hardware. >> >> Second, the non-speculative TLB miss page walk by hardware (or >> microcode) when reporting the fault. >> >> Third, the page walk inside the OS page fault handler. > > It's an even better reason to abolish page faulting altogether! > As posted before, it would be trivial to do at the hardware level, > fairly easy to do at the software level, and seriously compromise > only a very few, very perverse usages. > > But it still rocks the boat too much to be considered nowadays :-( How about a compromise where we just increase the page size? I know of one system hat uses 16KB pages. This should reduce the number of page faults, yet still require no application level changes and allow for those few programs that really need large sparse address spaces. But are page faults really a performance issue with today's larger memories? -- - Stephen Fuld (e-mail address disguised to prevent spam)
From: Nick Maclaren on 4 Aug 2010 13:49 In article <i3c4ni$m0r$1(a)news.eternal-september.org>, Stephen Fuld <SFuld(a)Alumni.cmu.edu.invalid> wrote: > >>> This is a reason not to have the hardware or microcode rewalk the page >>> tables when reporting a fault. Otherwise, you might end up having >>> walked the page tables 3 times: >>> >>> First, the speculative TLB miss page walk by hardware. >>> >>> Second, the non-speculative TLB miss page walk by hardware (or >>> microcode) when reporting the fault. >>> >>> Third, the page walk inside the OS page fault handler. >> >> It's an even better reason to abolish page faulting altogether! >> As posted before, it would be trivial to do at the hardware level, >> fairly easy to do at the software level, and seriously compromise >> only a very few, very perverse usages. >> >> But it still rocks the boat too much to be considered nowadays :-( > >How about a compromise where we just increase the page size? I know of >one system hat uses 16KB pages. This should reduce the number of page >faults, yet still require no application level changes and allow for >those few programs that really need large sparse address spaces. Not really, unfortunately, for two reasons. Firstly, most of the benefit comes from abolishing the need for transparent fixup of page faults. Secondly, increasing the page size often just increases the memory requirements for sparse address spaces. It's trivial to do the calculation for random address distributions, for many common ones, and the numbers are ugly - especially for the simple case of UUID values. Perhaps the best argument against it is that it has been tried, many times, and has failed every time (as a solution to this problem). The systems that use large pages to tackle it usually use very large ones (e.g. 4 MB) or variable ones, and use THEM to make certain segments effectively immune from page faults. >But are page faults really a performance issue with today's larger memories? Yes. They often make it worse. The sole issue for many applications is the proportion of their memory that can be mapped at any one time. Consider any matrix method that has no known blocking form, and necessarily uses accesses 'both ways round' closely together. As soon as the matrix exceeds the size mapped by the TLB, there is a BIG performance problem. Regards, Nick Maclaren.
From: George Neuner on 4 Aug 2010 13:53
On Wed, 04 Aug 2010 09:32:51 -0700, Stephen Fuld <SFuld(a)alumni.cmu.edu.invalid> wrote: >But are page faults really a performance issue with today's larger memories? Depends on how you use them. Some garbage collectors work by setting "no access" on the heap and then processing pages incrementally as the mutator faults on them. This deliberate (ab)use of VMM impacts not only the GC'd program but the entire system. And anyway, size doesn't matter if it's overcommitted. George |