Aerospace and linux
in [Kernel]
|
Prev: Slow pty's (was Re: libdivecomputer interfaces?)
Next: [PATCH] ipmi: Make sure drivers were registered before unregistering them
From: Andi Kleen on 10 Jun 2010 14:50 On Thu, Jun 10, 2010 at 12:38:10PM -0600, Brian Gordon wrote: > > It's also a serious consideration for standard servers. > Yes. Good point. > > > On server class systems with ECC memory hardware does that. > > > Normally server class hardware handles this and the kernel then reports > > memory errors (e.g. through mcelog or through EDAC) > > Agreed. EDAC is a good and sane solution and most companies do this. Sorry, but you mean ECC? IMHO EDAC Is not a good solution for error reporting (however I'm biased because I work on a better one) > Some do not due to naivity or cost reduction. EDAC doesn't cover > processor registers and I have fairly good solutions on how to deal > with that in tiny "home-grown" tasking systems. mcelog covers OS visible processor registers on x86 systems. If your hardware doesn't support it it's hard for the general case, although special cases are always possible. > > > Lower end systems which are optimized for cost generally ignore the > > problem though and any flipped bit in memory will result > > in a crash (if you're lucky) or silent data corruption (if you're unlucky) > > Right! And this is the area that I am interested in. Some people > insist on lowering the cost of the hardware without considering these > issues. One thing I want to do is to be as diligent as possible (even > in these low cost situations) and do the best job I can in spite of > the low cost hardware. AFAIK there's no support for this in a standard Linux kernel. That is some architectures do scrubbing in software, but the basic ECC implementation is still hardware. In general I suspect you'll need some application specific strategy, if your hardware doesn't help you in this. Having good hardware definitely helps, software is generally not happy if it cannot trust its memory enough. It's a bit like a human with no reliable air supply. That is the existing memory error handling mechanisms (like hwpoison) assume events are reliably detected and are relatively rare. > > So, some pages of RAM are going to be read-only and the data in those > pages came from some source (file system?). Can anyone describe a > high level strategy to occasionaly provide some coverage of this data? Just for block data there's some support for checksumming e.g, block integrity (needs special support in the device) or file systems (e.g. btrfs) However they all normally assume memory is reliable and are more focussed on errors coming from storage. > > So far I have thought about page descriptors adding an MD5 hash > whenever they are read-only and first being "loaded/mapped?" and then > a background daemon could occasionaly verify. In theory btrfs or block integrity could be probably extended to regularly re check page cache. It would not be trivial But to really catch errors before use you would need to recheck on every access, and that's hard (or rather extremly slow) in some cases (e.g. mmap) And this still wouldn't help with r/w memory. Normally on most workloads r/o (that is clean) memory is the only a small fraction of the active memory. -Andi -- ak(a)linux.intel.com -- Speaking for myself only. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Chris Friesen on 10 Jun 2010 14:50 On 06/10/2010 12:38 PM, Brian Gordon wrote: > On the more exotic end, I have also seen systems that have dual > redundant processors / memories. Then they add compare logic between > the redundant processors that compare most pins each clock cycle. If > any pins are not identical at a clock cycle, then something has gone > wrong (SEU, hardware failure, etc..) Some phone switches do this. Some of them also have at least two copies of everything in memory and will do transactional operations that can be rolled back if there is a hardware glitch. > So, some pages of RAM are going to be read-only and the data in those > pages came from some source (file system?). Can anyone describe a > high level strategy to occasionaly provide some coverage of this data? > So far I have thought about page descriptors adding an MD5 hash > whenever they are read-only and first being "loaded/mapped?" and then > a background daemon could occasionaly verify. Makes sense to me. You might also pick an on-disk format with extra checksumming so you could compare the on-disk checksum with the in-memory checksum. Chris -- The author works for GENBAND Corporation (GENBAND) who is solely responsible for this email and its contents. All enquiries regarding this email should be addressed to GENBAND. Nortel has provided the use of the nortel.com domain to GENBAND in connection with this email solely for the purpose of connectivity and Nortel Networks Inc. has no liability for the email or its contents. GENBAND's web site is http://www.genband.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Brian Gordon on 10 Jun 2010 15:20 Thank you both. This has been very helpful for me. I think I read two conclusions: 1) R/O is a small percentage of RAM 2) To cover this small precentage would be non-trivial Thank you both very much for your time and knowledge, I'll move along now. On Thu, Jun 10, 2010 at 12:46 PM, Chris Friesen <cfriesen(a)nortel.com> wrote: > On 06/10/2010 12:38 PM, Brian Gordon wrote: > >> On the more exotic end, I have also seen systems that have dual >> redundant processors / memories. �Then they add compare logic between >> the redundant processors that compare most pins each clock cycle. � If >> any pins are not identical at a clock cycle, then something has gone >> wrong (SEU, hardware failure, etc..) > > Some phone switches do this. �Some of them also have at least two copies > of everything in memory and will do transactional operations that can be > rolled back if there is a hardware glitch. > >> So, some pages of RAM are going to be read-only and the data in those >> pages came from some source (file system?). � Can anyone describe a >> high level strategy to occasionaly provide some coverage of this data? > >> So far I have thought about page descriptors adding an MD5 hash >> whenever they are read-only and first being "loaded/mapped?" and then >> a background daemon could occasionaly verify. > > Makes sense to me. �You might also pick an on-disk format with extra > checksumming so you could compare the on-disk checksum with the > in-memory checksum. > > Chris > > -- > The author works for GENBAND Corporation (GENBAND) who is solely > responsible for this email and its contents. All enquiries regarding > this email should be addressed to GENBAND. Nortel has provided the use > of the nortel.com domain to GENBAND in connection with this email solely > for the purpose of connectivity and Nortel Networks Inc. has no > liability for the email or its contents. GENBAND's web site is > http://www.genband.com > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Massimiliano Galanti on 10 Jun 2010 15:40 > What about .ro and .text sections of an executable? I would think > kernel support for that would be required. If its application data, > then all sorts of things are possible like you described. Ive also > seen critical ram variables be stored in triplicate and then > compared/voted just to ensure no silent SEU corruption. Maybe slightly off topic but... if flash is safer than ram, what about using XIP (where possible, e.g. on NORs)? That would not put .data sections into ram, at least. -- Massimiliano -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Brian Gordon on 10 Jun 2010 15:50
Sorry, I take it back. This wont work for me because I wont have NOR. Also, I only want the "in-place" to apply to read-only pages. This looks like all reads and writes get passed to the underlying storage and I can't suffer flash page erase/writes to update a variable. :) The device will wear out and meaningful work would be starved. On Thu, Jun 10, 2010 at 1:37 PM, Brian Gordon <legerde(a)gmail.com> wrote: > Yes. Thats exactly what I am looking for. Even if there is a speed > penalty, I wouldn't mind so much. However wikipedia says that XIP is > filesystem dependent and im stuck with FAT32 or NTFS. Wikipedia > claims NTFS can do XIP. Is this true under linux? > > On Thu, Jun 10, 2010 at 1:23 PM, Massimiliano Galanti > <massiblue(a)libero.it> wrote: >> >>> What about .ro and .text sections of an executable? I would think >>> kernel support for that would be required. If its application data, >>> then all sorts of things are possible like you described. Ive also >>> seen critical ram variables be stored in triplicate and then >>> compared/voted just to ensure no silent SEU corruption. >> >> Maybe slightly off topic but... if flash is safer than ram, what about using >> XIP (where possible, e.g. on NORs)? That would not put .data sections into >> ram, at least. >> >> -- >> Massimiliano >> > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |