RAID + LUKS + LVM performance
in [Kernel]
|
Prev: [PATCH] perf: Make the install relative to DESTDIR if specified
Next: [PATCH v8 7/7] TCPCT part 2g: parse cookie pair and 64-bit timestamp
From: Matthias Schniedermeyer on 12 Mar 2010 07:10 On 12.03.2010 08:47, Mathias Buren wrote: > Matthias Schniedermeyer <ms(a)citd.de> wrote on 2010-03-11 17:36:04: > > > Re: RAID + LUKS + LVM performance > > > > Matthias Schniedermeyer > > > > to: > > > > Mathias Buren > > > > 2010-03-11 17:39 > > > > Cc: > > > > linux-kernel > > > > On 11.03.2010 13:08, Mathias Buren wrote: > > > > > > Hi, > > > > > > (please cc me as I'm not subscribed) > > > > > > I've a friend who's going to set up a fileserver consisting of 8x 1.5TB > > > HDDs, an 8-port PCI-E RAID card (Areca ARC-1220 @ > > > http://www.areca.com.tw/products/pcie.htm ) etc. > > > The plan is create a RAID5 array spanning all the disks, then create 4 > > > partitions. These 4 partitions would be encrypted using LUKS (Twofish > or > > > AES256). > > > These 4 encrypted partition would be set up in RAID0 using Linux' > software > > > (mdadm), then LVM would be used on top of that (one big PV, one big VG > and > > > a big LV or so). > > > > > > The reason for this is that kcryptd is not multithreaded (afaik). By > having > > > 4 encrypted partitions, then md0 on top of them, I'm forcing 4 kcryptd > > > processes to run on all four cpu cores whenever something is written to > the > > > disks, which should improve (encryption) performance. > > > > > > Is this a good way of doing it, or is there a smarter way? > > > > The setup you describe would only work with SSDs. HDDs would seek > > themselves to death. > > > > The problem is the RAID-0 over the 4 partitions. At that point you would > > need, instead of the 4 partitions, something that is round-robin. So > > that the mapping of the (physical) blocks from the upper to the lower > > would be effectivly linear/unchanged. > > > > AFAIK something like that is (currently) not possible. > > Hm. But I thought, since the hw RAID card does its own RAID5 thing on the > harddrives, that they wouldn't seek themselves do death. Perhaps they > would, anyway... > > What's the best way to set this up then? Or will kcryptd be able to > encrypt/decrypt everything fast enough anyway (~>5-600MB/s I'd say)? Personally i have only experience with loop-aes (which has the exact same problem) and with AES128 i reach a top-speed of about 130MB/s on my Core i7-860 for a single thread (tested with a SSD). Using one of the Westmere Core i5-6XX, which support AES-NI, should get you better performance (if supported by krcyptd), altough you loose 2 cores as onyl have 2. But i don't know what kind of performance improvement (if any) AES-NI provides. Or you wait a few weeks and buy on Core I7-980X with 6 core and AES-NI. Bis denn -- Real Programmers consider "what you see is what you get" to be just as bad a concept in Text Editors as it is in women. No, the Real Programmer wants a "you asked for it, you got it" text editor -- complicated, cryptic, powerful, unforgiving, dangerous. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Milan Broz on 12 Mar 2010 08:20 On 03/12/2010 01:06 PM, Matthias Schniedermeyer wrote: > Using one of the Westmere Core i5-6XX, which support AES-NI, should get > you better performance (if supported by krcyptd), altough you loose 2 > cores as onyl have 2. But i don't know what kind of performance > improvement (if any) AES-NI provides. > Or you wait a few weeks and buy on Core I7-980X with 6 core and AES-NI. dm-crypt uses whatever is implemented in cryptoAPI, AES-NI of course works when you have proper modules loaded. The main reason why dm-crypt is single-threaded is that all these new crypto accelerators uses asynchronous crypto API, so if hw allows parallel operation it can use it already. (dm-crypt submits all sectors in bio as separate asynchronous crypto requests). But note: single threaded per volume - if you have LVM with multiple encrypted LVs - every LV have its own thread. If we build another multi-core(thread) processing in dm-crypt, it can help on multi-core CPU but complicate things otherwise. (I tried simple implementation and abandoned it, I would prefer if cryptoAPI can do some parallel processing itself here. That question was discussed several times on dm-crypt mailing list.) Anyway, only two real-world examples, where one thread in dm-crypt can cause real problems with speed - dm-crypt over fast RAID(5) - dm-crypt over fast SSD. And many users of these configurations have now AES-NI acceleration. Milan -- mbroz(a)redhat.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Matthias Schniedermeyer on 12 Mar 2010 10:50
On 12.03.2010 14:12, Milan Broz wrote: > On 03/12/2010 01:06 PM, Matthias Schniedermeyer wrote: > > And many users of these configurations have now AES-NI acceleration. Do you have any numbers on the kind of performance gain you get from using AES-NI? Bis denn -- Real Programmers consider "what you see is what you get" to be just as bad a concept in Text Editors as it is in women. No, the Real Programmer wants a "you asked for it, you got it" text editor -- complicated, cryptic, powerful, unforgiving, dangerous. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |