Filesystem recommendations
in [Debian]
|
Prev: overcoming the 32k objects limit is ext3 - which file system to use?
Next: Why is Acroversion not properly updated?
From: Joe Brenner on 29 Apr 2010 21:10 Boyd Stephen Smith Jr. <bss(a)iguanasuicide.net> wrote: > Joe Brenner wrote: > > Ron Johnson <ron.l.johnson(a)cox.net> wrote: > > > B. Alexander wrote: > > > > Ron Johnson<ron.l.johnson(a)cox.net> wrote: > > > >> XFS is the canonical fs for when you have lots of Big Files. I've > > > >> also seen simple benchmarks on this list showing that it's faster > > > >> than ext3/ext4. > > > > > > > > Thats cool. What about Lots of Little Files? That was another of the > > > > draws of reiser3. > > > > > > That same unofficial benchmark showed surprising small-file speed by > > > xfs. > > Would you happen to have any links to such benchmarks, unofficial or > > otherwise? > > My experience has been that whenever I look at filesystem benchmarks, > > they skip the many-small-files case. I've always had the feeling that > > most of the big filesystems cared a lot about scaling up in file-size, > > but not too much about anything else. > NB: This is my best recollection; I'm not looking this up right now. Please > check my facts, I'd love to know if I'm wrong. Like I said, I *have* looked at filesystem comparisons a number of times. It's my problem to check your assertions? Why isn't it your problem to check my assertions? > > I'm a Reiser3 user myself, and I've never had any problems with it. > > (The trouble with it being "long in the tooth" is mostly hypothetical, > > isn't it?) > > Not really. Outside of one mention of "bugs on reiserfs that will not be fixed", you're pretty much just describing the theory. I do understand that using relatively unsupported software, even if it's pretty mature software, can have it's problems. Just doing a few quick websearches, I'm reading about ReiserFS bugs fixed as recently as 2006, 2007... It's not like it's not getting any attention from developers. > In addition, as file system technology advances, reiserfs will become > less attractive for new installs and it will become more attractive to > migrate way from it. I think you're better off if you rely on really well-tested migration tools (e.g. "tar"). -- To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org Archive: http://lists.debian.org/201004300103.o3U13KCh005837(a)kzsu.stanford.edu
From: Mark Allums on 30 Apr 2010 10:40 On 4/29/2010 7:36 AM, Stan Hoeppner wrote: In the U.S., given the numbers of > cheap APC, Triplite, and Belkin UPS on the shelves at $big_box_store I'd say > most U.S. desktop users have a UPS. I know I do. Naw. It ain't so. Most US users don't even know what a UPS is. APC quit calling their equipment a "UPS", and now just call it a "backup battery" because of alphabet soup fatigue. And Jane Church-Secretary still doesn't know what it is or what it's for. Or or Jack Retired-Contractor. Joe Average. MAA -- To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org Archive: http://lists.debian.org/4BDAEB14.7000202(a)allums.com
From: Boyd Stephen Smith Jr. on 30 Apr 2010 11:30 On Thursday 29 April 2010 20:03:20 Joe Brenner wrote: > Boyd Stephen Smith Jr. <bss(a)iguanasuicide.net> wrote: > > Joe Brenner wrote: > > > Ron Johnson <ron.l.johnson(a)cox.net> wrote: > > > > B. Alexander wrote: > > > > > Ron Johnson<ron.l.johnson(a)cox.net> wrote: > > > > >> XFS is the canonical fs for when you have lots of Big Files. I've > > > > >> also seen simple benchmarks on this list showing that it's faster > > > > >> than ext3/ext4. > > > > > > > > > > Thats cool. What about Lots of Little Files? That was another of > > > > > the draws of reiser3. > > > > > > > > That same unofficial benchmark showed surprising small-file speed by > > > > xfs. > > > > > > Would you happen to have any links to such benchmarks, unofficial or > > > otherwise? > > > > > > My experience has been that whenever I look at filesystem benchmarks, > > > they skip the many-small-files case. I've always had the feeling that > > > most of the big filesystems cared a lot about scaling up in file-size, > > > but not too much about anything else. > > > > NB: This is my best recollection; I'm not looking this up right now. > > Please check my facts, I'd love to know if I'm wrong. > > Like I said, I *have* looked at filesystem comparisons a number of times. So have I. I didn't mean to imply otherwise. I looked at them for deciding on reiserfs, then I replicated the results on my own hardware using bonnie++ before restoring my backups. I also look around for results about once a year, to see if much has changed, or if there are new file systems I should look at. Less often, I'll try and run my own bonnie++ tests, but not rigorously; they occur on my system under whatever load I happen to be running. > It's my problem to check your assertions? Trust, but verify. The note was only indicative that I wasn't didn't have data or analysis in front of me, so I was running off of memory. Usually, that's fine, but I was talking about specific file system implementation features in multiple file systems. Since I don't implement or debug file systems every day, I thought an idle warning was in order. > Why isn't it your problem to > check my assertions? It is. > > > I'm a Reiser3 user myself, and I've never had any problems with it. > > > > > > (The trouble with it being "long in the tooth" is mostly hypothetical, > > > isn't it?) > > > > Not really. > > Outside of one mention of "bugs on reiserfs that will not be fixed", > you're pretty much just describing the theory. I do understand that > using relatively unsupported software, even if it's pretty mature > software, can have it's problems. That's not a theory; or that least it is not hypothetical. It's proven true over and over, and software ranging from OSes to libraries to RDBMSes to desktop applications. > Just doing a few quick websearches, I'm reading about ReiserFS bugs > fixed as recently as 2006, 2007... It's not like it's not getting any > attention from developers. Took me a little while, but I see bug-fix patches from this year. It's not be abandoned quite as quickly as I was lead to believe. I still don't recommend it for new installs, but there's a-priori reason to migrate away from it right now. > > In addition, as file system technology advances, reiserfs will become > > less attractive for new installs and it will become more attractive to > > migrate way from it. > > I think you're better off if you rely on really well-tested migration > tools (e.g. "tar"). These have significant overhead over file system-specific methods. They are the both universal and fairly conservative, so your advocacy is well- justified. It's also pretty easy to do them wrong, or get a poorly performing file system out of them. It's easy to forget extended attributes and ACLs when using cp/tar/rsync; there may be other file system data that needs to be preserved, too (HPFS+ forks?). Taking a kernel tarball from a ext3 file system and extracting it on a reiserfs file system takes much longer than taking a kernel tarball from a reiserfs file system and extracting to on a reiserfs file system. -- Boyd Stephen Smith Jr. ,= ,-_-. =. bss(a)iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/ \_/
From: Boyd Stephen Smith Jr. on 3 May 2010 13:10 On Monday 26 April 2010 16:34:38 Boyd Stephen Smith Jr. wrote: > On Monday 26 April 2010 16:05:31 B. Alexander wrote: > > On Mon, Apr 26, 2010 at 2:22 PM, Boyd Stephen Smith Jr. < > > > > bss(a)iguanasuicide.net> wrote: > > > I'm also a current reiser3 user. I find the ability to shrink the > > > filesystem > > > to be something I am not willing to do without. > > > > You know, I said the same thing, but then as the kernel and GRUB and the > > like advanced, I noticed that my reiserfs partitions would have to replay > > the journal every time I rebooted, even after a clean shutdown. I started > > calculating how many times I shrunk any of my partitions in the last 8 > > years, and I can only recall twice. And since I have several terabytes > > around the house, I figure I can migrate data and delete/recreate > > partitions if I really need to reduce it. > > That doesn't seem right. I have been using reiser3 since 2005, and my > system does not require a journal replay if I do a clean shutdown/reboot. > A forced reboot through Alt+SysRq+B does trigger a journal replay (as it > should). > > I also have 4+ tebibytes but most of them are allocated to filesystems. > I've had to shrink filesystems dozens of times since 2005, during or after > a data move. > > I don't use partitions (much), having been using LVM happily for everything > except /boot. I'm hoping to be able to move that onto LVM once I move to > GRUB2 and GPT. > > > > I have not read the rest of the thread, but my off-the-cuff > > > recommendation would be to start migration to btrfs. Now that the > > > on-disk format has stabilized, I am going to start testing it for > > > filesystems other than /usr/local, /var, and /home. Assuming I can > > > keep those running well for 6-12 > > > months, I will migrate /usr/local, /var, and then /home, in that order, > > > with a > > > 1-3 month gap in between migrations. > > > > I might play with it for some non-critical partitions, or ones that I can > > mirror on an established filesystem, even if it is only to use in an > > "Archive Island" scenario, where I have a LV that I can mount, sync and > > umount. However, btrfs is not included in the kernel, is it? As I recall, > > nilfs2 has kernel support, but that was the only one of the new > > filesystems, at the time when I started looking at this. > > btrfs is included in 2.6.31.12-0.2-default in openSUSE 11.2. It is also > included in linux-image-2.6-686 and linux-image-2.6-amd64 for > lenny-backports, testing, and sid. I don't normally deal with other > architectures/distributions, so it might also be available there. > > > > I've already encountered an issue related to btrfs in my very isolated > > > deployments. The initramfs created by update-initramfs does not appear > > > to mount it properly. Instead I am given an '(initramfs)' prompt and I > > > have to > > > mount the filesystem manually (a simple two-argument mount command > > > suffices) > > > and continue the boot process. > > > > That is enough to give me pause... > > It doesn't appear to be a file system issue, but rather a problem with the > initramfs scripts. It could also be rooted in my configuration. I know > that my "root=" kernel parameter has to differ from the device name in my > /etc/fstab in order to get the initramfs to correctly initialize LVM. I wanted to report that I was able to diagnose and solve this. The problem was that /lib/udev/vol_id in my initramfs could not identify a btrfs file system, which caused the scripts to try and mount the root file system using '-t unknown' as part of the command-line arguments. I upgraded initramfs-tools and udev to the Sid versions. This should cause a initramfs rebuild as part of the postinst, but if it doesn't do so on your system, be sure to run (update-initramfs -k all -u). Now my system boots unattended with a btrfs '/'. So, at this time, btrfs can be used for non-'/' file systems with the tools from lenny-backports. However, newer udev/initramfs-tools are required to boot with a btrfs '/'. I hope they are included in the Squeeze release. I have not, and probably will not test a btrfs '/boot'. I have also been encountering issues with suspend and resume on this new install. I don't currently believe these are btrfs-related, either, but fair warning. -- Boyd Stephen Smith Jr. ,= ,-_-. =. bss(a)iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/ \_/
From: Ron Johnson on 3 May 2010 22:20
On 04/29/2010 02:17 PM, Joe Brenner wrote: > Ron Johnson<ron.l.johnson(a)cox.net> wrote: >> B. Alexander wrote: >>> Ron Johnson<ron.l.johnson(a)cox.net> wrote: > >> [snip] > >>>> XFS is the canonical fs for when you have lots of Big Files. I've >>>> also seen simple benchmarks on this list showing that it's faster >>>> than ext3/ext4. > >>> Thats cool. What about Lots of Little Files? That was another of the draws >>> of reiser3. I have a space I mount on /media/archive, which has everything >>> from mp3/oggs and movies, to books to a bunch of tiny files. This will >>> probably be the first victim for the xfs test partition. >> >> That same unofficial benchmark showed surprising small-file speed by >> xfs. > > Would you happen to have any links to such benchmarks, unofficial or > otherwise? They were posted to this list (within the last 6 months, I think). > My experience has been that whenever I look at filesystem benchmarks, > they skip the many-small-files case. It also did small file testing. > I've always had the feeling that > most of the big filesystems cared a lot about scaling up in file-size, > but not too much about anything else. Yes, that's what xfs was designed for, and where it's strength always lay. But the benchmarks I refer to showed good results for xfs even with small files. > I'm a Reiser3 user myself, and I've never had any problems with it. > > (The trouble with it being "long in the tooth" is mostly hypothetical, > isn't it?) > It's certainly not suffering yet suffering from bit-rot. However, while ext4, xfs and btrfs are *definitely* being continuously improved I doubt that ReiserFS 3.5 is. (As usual, I could be totally wrong.) -- Dissent is patriotic, remember? -- To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org Archive: http://lists.debian.org/4BDF837D.7060000(a)cox.net |