difference between striping using mdadm and LVM
in [General Linux]
|
From: Aragorn on 22 Jan 2010 08:42 On Friday 22 January 2010 00:21 in comp.os.linux.misc, somebody identifying as David Brown wrote... > Aragorn wrote: > >> On Wednesday 20 January 2010 23:59 in comp.os.linux.misc, somebody >> identifying as David Brown wrote... >> >>> rsync copying is even cleaner - the backup copy is directly >>> accessible. And when combined with hard link copies in some way >>> (such as rsnapshot) you can get snapshots. >> >> I have seen this method being discussed before, but to be honest I've >> never even looked into "rsnapshot". I do intend to explore it for >> the future, since the ability to make incremental backups seems very >> interesting. > > Another poster has given you a pretty good explanation of how rsync > snapshot backups work. Yep, Bill is no stranger to me - at least, Usenet-wise; I've never met him in person. ;-) He's a longstanding regular of both alt.os.linux.mandr* groups and hangs out - Usenet-wise - in just about the same groups as I do. ;-) > I'll just give a few more points here. > > rsync is designed to make a copy of a directory as efficiently as > possible. It will only copy over files that have changed or been > added, and even for changed files it can often copy over just the > changes rather than the whole file. And if you are doing the rsync > over a slow network, you can compress the transfers. There are > additional flags to delete files in the destination that are no longer > present in the source, and to omit certain files from the copy > (amongst many other flags). Okay, sounds interesting. And yes, part of the backing up will be over the network to a separate machine. > For snapshots, this is combined with the "cp -al" command that copies > a tree but hard-links files rather than copying them. So you do > something like rsync copy the source tree to a "current copy", then > "cp -al" the "current copy" to a dated backup snapshot directory. The > next day, you repeat the process - only changes from the source to > the "current copy" are transferred, and any files left untouched will > be hardlinked each time - you only ever have one real copy of each > file, with hardlinks in each snapshot. It's not perfect - for > example, a file rename will cause a new transfer, for example > (the "--fuzzy" flag can be used to avoid the transfer, but not the > file duplication). And if you have partial transfers you can end up > breaking the hard-link chaining and end up with extra copies of the > files (and thus extra disk space). I'd rather have a duplicate file with differing names than having to find out that I've lost the only copy. ;-) > rsnapshot and dirvish are two higher level backup systems build on > this technique. I have heard "dirvish" being mentioned, but I had no idea what it was. I guess I'll have to look into it, but so far I'm already impressed by "rsnapshot" by simply reading what it can do and how it operates. ;-) > Another option, which is a bit more efficient for the transfer and can > help avoid duplicates if you have occasional hiccups, is to use the > "--link-dest" option to provide the source of your links. This avoids > the extra "cp -al" step for greater efficiency, and also lets you > specify a number of old snapshots - helpful if some of these were > incomplete. > > Remember also that rsync is aimed at network transfers - you want to > keep your backups on a different machine (although it's nice with > local snapshots of /etc). Yep, as I said, part of the backups will be stored on a separate machine, and the rest locally - but on a different RAID array - on the same machine, and on removable media. The REV drive I have can handle 70 GB of data - according to Iomega, it can handle 140 GB with compression, but I suppose that they're talking of using their own compression algorithms and their Designed For Windows Only proprietary software. In my experience, it is impossible to make such claims as that compression would double the storage capacity, since a number of document formats are already compressed to begin with and you cannot compress them any further. > At the very least, keep them on a different file system partition than > the original - that way you have protection against file system > disasters. Yep. Like I said, the locally stored backups will be backups of the other RAID array, stored on the second array. > Obviously you want to avoid making any changes to the files in the > snapshots, though deleting them works perfectly (files disappear only > once all the hard links are gone). Yes, I'm aware of that. > It is also a good idea to hide the backup tree from locate/updatedb, > if you have it - your 100 daily snapshots may not take much more disk > space than a single copy, but it does take 100 times as many files and > directories. I am thinking about simply not having the backup volume mounted during normal operation and only have it mounted just prior to when the backup scripts start running. On the other hand, I could indeed also simply make sure that "updatedb" doesn't "see" the backup partition. >> My reason for using "zip" rather than "tar" for IRC logs is that my >> colleagues run Windoze and so their options are limited. ;-) > > Tell your Windoze colleagues to get a proper zipper program, instead > of relying on windows own half-hearted "zip folders" or illegal > unregistered copies of WinZip. 7zip is totally free, and vastly > better - amongst other features, it supports .tar.bz2 without > problems. I could also tell them to use GNU/Linux instead of Windoze - and it would make a great difference in the amount of spam I get from their /botnetted/ computers, but to them GNU/Linux is for servers and PCs or laptops must run Windoze. Or OS-X - two of them have a Mac as well. >>> It also needs to be saved in a safe and reliable place - many people >>> have had regular backups saved to tape only to find later that the >>> tapes were unreadable. >> >> That is always a risk, just as it was with the old music cassette >> tapes. Magnetic storage is actually not advised for backups. > > I agree - I dislike tapes for backup systems. But I also dislike > optical storage - disks are typically not big enough for complete > backups, so you have to have a really messy system with multiple disks > for a backup set, or even worse, incremental backup patch sets. Even > if you can fit everything on a single disk, it requires manual > intervention to handle the disks and store them safely off-site, and > you need to test them regularly (that's important!). The limitation in volume is one of the reasons why optical storage is not a good idea for backups, as it implies - as you said - human intervention to switch the media around. >> Well, just about everything in that machine is very expensive. And >> on the other hand, I did have another server here - which was >> malfunctioning but which has been repaired now - so I might as well >> put that one to use as a back-up machine in the event that my main >> machine would fail somehow - something which I am not looking forward >> to, of course! ;-) >> >> I also can't use the Xen live migration approach, because I intend to >> set up my main machine with 64-bit software, while the other server >> is a strictly 32-bit machine. But redundancy - i.e. a duplicate >> set-up of the main servers - should be adequate enough for my >> purposes. > > Can't Xen cope with mixes of 64-bit and 32-bit machines? I've never > used it - my servers use OpenVZ (no problem with mixes of 32-bit and > 64-bit virtual machines with a 64-bit host), and on desktops I use > Virtual Box (you can mix 32-bit and 64-bit hosts and guests in any > combination). Xen can deal with that, sure, but the problem is that the other physical machine only has 32-bit processors, while I intend to use 64-bit software for the virtual machines on my main computer. You can't migrate a running 64-bit virtual machine to a physical machine which only supports 32-bit. ;-) >>>>>>>> There is even a "live spare" solution, but to my knowledge this >>>>>>>> is specific to Adaptec - they call it RAID 5E. >>> >>> The array will be in degraded mode while the rebuild is being done, >>> just like if it were raid5 with a hot spare - and it will be equally >>> slow during the rebuild. So no points there. >> >> Well, it's not really something that - at least, in my impression - >> is advised as "a particular RAID solution", but rather as "a nice >> extension to RAID 5". > > I have to conclude it is more like "an inefficient and proprietary > extension to raid 5 that looked good on the marketing brochure - who > cares about reality?" :-) Nobody who has anything to sell. ;-) >>>> Many people already consider me certifiably insane for having spent >>>> that much money - 17'000 Euro, as I wrote higher up - on a >>>> privately owned computer system. But then again, for the intended >>>> purposes, I need fast and reliable hardware and a lot of >>>> horsepower. :-) >>> >>> I'm curious - what is the intended purpose? I think I would have a >>> hard job spending more than about three or four thousand Euros on a >>> single system. >> >> Well, okay, here goes... It's intended to be a kind of "mainframe" - >> which is what I call it on occasion when referring to that machine >> among the other machines I own. >> >> [...] >> Now, as for my intended purposes, I am going to set up this machine >> with Xen, as I have mentioned earlier. There will be three primary >> XenLinux virtual machines running on this system, all of which will >> be Gentoo installations. >> >> The three main virtual machines will be set up as follows: >> >> (1) The Xen dom0 virtual machine. [...] >> >> (2) A workstation virtual machine. >> [...] but it will also be a driver domain, i.e. it will have >> direct access to the GeForce, the soundchip and the USB hubs. >> It'll boot up to runlevel 3, but it'll have KDE 4.x installed, >> along with loads of applications. As it has direct access to >> the USB hubs, it'll also be running a CUPS server for my >> USB-connected multifunctional device, a Brother MFC-9880. It'll >> also be running an NFS server for multimedia files. >> >> (3) A server virtual machine which I intend to set up - if possible - >> with an OpenVZ kernel. [...] >> The OpenVZ system will be running several isolated userspaces - >> which are called "zones", just as in (Open)Solaris - one of which >> I intend to set up as the sole system from which /ssh/ login from >> the internet is allowed, and doing nothing else. The idea is >> that access to any other machine in the network - physical or >> virtual - must pass through this one virtual machine, making it >> harder for an eventual black hat to do any damage. Then, there >> will also be a generic "zone" for a DNS server and one, possibly >> two websites, and one, possibly two mailservers. Lastly, >> another "zone" will be running an IRC server and an IRC services >> package, possibly also with a few eggdrops. > > This sounds like a fun system! However, I would have split this into > two distinct machines - a server and a workstation. You are mixing > two very different types of use on a single machine, giving you > something that is bound to be a compromise (or a lot more expensive > than necessary). Not really a compromise, considering the horsepower of the machine. It does however start to sound a lot like a mainframe, and part of my reasons for wanting to do this is to create an exceptional set-up which has not been done before in this form[1] and which should be able to knock my Windoze-loving friends out of their socks.[2] ;-) [1] OpenVZ on Xen has already been done before, but not with Gentoo, to my knowledge. Using two keyboards and two videocards for separate virtual machines via Xen has also already been done, but never with Gentoo, and not in conjunction with an OpenVZ server running in one of the domUs. [2] "Can you do this with your Windows XP/Vista/7 too?" :pp > A good workstation will have a processor aimed at high peak > performance for single threads, with reasonable performance for up to > 4 threads (more if you do a lot of compiling or other highly parallel > tasks). Since Gentoo is sources-based, compiling is part of the picture. ;-) (I seem to remember that compiling/building a vanilla kernel on that machine (with GNU/Linux running on the bare metal, not in a virtual machine context) took about 50 seconds.) > Memory should be optimised for latency (this is more important than > fast bandwidth), [...] I think pc5300 works very well with these Opterons. > as should the disks (main disk(s) should be SSD, [... Ah, but with this I do not agree. SSDs, even the enterprise-grade ones - which are still far more expensive than a SAS disk of equal or even greater capacity - have a limited lifespan if they are being written to a lot, so anyone using SSDs should consider putting only read-only filesystems on them. > possibly with harddisks for bulk storage). You want good graphics and > sound. For software, you want your host OS to be the main working OS - > put guest OS's under Virtual Box if you want. Hmm... No, I again disagree. First of all, if the host operating system (or privileged guest in a Xen-context) is being used for daily work, then it becomes a liability to the guest. You want the host (or privileged guest) as static and lean as possible. It's the Achilles heel of your entire system. Secondly, VirtualBox might be very popular with a number of desktop users - and particularly so they could run Windows inside of it - but I do not consider that a proper virtualization context. VirtualBox runs on the host operating system as a process. It's not as efficient as Xen. And not quite as cool either. :p > For a server, your processor is aimed at high throughput on multiple > threads, and memory should be large - even if that means slow. Disks > should be large and reliable (raid). Graphics can be integrated on > the motherboard - you need a console keyboard and screen for the > initial installation, and they are disconnected afterwards (except > possibly for disaster recovery). In my case, as explained in my earlier post, the XenLinux privileged guest will have its own videocard and keyboard - there is no on-board video on this motherboard anyway - but the output of that videocard is connected to the second channel of one of the two monitors, and the other videocard - driven by the first domU - is connected to the first channel on both monitors. I can switch video output to the screen with the flick of a (mechanical) switch on the monitor. > These days you want your host OS to be minimal, and have the real work > done in virtual machines. Go for OpenVZ as much as possible - OpenVZ > machines are very light, and very fast to set up (on the server at > work, I can set up a new OpenVZ virtual machine in a couple of > minutes). Use Xen or KVM if you need more complete virtualisation. Well, it's going to be OpenVZ on top of Xen - if I can get my hands on the sources for the OpenVZ 2.6.27 kernel, because they only seem to supply .rpm packages and Gentoo's package manager can't handle those - for the server domU, and the rest will probably be running the most recent paravirtualized vanilla kernels. > Of course, when you already have the hardware, you use what you have. And it'll be a cool enough set-up the way I have it all planned. :-) >> [...] I have a simple Linksys WRT45GL router now with the standard >> firmware - which is Linux, by the way ;-) - and it'll do well enough >> to do port forwarding to the respective virtual machines. Additional >> firewalling can be done via /iptables/ on the respective virtual >> machines. > > I like to install OpenWRT on WRT54GL devices - it makes them far more > flexible than the original firmware. Of course, if the original > firmware does all you need, then that's fine. Yeah, I know quite a few people who ue OpenWRT, but then again, at present I see no reason why I would be putting myself through the trouble of flashing the firmware on that thing. ;-) The standard firmware is already quite good, mind you. ;-) >> Gentoo is far from ideal - given some issues over at the Gentoo >> Foundation itself and the fact that the developers seem mostly >> occupied with discussing how cool they think they are, rather than to >> actually do something sensible, and they've also started to implement >> a few defaults of which they themselves say that these are not the >> best choices but that they are the choices of which they think most >> users will opt for them - but at least the basic premise is still >> there, i.e. you do build it from sources, and as such you have more >> control over how the resulting system will be set-up, both in terms >> of hardware optimizations and in terms of software interoperability. > > Have you looked at Sabayon Linux? It's originally based on Gentoo, > but you might find the developer community more to your liking. I have only briefly looked into it. It appears to come as a Live DVD only, and it appears to favor Gnome, so applying all the USE flags to get it to support Qt3, Qt4 and KDE 4 might be a lot of work. I'll give it a closer look, though. ;-) -- *Aragorn* (registered GNU/Linux user #223157)
From: Aragorn on 22 Jan 2010 09:01 On Friday 22 January 2010 05:37 in comp.os.linux.misc, somebody identifying as Rahul wrote... > Aragorn <aragorn(a)chatfactory.invalid> wrote in > news:hj9ftp$7p4$1(a)news.eternal-september.org: > >> Quite honestly, I'm enjoying this thread, because I get to hear >> interesting feedback - and I think you do to, from your point of view >> - and I have a feeling that Rahul, the OP, is sitting there enjoying >> himself over all the valid arguments being discussed here in the >> debate over various RAID types. ;-) > > Absolutely. I am still lurking around. After all I have the blame as > being the OP who started us off on this interesting RAID discussion. It *is* interesting - I guess we've all learned something new here - and it is quite rare as well that a thread this deep hasn't strayed off-topic yet. ;-) > I'm moving away from tapes to disk-to-disk backups. In these days of > cheap disk it's starting to make so much more sense. On that I would agree. Tape cartridges are expensive, not as reliable for long-term storage, and they are vulnerable. Hard disks are a much more convenient choice in this day and age, and especially so the ones that can be hot-swapped or unplugged from the system - e.g. USB or eSATA. >> Well, what I personally find overkill in this is that he intends to >> use the entire array only for the "/home" filesystem. That seems >> like an awful waste of some great resources that I personally would >> put to use more efficiently - e.g. you could have the entire "/var" >> tree on it, and an additional "/srv" tree. > > Does it seem a waste when you think that the /home is being served out > via NFS to ~275 servers? Ahh, no, but you didn't tell us that, Grasshopper! :p > Those are the "compute nodes" doing most of the I/O so that's where I > need high-performance storage. Well, what I understood from your earlier mention was that it was intended to *be* a high-performance computing node. > Well, I already have a fast disk for /var etc. But those are merely > local to the server connected to the storage. This server is supposed > to do just one thing and do it well: Serve out the central storage via > NFS. Then it makes sense, indeed, and then the set-up that David and I both agreed on would indeed be the best for you, i.e. (to summarize) six disks per array set up as RAID 10 with the seventh disk being a spare, and /mdadm/ on top of the three arrays to combine them into a striped "/home". Should be quite a "bit blaster". :-) -- *Aragorn* (registered GNU/Linux user #223157)
From: Aragorn on 22 Jan 2010 09:06 On Friday 22 January 2010 15:01 in comp.os.linux.misc, somebody identifying as Aragorn wrote... > [...] i.e. (to summarize) six disks per array set up as RAID 10 with > the seventh disk being a spare, and /mdadm/ on top of the three arrays > to combine them into a striped "/home". Hmm... Seems like I've butterfingered again. That or I just had a brain fart. :p You've got 45 disks, distributed over three enclosures. That makes 15 disks per enclosure, so you could make that into a 14-disk RAID 10 array per enclosure with the 15th disk as a spare. Or a 12-disk RAID 10 and the three remaining disks as spares. ;-) -- *Aragorn* (registered GNU/Linux user #223157)
From: David Brown on 22 Jan 2010 09:56 On 22/01/2010 14:42, Aragorn wrote: > On Friday 22 January 2010 00:21 in comp.os.linux.misc, somebody > identifying as David Brown wrote... > >> Aragorn wrote: >> <snipping this bit to save space - I think we've covered the important stuff about rsync, unless anyone else has questions or comments> > >>> My reason for using "zip" rather than "tar" for IRC logs is that my >>> colleagues run Windoze and so their options are limited. ;-) >> >> Tell your Windoze colleagues to get a proper zipper program, instead >> of relying on windows own half-hearted "zip folders" or illegal >> unregistered copies of WinZip. 7zip is totally free, and vastly >> better - amongst other features, it supports .tar.bz2 without >> problems. > > I could also tell them to use GNU/Linux instead of Windoze - and it > would make a great difference in the amount of spam I get from > their /botnetted/ computers, but to them GNU/Linux is for servers and > PCs or laptops must run Windoze. Or OS-X - two of them have a Mac as > well. > I can understand it being difficult to persuade people to move from Windows, but it should be a lot easier to persuade typical windows users to install 7zip. Send them a file called "NakedPictures.7z" and a link to the 7zip download page - that should be sufficient argument. >>>> It also needs to be saved in a safe and reliable place - many people >>>> have had regular backups saved to tape only to find later that the >>>> tapes were unreadable. >>> >>> That is always a risk, just as it was with the old music cassette >>> tapes. Magnetic storage is actually not advised for backups. >> >> I agree - I dislike tapes for backup systems. But I also dislike >> optical storage - disks are typically not big enough for complete >> backups, so you have to have a really messy system with multiple disks >> for a backup set, or even worse, incremental backup patch sets. Even >> if you can fit everything on a single disk, it requires manual >> intervention to handle the disks and store them safely off-site, and >> you need to test them regularly (that's important!). > > The limitation in volume is one of the reasons why optical storage is > not a good idea for backups, as it implies - as you said - human > intervention to switch the media around. > >>> Well, just about everything in that machine is very expensive. And >>> on the other hand, I did have another server here - which was >>> malfunctioning but which has been repaired now - so I might as well >>> put that one to use as a back-up machine in the event that my main >>> machine would fail somehow - something which I am not looking forward >>> to, of course! ;-) >>> >>> I also can't use the Xen live migration approach, because I intend to >>> set up my main machine with 64-bit software, while the other server >>> is a strictly 32-bit machine. But redundancy - i.e. a duplicate >>> set-up of the main servers - should be adequate enough for my >>> purposes. >> >> Can't Xen cope with mixes of 64-bit and 32-bit machines? I've never >> used it - my servers use OpenVZ (no problem with mixes of 32-bit and >> 64-bit virtual machines with a 64-bit host), and on desktops I use >> Virtual Box (you can mix 32-bit and 64-bit hosts and guests in any >> combination). > > Xen can deal with that, sure, but the problem is that the other physical > machine only has 32-bit processors, while I intend to use 64-bit > software for the virtual machines on my main computer. You can't > migrate a running 64-bit virtual machine to a physical machine which > only supports 32-bit. ;-) > OK, running 64-bit virtual machines on a 32-bit processor is tough (QEMU can do it, if you are happy to wait long enough!). >>>>>>>>> There is even a "live spare" solution, but to my knowledge this >>>>>>>>> is specific to Adaptec - they call it RAID 5E. >>>> >>>> The array will be in degraded mode while the rebuild is being done, >>>> just like if it were raid5 with a hot spare - and it will be equally >>>> slow during the rebuild. So no points there. >>> >>> Well, it's not really something that - at least, in my impression - >>> is advised as "a particular RAID solution", but rather as "a nice >>> extension to RAID 5". >> >> I have to conclude it is more like "an inefficient and proprietary >> extension to raid 5 that looked good on the marketing brochure - who >> cares about reality?" :-) > > Nobody who has anything to sell. ;-) > >>>>> Many people already consider me certifiably insane for having spent >>>>> that much money - 17'000 Euro, as I wrote higher up - on a >>>>> privately owned computer system. But then again, for the intended >>>>> purposes, I need fast and reliable hardware and a lot of >>>>> horsepower. :-) >>>> >>>> I'm curious - what is the intended purpose? I think I would have a >>>> hard job spending more than about three or four thousand Euros on a >>>> single system. >>> >>> Well, okay, here goes... It's intended to be a kind of "mainframe" - >>> which is what I call it on occasion when referring to that machine >>> among the other machines I own. >>> >>> [...] >>> Now, as for my intended purposes, I am going to set up this machine >>> with Xen, as I have mentioned earlier. There will be three primary >>> XenLinux virtual machines running on this system, all of which will >>> be Gentoo installations. >>> >>> The three main virtual machines will be set up as follows: >>> >>> (1) The Xen dom0 virtual machine. [...] >>> >>> (2) A workstation virtual machine. >>> [...] but it will also be a driver domain, i.e. it will have >>> direct access to the GeForce, the soundchip and the USB hubs. >>> It'll boot up to runlevel 3, but it'll have KDE 4.x installed, >>> along with loads of applications. As it has direct access to >>> the USB hubs, it'll also be running a CUPS server for my >>> USB-connected multifunctional device, a Brother MFC-9880. It'll >>> also be running an NFS server for multimedia files. >>> >>> (3) A server virtual machine which I intend to set up - if possible - >>> with an OpenVZ kernel. [...] >>> The OpenVZ system will be running several isolated userspaces - >>> which are called "zones", just as in (Open)Solaris - one of which >>> I intend to set up as the sole system from which /ssh/ login from >>> the internet is allowed, and doing nothing else. The idea is >>> that access to any other machine in the network - physical or >>> virtual - must pass through this one virtual machine, making it >>> harder for an eventual black hat to do any damage. Then, there >>> will also be a generic "zone" for a DNS server and one, possibly >>> two websites, and one, possibly two mailservers. Lastly, >>> another "zone" will be running an IRC server and an IRC services >>> package, possibly also with a few eggdrops. >> >> This sounds like a fun system! However, I would have split this into >> two distinct machines - a server and a workstation. You are mixing >> two very different types of use on a single machine, giving you >> something that is bound to be a compromise (or a lot more expensive >> than necessary). > > Not really a compromise, considering the horsepower of the machine. It > does however start to sound a lot like a mainframe, and part of my > reasons for wanting to do this is to create an exceptional set-up which > has not been done before in this form[1] and which should be able to > knock my Windoze-loving friends out of their socks.[2] ;-) > > [1] OpenVZ on Xen has already been done before, but not with Gentoo, to > my knowledge. Using two keyboards and two videocards for separate > virtual machines via Xen has also already been done, but never with > Gentoo, and not in conjunction with an OpenVZ server running in one > of the domUs. > > [2] "Can you do this with your Windows XP/Vista/7 too?" :pp > >> A good workstation will have a processor aimed at high peak >> performance for single threads, with reasonable performance for up to >> 4 threads (more if you do a lot of compiling or other highly parallel >> tasks). > > Since Gentoo is sources-based, compiling is part of the picture. ;-) > > (I seem to remember that compiling/building a vanilla kernel on that > machine (with GNU/Linux running on the bare metal, not in a virtual > machine context) took about 50 seconds.) > That's a fast kernel compile! Are you cheating and using ccache? >> Memory should be optimised for latency (this is more important than >> fast bandwidth), [...] > > I think pc5300 works very well with these Opterons. > >> as should the disks (main disk(s) should be SSD, [... > > Ah, but with this I do not agree. SSDs, even the enterprise-grade > ones - which are still far more expensive than a SAS disk of equal or > even greater capacity - have a limited lifespan if they are being > written to a lot, so anyone using SSDs should consider putting only > read-only filesystems on them. > SSDs are certainly more expensive for their size (by a factor of about 30 or so, I think). But you can get a 64 GB SSD for a very reasonable price, and it is more than sufficient for your OS, applications (except perhaps large games), and main data files. Bulk data, such as video files, will need to go on a hard disk. But everything else can typically go on the SSD. If you are only using 30 GB for your system, who cares if you have 30 GB free or 970 GB free? An SSD is not sufficient on its own for a typical workstation, but it can form an excellent base. The "limited lifespan" of SSDs is mostly a myth, and partly outdated information. With a modern SLC drive, you can write to them continually at full speed for /decades/ before you would expect to suffer from wear. Reliability is orders of magnitude better than for hard disks. >> possibly with harddisks for bulk storage). You want good graphics and >> sound. For software, you want your host OS to be the main working OS - >> put guest OS's under Virtual Box if you want. > > Hmm... No, I again disagree. First of all, if the host operating > system (or privileged guest in a Xen-context) is being used for daily > work, then it becomes a liability to the guest. You want the host (or > privileged guest) as static and lean as possible. It's the Achilles > heel of your entire system. > That's true for a server - not for a workstation. You are correct that the guest is no safer (or more reliable) than the host, but for a workstation, that's fine. You use the host for your main work, as efficiently as possible (i.e., direct access to hardware, etc.). If you have something risky (testing out software, running windows), you do it in a guest without risking the host. But if you have something that has to be safer and more reliable than the system you are using as your main workhorse every day, it should be on a different physical system - the server. > Secondly, VirtualBox might be very popular with a number of desktop > users - and particularly so they could run Windows inside of it - but I > do not consider that a proper virtualization context. VirtualBox runs > on the host operating system as a process. It's not as efficient as > Xen. And not quite as cool either. :p > It is a somewhat different concept from Xen - you are correct that it runs as a process on the host. But you are incorrect to write that off as a disadvantage or "not proper virtualization". It is a different type of virtualisation, with advantages as well as disadvantages. It is easier in use, and easier to integrate (sharing files and clipboards, moving smoothly between guest and host, mixing host and guest windows, etc.). But it doesn't allow the guest the same level of controlled hardware access that a hypervisor solution like Xen gives you. That's why I recommend VBox for workstations, but it is probably not the best choice for a server. Different tools for different jobs. >> For a server, your processor is aimed at high throughput on multiple >> threads, and memory should be large - even if that means slow. Disks >> should be large and reliable (raid). Graphics can be integrated on >> the motherboard - you need a console keyboard and screen for the >> initial installation, and they are disconnected afterwards (except >> possibly for disaster recovery). > > In my case, as explained in my earlier post, the XenLinux privileged > guest will have its own videocard and keyboard - there is no on-board > video on this motherboard anyway - but the output of that videocard is > connected to the second channel of one of the two monitors, and the > other videocard - driven by the first domU - is connected to the first > channel on both monitors. I can switch video output to the screen with > the flick of a (mechanical) switch on the monitor. > >> These days you want your host OS to be minimal, and have the real work >> done in virtual machines. Go for OpenVZ as much as possible - OpenVZ >> machines are very light, and very fast to set up (on the server at >> work, I can set up a new OpenVZ virtual machine in a couple of >> minutes). Use Xen or KVM if you need more complete virtualisation. > > Well, it's going to be OpenVZ on top of Xen - if I can get my hands on > the sources for the OpenVZ 2.6.27 kernel, because they only seem to > supply .rpm packages and Gentoo's package manager can't handle those - > for the server domU, and the rest will probably be running the most > recent paravirtualized vanilla kernels. > >> Of course, when you already have the hardware, you use what you have. > > And it'll be a cool enough set-up the way I have it all planned. :-) > >>> [...] I have a simple Linksys WRT45GL router now with the standard >>> firmware - which is Linux, by the way ;-) - and it'll do well enough >>> to do port forwarding to the respective virtual machines. Additional >>> firewalling can be done via /iptables/ on the respective virtual >>> machines. >> >> I like to install OpenWRT on WRT54GL devices - it makes them far more >> flexible than the original firmware. Of course, if the original >> firmware does all you need, then that's fine. > > Yeah, I know quite a few people who ue OpenWRT, but then again, at > present I see no reason why I would be putting myself through the > trouble of flashing the firmware on that thing. ;-) The standard > firmware is already quite good, mind you. ;-) > If it ain't broke, don't fix it? But it's more fun to break it, then see if you can fix it again afterwards... >>> Gentoo is far from ideal - given some issues over at the Gentoo >>> Foundation itself and the fact that the developers seem mostly >>> occupied with discussing how cool they think they are, rather than to >>> actually do something sensible, and they've also started to implement >>> a few defaults of which they themselves say that these are not the >>> best choices but that they are the choices of which they think most >>> users will opt for them - but at least the basic premise is still >>> there, i.e. you do build it from sources, and as such you have more >>> control over how the resulting system will be set-up, both in terms >>> of hardware optimizations and in terms of software interoperability. >> >> Have you looked at Sabayon Linux? It's originally based on Gentoo, >> but you might find the developer community more to your liking. > > I have only briefly looked into it. It appears to come as a Live DVD > only, and it appears to favor Gnome, so applying all the USE flags to > get it to support Qt3, Qt4 and KDE 4 might be a lot of work. I'll give > it a closer look, though. ;-) >
From: Aragorn on 22 Jan 2010 10:41
On Friday 22 January 2010 15:56 in comp.os.linux.misc, somebody identifying as David Brown wrote... > On 22/01/2010 14:42, Aragorn wrote: > >> On Friday 22 January 2010 00:21 in comp.os.linux.misc, somebody >> identifying as David Brown wrote... >> >>> Can't Xen cope with mixes of 64-bit and 32-bit machines? I've never >>> used it - my servers use OpenVZ (no problem with mixes of 32-bit and >>> 64-bit virtual machines with a 64-bit host), and on desktops I use >>> Virtual Box (you can mix 32-bit and 64-bit hosts and guests in any >>> combination). >> >> Xen can deal with that, sure, but the problem is that the other >> physical machine only has 32-bit processors, while I intend to use >> 64-bit software for the virtual machines on my main computer. You >> can't migrate a running 64-bit virtual machine to a physical machine >> which only supports 32-bit. ;-) > > OK, running 64-bit virtual machines on a 32-bit processor is tough > (QEMU can do it, if you are happy to wait long enough!). The last time I was over at the home of my now ex-colleague, he had a 64-bit OpenSuSE running on a 32-bit host operating system - which I believe to have been Windows XP - using VMWare. There was an incredible discrepancy in the time as shown on the clock in each system's panel. I don't remember whether the skewed clock was running fast or slow, but it differed by about half an hour every five minutes. It was insane. He had to install an ntpd in the OpenSuSE guest, because it refused to connect to the ntpd running on the host - probably a firewalling issue, although I seem to remember that he left that port open. Of course, that's VMWare. And Windows. So it's not anything I care enough about to troubleshoot it. :p >>> A good workstation will have a processor aimed at high peak >>> performance for single threads, with reasonable performance for up >>> to 4 threads (more if you do a lot of compiling or other highly >>> parallel tasks). >> >> Since Gentoo is sources-based, compiling is part of the picture. ;-) >> >> (I seem to remember that compiling/building a vanilla kernel on that >> machine (with GNU/Linux running on the bare metal, not in a virtual >> machine context) took about 50 seconds.) > > That's a fast kernel compile! Are you cheating and using ccache? Nope. Only did a "make -j5". It's a ccNUMA system, with two dualcore Opterons, running at 2.6 GHz. It *is* fast indeed! I remember that using the same make options, my dual Xeon machine - 32-bit, 2.2 GHz, hyperthreading enabled, 400 MHz FSB, single memory controller on the motherboard - needed about 5 minutes. With the default "-j2" it took about 40 to 45 minutes, depending on the kernel generation. The first time I ever compiled a kernel from sources was the vanilla 2.6.5 kernel, and on that dual Xeon machine. At that time, I still hadn't figured out that you could do parallel compiles using "-j" and a number. The last kernel I've compiled on it was 2.6.17.something, with the "-j5", and that was the one that took (little over) 5 minutes to build. The one I built on the new machine in 50 seconds was the 2.6.22 kernel with Xen patches for dom0. Although this was in Gentoo, the 2.6.22 XenLinux kernels came from Ubuntu (but supplied by Gentoo). There were also 2.6.20 XenLinux-patched kernels from RedHat, and the Xen.org-supplied 2.6.18 kernels. These days, those are all obsolete, of course, since as of 2.6.30, vanilla Linux has all the code for running as dom0, domU or on the bare metal, and you can have all three usages inside a single binary kernel. The kernel will decide whether it is running on the bare metal or paravirtualized on Xen (or KVM), and whether it's a privileged or unprivileged Xen guest. Of course, for a custom system like mine, building specialized kernels for dom0 and domU is more preferable than a one-size-fits-all kernel image. >>> [...] possibly with harddisks for bulk storage). You want good >>> graphics and sound. For software, you want your host OS to be the >>> main working OS - put guest OS's under Virtual Box if you want. >> >> Hmm... No, I again disagree. First of all, if the host operating >> system (or privileged guest in a Xen-context) is being used for daily >> work, then it becomes a liability to the guest. You want the host >> (or privileged guest) as static and lean as possible. It's the >> Achilles heel of your entire system. > > That's true for a server - not for a workstation. You are correct > that the guest is no safer (or more reliable) than the host, but for a > workstation, that's fine. Well, only if you consider the guest to be something that needs to be sandboxed. And of course, Windows *should* be. And sealed in behind solid concrete walls by people wearing Hazmat suits. :p > You use the host for your main work, as efficiently as possible (i.e., > direct access to hardware, etc.). If you have something risky > (testing out software, running windows), you do it in a guest without > risking the host. Yes, the sandbox scenario. For that kind of usage, host-based virtual machine monitors are a good choice, of course. > But if you have something that has to be safer and more reliable than > the system you are using as your main workhorse every day, it should > be on a different physical system - the server. Well, with hardware like mine and with something as powerful as Xen, I consider my usage of that system quite efficient and more interesting. You also have to keep in mind here that I view a GNU/Linux (or any UNIX) workstation as something other than "a PC". To me, any UNIX system is a client/server architecture and I treat and view it as such. It's a different philosophy from the single-user thinking that is typical for Windows users. And thus, once again, it's all in the eye of the beholder. ;-) >> Secondly, VirtualBox might be very popular with a number of desktop >> users - and particularly so they could run Windows inside of it - but >> I do not consider that a proper virtualization context. VirtualBox >> runs on the host operating system as a process. It's not as >> efficient as Xen. And not quite as cool either. :p > > It is a somewhat different concept from Xen - you are correct that it > runs as a process on the host. But you are incorrect to write that > off as a disadvantage or "not proper virtualization". Well, it obviously *is* virtualization, but as I wrote higher up, I consider that good for sandboxing, but not for my purposes. I like the mainframe-style approach of Xen, where everything is a virtual machine, including the management system. It's a matter of taste, really, so when I said "I don't consider that" and so on, I was really speaking of me, myself and I. ;-) > It is a different type of virtualisation, with advantages as well as > disadvantages. It is easier in use, and easier to integrate (sharing > files and clipboards, moving smoothly between guest and host, mixing > host and guest windows, etc.). I have seen screenshots of a virtualization solution - don't ask me which one because I don't remember - where GNU/Linux and Windows were actually *sharing* the desktop. Neither was running in any windowed context and one didn't have to switch full screen mode between the host (GNU/Linux) and the guest (Windows). It was all seamless, with X desktop's taskbar at the top and the Windows taskbar at the bottom of the screen. It was weird. 8-) > But it doesn't allow the guest the same level of controlled > hardware access that a hypervisor solution like Xen gives you. That's > why I recommend VBox for workstations, but it is probably not the best > choice for a server. Different tools for different jobs. And differing opinions on what constitutes "a workstation". :-) >>> I like to install OpenWRT on WRT54GL devices - it makes them far >>> more >>> flexible than the original firmware. Of course, if the original >>> firmware does all you need, then that's fine. >> >> Yeah, I know quite a few people who ue OpenWRT, but then again, at >> present I see no reason why I would be putting myself through the >> trouble of flashing the firmware on that thing. ;-) The standard >> firmware is already quite good, mind you. ;-) > > If it ain't broke, don't fix it? Exactly! :-) > But it's more fun to break it, then see if you can fix it again > afterwards... And if you can't, then you've got yourself an expensive paperweight. :p -- *Aragorn* (registered GNU/Linux user #223157) |