Prev: [UNICOLOMBIA] Matriculas abiertas
Next: tool tips remaining after window switching
From: Chris Davies on 27 Apr 2010 18:30
Boyd Stephen Smith Jr. <bss(a)iguanasuicide.net> wrote:
> For "normal" file operations, taking an LVM snapshot of the mounted
> filesystem and then making your backup from that should be sufficient.

Agreed. I was pointing out that rsync isn't necessarily the appropriate
solution in all cases.

Chris


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/to1ka7xhdm.ln2(a)news.roaima.co.uk
From: Ron Johnson on 29 Apr 2010 07:30
On 04/27/2010 02:51 PM, Boyd Stephen Smith Jr. wrote:
[snip]
>
> For "normal" file operations, taking an LVM snapshot of the mounted filesystem
> and then making your backup from that should be sufficient. This should even
> work for postgreSQL database files (though, it is not optimal). MySQL has a
> history of being more flaky, but it might work there as well.
>
> If you snapshot a mounted file system, the snapshot will be approximately
> equivalent to the original file system, uncleanly unmounted at that exact
> moment (think: power failure). It's possible to then take backups of an
> active system with no downtime (although I/O load will certainly go up during
> the backup). If you mount the snapshot as part of the backup procedure, a
> journaled file system will want to replay the journal then. Otherwise, a
> journal replay will be required at restore time. PostgreSQL (etc.) will also
> end up doing a journal replay / recovery at restore time.
>

A proper RDBMS will have a hot-backup feature, so I'd still say that
a file-only backup is the way to go.

--
Dissent is patriotic, remember?


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/4BD96C86.3060301(a)cox.net
From: Boyd Stephen Smith Jr. on 29 Apr 2010 10:40
On Thursday 29 April 2010 06:24:54 Ron Johnson wrote:
> On 04/27/2010 02:51 PM, Boyd Stephen Smith Jr. wrote:
> > For "normal" file operations, taking an LVM snapshot of the mounted
> > filesystem and then making your backup from that should be sufficient.
> > This should even work for postgreSQL database files (though, it is not
> > optimal). MySQL has a history of being more flaky, but it might work
> > there as well.
> >
> > If you snapshot a mounted file system, the snapshot will be approximately
> > equivalent to the original file system, uncleanly unmounted at that exact
> > moment (think: power failure). It's possible to then take backups of an
> > active system with no downtime (although I/O load will certainly go up
> > during the backup). If you mount the snapshot as part of the backup
> > procedure, a journaled file system will want to replay the journal then.
> > Otherwise, a journal replay will be required at restore time. PostgreSQL
> > (etc.) will also end up doing a journal replay / recovery at restore
> > time.
>
> A proper RDBMS will have a hot-backup feature, so I'd still say that
> a file-only backup is the way to go.

Just because something doesn't advertise itself as an RDBMS doesn't mean that
it won't be in the middle of doing something "odd"[1] to a file. You should,
if at all possible, quiesce the filesystem before you back it up, or at least
quiesce the individual files prior to back up.

cp -a and rsync will not do either by themselves. Most of what my message
said about doing a unmount, snapshot, mount dance would also apply to a
remount r/o, snapshot, remount r/w dance and it may be more practical for some
file systems.

Both MySQL and PostgreSQL have "hot" backup procedures that do not require
taking down the database daemon. Having PostgreSQL relocate its journal files
rather than delete them can result in a "live" backup, that is literally only
a few dozen write-queries behind the production server, at most.

[1] I should really be more specific instead of saying "odd" and "normal".
"Normal" means writing to the end of a file, truncating a file at any offset,
reading from any part of a file. "Odd" means (over) writing at a position
that is not the end of the file. RDBMS are more likely to do the last
(updating fixed-length records, or the fixed-length parts of records), but
they are not the only application that might have a good reason for doing it
that way.
--
Boyd Stephen Smith Jr. ,= ,-_-. =.
bss(a)iguanasuicide.net ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-'
http://iguanasuicide.net/ \_/
From: Bernard on 9 May 2010 12:20
Chris Davies wrote:
> Bernard <bdebreil(a)teaser.fr> wrote:
>
>> On my previous systems, I used to create overall backups of partitions,
>> using 'SystemRescueCD'. This is no longer possible ever since 2007 that
>> I have a RAID1 mirroring system on my Debian.
>>
>
> You used to boot from SystemRescueCD and backup the partitions
> corresponding to the (unmounted) filesystems block by block? Or you
> mounted them and backed up the (mounted) filesystems?
>

I've never dared backing up mounted partitions, for fearing of mishaps


> You can continue to do either of these that even though you've got RAID1
> and LVM. Start up RAID1 and LVM, and then back up the logical partitions
> (or filesystems) that result. Ignore the physical ones.
>
>
>
>> SystemRescueCD does not backup LVM volumes, or, if it does, that must
>> be a very new feature, and I have not found any doc on this.
>>
>
> TBH I'm not aware of any backup feature within SystemRescueCD, but there
> are many useful tools there. I've successfully fixed up software RAID1
> configurations, and it must work with LVM because most of my systems'
> filesystems are based on RAID1/LVM combinations and it works for me.
>

SystemRescueCD only worked as an external boot system. Amongst the tools
it proposes, is 'partimage', which creates images of partitions and
compresses them. When I stated that 'SystemRescueCD' did not backup LVM
volumes, I meant that 'partimage' lacked this capability. As far as I
know, recent versions cannot do any better.
>
>> So, what should I do so as to backup my system ? I once tried 'dd', but
>> I think I remember it took forever [...]
>>
>
> Without the bs argument you would have been reading the disk 512bytes
> at a time. Try bs=1000k for better throughput.
>
I tried 'dd' again. It seems more promising, since it does backup
everything. files are huge (about same size as the original partition,
including empty space), but, in the end, it barely takes more than one
hour for each of my 64 GB partitions. (By the way, I kind of recall
having stated somewhere that my partitions were 200 GB in size, but I
was wrong. I have two internal hard drives of 80 GB each). My
partitions are :

/dev/sda1 and /dev/sdb1 => assembled onto a RAID 1 array as /dev/md0
(boot partition) (about half of a GB)

/dev/sda2 and /dev/sdb2 => assembled onto a RAID 1 array as /dev/md1 and
also as /dev/mapper/vg00-root (about 64 GB each)

/dev/sda3 => fat32 partition, with MSWIN XP installed on it (10 GB)
>
>> I don't really know whether I would have been able to use the resulting
>> file to successfully restore the partitions if needed.
>>
>
> How do you know any of your backups work? Test them!
>

Sure ! Not so easy though ! To carry such test, you ought to have
enough space available, unless you carry mini tests on small partitions.

This time, I think I have what I need. I purchased yet another external
HD, with a capacity of 500 GB, and, after I had erased whatever it
contained for use under MSWIN, I created four partitions using 'cfdisk',
all Linux. Then I formated them to ext2 using 'mkfs -t ext2..' (mkfs -t
ext2 /dev/sdc1, mkfs -t ext2 /dev/sdc2, .. ,mkfs -t ext2 /dev/sdc4)
Next, I launched 'dd' as follows :

/mnt/sdc4/# dd if=/dev/sda1 of=sda1_04_10.img

Don't know if this was the best way to do, in any case I did just that.
It only took a few minutes, since that partition is only half of a GB
long. Then I attempted to restore the saved partition onto another
partition of my external HD:

/mnt/sdc4/# dd if=sda1_04_10.img of=/dev/sdc2

This being done, and after unmounting/re-mounting the devices, I found
that /dev/sdc2 was indeed a copy of /dev/sda1...

BUT...

I must say that my original /dev/sda1 was my 'boot' partition. As such,
the RAID/mdadm programs did not create a LVM system ; it kept using the
original ext3.

Once this was achieved, I carried on with another partition (sda2). This
one was about 64 GB in size.

/mnt/sdc4/# dd if=/dev/sda2 of=sda2_04_10.img

Next morning (but I figured that the process only took a little more
than one hour), I attempted to restore said partition into /dev/sdc4

/mnt/sdc4/# dd if=sda2_04_10.img of=/dev/sdc1

the result of restoring the partition sda2 onto the sdc1 partition of my
external disk, was not the same as that obtained in restoring sda1 onto
sdc2. I can't read, or even mount sdc1 :

root(a)new-host:/home/bd# mount /dev/sdc1 /mnt/exthd
mount: unknown filesystem type 'lvm2pv'
root(a)new-host:/home/bd#

So, my hard disk partition sda2 has been restored from its image, onto a
lvm2pv logical partition.

Now, my question is : what can I do with this partition so as to
restore something useful ? In case something crashes on my desktop, I
imagine that, after replacing one or both hard disks, I would just have
to partition and then use 'dd' to restore both sda1 (as ext3 RAID array)
and sda2 (as LVM RAID array) and that the rest would follow
automatically upon reboot ? Do you confirm this ?

But, as a matter of testing, what can I do ? Using 'mdadm', can I
re-create a RAID array /dev/md1 or/and /dev/mapper/vg00-root ?
How shall I do that, supposing that I have obtained two similar
partitions (sda2 and sdb2 imaged and restored on two partitions of my
external HD ? I attempted to manage with the 'mdadm' man pages, but
this is not very obvious to me

>
>
>


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/4BE6DEC1.60704(a)teaser.fr
From: Alex Samad on 9 May 2010 17:10
On Sun, 2010-05-09 at 18:11 +0200, Bernard wrote:
> Chris Davies wrote:
> > Bernard <bdebreil(a)teaser.fr> wrote:
[snip]

> >
> >> So, what should I do so as to backup my system ? I once tried 'dd', but
> >> I think I remember it took forever [...]
> >>
> >
> > Without the bs argument you would have been reading the disk 512bytes
> > at a time. Try bs=1000k for better throughput.
> >
> I tried 'dd' again. It seems more promising, since it does backup

dd is not a backup tool for running systems

> everything. files are huge (about same size as the original partition,
> including empty space), but, in the end, it barely takes more than one
> hour for each of my 64 GB partitions. (By the way, I kind of recall
> having stated somewhere that my partitions were 200 GB in size, but I
> was wrong. I have two internal hard drives of 80 GB each). My
> partitions are :
>
> /dev/sda1 and /dev/sdb1 => assembled onto a RAID 1 array as /dev/md0
> (boot partition) (about half of a GB)
>
> /dev/sda2 and /dev/sdb2 => assembled onto a RAID 1 array as /dev/md1 and
> also as /dev/mapper/vg00-root (about 64 GB each)
>
> /dev/sda3 => fat32 partition, with MSWIN XP installed on it (10 GB)
> >
> >> I don't really know whether I would have been able to use the resulting
> >> file to successfully restore the partitions if needed.
> >>
> >
> > How do you know any of your backups work? Test them!
> >
>
> Sure ! Not so easy though ! To carry such test, you ought to have
> enough space available, unless you carry mini tests on small partitions.
>
> This time, I think I have what I need. I purchased yet another external
> HD, with a capacity of 500 GB, and, after I had erased whatever it
> contained for use under MSWIN, I created four partitions using 'cfdisk',
> all Linux. Then I formated them to ext2 using 'mkfs -t ext2..' (mkfs -t
> ext2 /dev/sdc1, mkfs -t ext2 /dev/sdc2, .. ,mkfs -t ext2 /dev/sdc4)

What is the aim of formatting the disk this way. Why not have a single
partition and backup into there, into some sort of directory structure
like <mntpoint>/YYYYMMDD/

> Next, I launched 'dd' as follows :
>
> /mnt/sdc4/# dd if=/dev/sda1 of=sda1_04_10.img
>
> Don't know if this was the best way to do, in any case I did just that.
> It only took a few minutes, since that partition is only half of a GB
> long. Then I attempted to restore the saved partition onto another
> partition of my external HD:
>
> /mnt/sdc4/# dd if=sda1_04_10.img of=/dev/sdc2
>
> This being done, and after unmounting/re-mounting the devices, I found
> that /dev/sdc2 was indeed a copy of /dev/sda1...
>
> BUT...
>
> I must say that my original /dev/sda1 was my 'boot' partition. As such,
> the RAID/mdadm programs did not create a LVM system ; it kept using the
> original ext3.

great you used DD to backup a filesystem that isn't/doesn't change.
Waste of space using DD

>
> Once this was achieved, I carried on with another partition (sda2). This
> one was about 64 GB in size.
>
> /mnt/sdc4/# dd if=/dev/sda2 of=sda2_04_10.img
>
> Next morning (but I figured that the process only took a little more
> than one hour), I attempted to restore said partition into /dev/sdc4
>
> /mnt/sdc4/# dd if=sda2_04_10.img of=/dev/sdc1
>
> the result of restoring the partition sda2 onto the sdc1 partition of my
> external disk, was not the same as that obtained in restoring sda1 onto
> sdc2. I can't read, or even mount sdc1 :

I wonder why.... in 1 hour the filesystem would have been used so the
initial part of the backup would have been done whilst the fs was in one
state and the last part of the backup whilst it was in a different
state.


>
> root(a)new-host:/home/bd# mount /dev/sdc1 /mnt/exthd
> mount: unknown filesystem type 'lvm2pv'
> root(a)new-host:/home/bd#
>
> So, my hard disk partition sda2 has been restored from its image, onto a
> lvm2pv logical partition.
>
> Now, my question is : what can I do with this partition so as to
> restore something useful ? In case something crashes on my desktop, I
> imagine that, after replacing one or both hard disks, I would just have
> to partition and then use 'dd' to restore both sda1 (as ext3 RAID array)
> and sda2 (as LVM RAID array) and that the rest would follow
> automatically upon reboot ? Do you confirm this ?

if you really must use dd which is not a backup utility. you have to
stop all the processes and mount all the filesystems as read only.

in fact install another os on the 500G drive, that just does the backup
of the original system to the 500G drive.

maybe have a look at http://www.mondorescue.org/

>
> But, as a matter of testing, what can I do ? Using 'mdadm', can I
> re-create a RAID array /dev/md1 or/and /dev/mapper/vg00-root ?
> How shall I do that, supposing that I have obtained two similar
> partitions (sda2 and sdb2 imaged and restored on two partitions of my
> external HD ? I attempted to manage with the 'mdadm' man pages, but
> this is not very obvious to me

I accept the fact that if I loose my system I am going to have to
partially build a new system, install the base system.

I have kept copies of my apt/dpkg db for restoration if need and I use
rsync to backup all of my files (except for obvious things like /dev),
this backups to a compressed fuse fs so I get compression as well,
incremental backups.

another tool to look at is tar

Alex
>
> >
> >
> >
>
>



--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/1273439063.18606.10.camel(a)alex-mini.samad.com.au
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: [UNICOLOMBIA] Matriculas abiertas
Next: tool tips remaining after window switching