Backups: Difference between revisions
No edit summary |
m Text replacement - "<(\/?)source" to "<$1syntaxhighlight" |
||
| (3 intermediate revisions by one other user not shown) | |||
| Line 7: | Line 7: | ||
<li><strong>File</strong>: "Full" file system archive to online storage before re-building a host (completely reformatting disks and partitions)</li> | <li><strong>File</strong>: "Full" file system archive to online storage before re-building a host (completely reformatting disks and partitions)</li> | ||
<li><strong>System Image</strong>: A complete operating system image to allow either [[cloning]] to new hardware, or full system restoration</li> | <li><strong>System Image</strong>: A complete operating system image to allow either [[cloning]] to new hardware, or full system restoration</li> | ||
<li><strong>Database</strong>: See [[Mysqldump]] | |||
</ol> | </ol> | ||
== Example Use Cases == | == Example Use Cases == | ||
| Line 72: | Line 73: | ||
|text = the file <code>/etc/fstab</code> sets up your file system devices and has started using universally unique identifiers to avoid problems with pluggable external disks.}} | |text = the file <code>/etc/fstab</code> sets up your file system devices and has started using universally unique identifiers to avoid problems with pluggable external disks.}} | ||
You can learn the identifier with the command: < | You can learn the identifier with the command: <syntaxhighlight lang="bash">sudo vol_id -u /dev/sda1</syntaxhighlight> | ||
But it turns out there is a simpler way of finding out what the volume id is for a disk drive: | But it turns out there is a simpler way of finding out what the volume id is for a disk drive: | ||
< | <syntaxhighlight lang="bash"> | ||
ls -l /dev/disk/by-uuid/ | ls -l /dev/disk/by-uuid/ | ||
total 0 | total 0 | ||
| Line 81: | Line 82: | ||
lrwxrwxrwx 1 root root 10 2008-10-21 09:32 8e46d1ff-5f34-46b1-a51a-0dac169123b7 -> ../../sdb1 | lrwxrwxrwx 1 root root 10 2008-10-21 09:32 8e46d1ff-5f34-46b1-a51a-0dac169123b7 -> ../../sdb1 | ||
lrwxrwxrwx 1 root root 10 2008-10-21 09:32 c82c1eb4-439c-4982-8764-ac207d4f9622 -> ../../sda1 | lrwxrwxrwx 1 root root 10 2008-10-21 09:32 c82c1eb4-439c-4982-8764-ac207d4f9622 -> ../../sda1 | ||
</ | </syntaxhighlight | ||
< | <syntaxhighlight lang="bash"> | ||
cat /proc/filesystems | cat /proc/filesystems | ||
</ | </syntaxhighlight> | ||
shows you what file system types are supported under your currently running kernel | shows you what file system types are supported under your currently running kernel | ||
| Line 93: | Line 94: | ||
* /sys/ | * /sys/ | ||
< | <syntaxhighlight lang="bash"> | ||
sudo rsync -ravlHz --progress --stats --exclude=/mnt/usbdrive/* --exclude=tmp*** --exclude=/proc/** --exclude=/sys/** --cvs-exclude --dry-run / /mnt/usbdrive/backups/liberty | sudo rsync -ravlHz --progress --stats --exclude=/mnt/usbdrive/* --exclude=tmp*** --exclude=/proc/** --exclude=/sys/** --cvs-exclude --dry-run / /mnt/usbdrive/backups/liberty | ||
</ | </syntaxhighlight> | ||
I added the --dry-run option in there because you should always test first, and because I do not want anyone blindly copy and pasting this command without testing and tweaking it. | I added the --dry-run option in there because you should always test first, and because I do not want anyone blindly copy and pasting this command without testing and tweaking it. | ||
| Line 104: | Line 105: | ||
This is the command that I used to create a full system backup of my laptop hard drive to my external USB drive: | This is the command that I used to create a full system backup of my laptop hard drive to my external USB drive: | ||
< | <syntaxhighlight lang="bash"> | ||
mondoarchive \ | mondoarchive \ | ||
-OV # do a backup, and verify \ | -OV # do a backup, and verify \ | ||
| Line 115: | Line 116: | ||
-S /media/disk/tmp # write scratch files to this directory | -S /media/disk/tmp # write scratch files to this directory | ||
-T /media/disk/tmp # write temporary files to this directory | -T /media/disk/tmp # write temporary files to this directory | ||
</ | </syntaxhighlight> | ||
At first, the backup failed with a message that it thought my drive was full. But in reality, it was a problem with the tmp partition being too small so then I added the -S and -T options and it worked fine. | At first, the backup failed with a message that it thought my drive was full. But in reality, it was a problem with the tmp partition being too small so then I added the -S and -T options and it worked fine. | ||
| Line 144: | Line 145: | ||
On a more recent "backup" effort, I needed to archive off the contents of a laptop to an external USB drive that was mounted on another system. The source system was a Windows XP machine, while the target machine was running Linux. The tools I used to make the backup were <code>dd</code><ref>http://en.wikipedia.org/wiki/Dd_(Unix)</ref> and a LiveCD of the [http://linuxmint.com/ Linux Mint] distribution. By inserting the Linux Mint Live CD, and rebooting the laptop, I would have access to a bash shell that I could then run the dd tool from. Not only that, but I could use the Secure Shell to pipe the command over the network to the target machine's mounted external 1TB drive. Note that in my case, I used a private key (identity file) to authenticate my ssh session. | On a more recent "backup" effort, I needed to archive off the contents of a laptop to an external USB drive that was mounted on another system. The source system was a Windows XP machine, while the target machine was running Linux. The tools I used to make the backup were <code>dd</code><ref>http://en.wikipedia.org/wiki/Dd_(Unix)</ref> and a LiveCD of the [http://linuxmint.com/ Linux Mint] distribution. By inserting the Linux Mint Live CD, and rebooting the laptop, I would have access to a bash shell that I could then run the dd tool from. Not only that, but I could use the Secure Shell to pipe the command over the network to the target machine's mounted external 1TB drive. Note that in my case, I used a private key (identity file) to authenticate my ssh session. | ||
< | <syntaxhighlight lang="bash"> | ||
sudo dd if=/dev/sda2 | ssh -i /home/mint/id_rsa-greg-notebook greg@192.168.1.11 "dd of=/media/disk-a/backups/sheila-laptop/acer.image.2" | sudo dd if=/dev/sda2 | ssh -i /home/mint/id_rsa-greg-notebook greg@192.168.1.11 "dd of=/media/disk-a/backups/sheila-laptop/acer.image.2" | ||
| Line 152: | Line 153: | ||
# and also on Linux, a SIGINFO signal to the process id of a dd process will show you the status, and then continue. | # and also on Linux, a SIGINFO signal to the process id of a dd process will show you the status, and then continue. | ||
kill -USR1 $(pidof dd) | kill -USR1 $(pidof dd) | ||
</ | |||
# a local copy is MUCH faster than a network copy -- getting 17 MB/s instead of ~400 kB/s | |||
sudo dd if=/dev/sda5 bs=4096 conv=noerror of=/media/disk-a/backups/sheila-laptop/acer.image.5 | |||
</syntaxhighlight> | |||
== Performance == | == Performance == | ||
| Line 158: | Line 162: | ||
Backup up ~60 GB of data across a USB 1 interface using rsync and -z for compression will take over 1 day. In fact it will take over 10 minutes to generate the file list. My experience was with 874,490 files. | Backup up ~60 GB of data across a USB 1 interface using rsync and -z for compression will take over 1 day. In fact it will take over 10 minutes to generate the file list. My experience was with 874,490 files. | ||
Similarly, if you are backing up across a network, then you can get dramatically slower results than if the source and target are local. For example, in my most recent backup, I was able to copy 18GB over a wireless network but it took 43,712 seconds (over 12 hours). 19GB over a local USB2 interface took 1,037 seconds (17 minutes). | |||
In most backup scenarios, the first backup is the one that takes the most time. | In most backup scenarios, the first backup is the one that takes the most time. | ||