Exploring btrfs for backups Part 2: Installing on My /Home Directory and using my new Python Script
By EricMesa
- 6 minutes read - 1261 wordsI got my new hard drive that would replace my old, aging /home hard drive. As you read in part 1, I wanted to put btrfs on it. This is my journey to get it up and running. Plugged it into my hard drive toaster and ran gparted.
[caption id=“attachment_7889” align=“aligncenter” width=“421”] 
Gparted for new drive[/caption]
[caption id=“attachment_7890” align=“aligncenter” width=“500”] 
Gparted for new drive1[/caption]
[caption id=“attachment_7891” align=“aligncenter” width=“421”] 
Gparted for new drive2[/caption]
[caption id=“attachment_7892” align=“aligncenter” width=“500”] 
Gparted for new drive3[/caption]
Because of the peculiarities of btrfs and what I wanted to do with subvolumes, that wasn’t enough. I’d run an rsync, but forgot to create a home subvolume first. The rysnc still had a long way to go so it wasn’t so bad when I deleted the rsync’d folder. Created the subvolume.
# btrfs subvolume create /run/media/username/Home1/home
Create subvolume '/run/media/username/Home1/home'
# btrfs sub list /run/media/username/Home1/
ID 275 gen 199 top level 5 path home
ran the rsync again
# nice rsync --human-readable --verbose --progress --perms --times --links --acls --xattrs -recursive --one-file-system --exclude=/.Trash-0 /home/ /run/media/username/Home1/home/
Even though I’d lost about an hour’s worth of transfers I took advantage to rid myself of a few pointless files. (So many images in gwibber’s cache (which I don’t even use anymore) that it was too many files for a rm * command. Sure they’re small files but I’m sure the overhead of each file was why it was in the folder for over half an hour. Then again, maybe each file was tiny but the folder wasn’t. I didn’t do a close enough comparison, but based on what I remember from my last df, I think I gained 20-30 GB by deleting that cache. WOW! And I still have other things to consider deleting in the folder like Beagle - I don’t use that for search anymore. Not sure anyone does. It was at USB 2.0 speeds - don’t have e-sata on that computer - which meant 50MB/s max (that I saw while I was watching the commandline) and closer to mid-20s on most files. Not 100% sure why, but the rsync would only run as root. As it’s currently running, even though I passed it the –perms it appears that everything is owned by root. Not sure if rsync will fix that at the end. If it doesn’t, I’ll have to do that manually after the rsync is done. I’m live-blogging this part of the process as it happens. I’m up to 42595 forthe ir-check denominator. (Which I’d assumed was how many files it needed to copy) It’s now iterating through a fairly large Firefox cache. Lesson to others migrating their home drives with rsync - clear those caches! While waiting I looked up what the numbers mean - yes, that’s the number of files, but right now it’s ir- instead of to- which means rysync hasn’t finished counting yet. Well, enough live-blogging for now - I’m going to go shower and other things. I’ll be back when it’s done. Given that it has done 18 GB in about an hour and it has to do 760…. it’ll probably be an overnight job. It did take quite a few hours. When I looked in dmesg there were a lot of read errors on the old drive. It’d failed SMART a year ago. Looks like I replaced it in the nick of time.
Ran a df and it matched up. Unfortunately, everything was now owned by root so I had to chown and chmod the home directory.
Time to edit fstab. First it was time to figure out the block id.
# blkid
/dev/sda1: LABEL="/boot" UUID="6f51a1f6-4267-45eb-84eb-45ade094b037" TYPE="ext3" PARTUUID="000a338b-01"
/dev/sda2: LABEL="SWAP-sda2" UUID="45897b4e-f7d6-4ae5-bdf5-6f2a23c9a769" TYPE="swap" PARTUUID="000a338b-02"
/dev/sda3: LABEL="/" UUID="c4710ace-faf2-4f0f-8609-cb0be82dce34" TYPE="ext3" PARTUUID="000a338b-03"
/dev/sdb1: LABEL="/home" UUID="ea558d02-2dc2-4c8e-ad82-12432050746b" TYPE="ext3" PARTUUID="000475af-01"
/dev/sdc1: LABEL="backup" UUID="8b1f32ff-edc5-48b8-8ec5-c711fb2083b3" TYPE="ext3" PARTUUID="0008b01a-01"
/dev/sdd1: LABEL="Home1" UUID="89cfd56a-06c7-4805-9526-7be4d24a2872" UUID_SUB="590a694e-f873-45ae-997b-0bdc4a8bc5eb" TYPE="btrfs" PARTUUID="c8a5d7c4-002f-467d-8414-722d3a65a6a5"
The fact that there’s a UUID_SUB makes me believe that will be the UUID for my /home subvolume, which would solve the problem what if I mounted the /dev/sdd1 at /home, it’d have a home directory inside the home directory. Perhaps that’s not an issue as I mounted it into a test directory and it didn’t double up on the homes. My google-fu says I should just use the regular UUID. My fstab looked like this at first:
#
# /etc/fstab
# Created by anaconda on Sat May 16 03:24:18 2009
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or vol_id(8) for more info
#
UUID=c4710ace-faf2-4f0f-8609-cb0be82dce34 / ext3 defaults 1 1
UUID=ea558d02-2dc2-4c8e-ad82-12432050746b /home ext3 defaults,user_xattr 1 2
UUID=6f51a1f6-4267-45eb-84eb-45ade094b037 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
UUID=45897b4e-f7d6-4ae5-bdf5-6f2a23c9a769 swap swap defaults 0 0
#added by Eric for share drive
babyluigi.mushroomkingdom:/fileshares /media/nfs/babyluigi nfs rsize=1024,wsize=1024,auto,users,soft,intr 0 0
babyluigi.mushroomkingdom:/media/xbmc /media/nfs/xbmc-mount nfs rsize=1024,wsize=1024,auto,users,soft,intr 0 0
#added by Eric for backup drive
UUID=8b1f32ff-edc5-48b8-8ec5-c711fb2083b3 /media/backup ext3 defaults 0 0
Afterwards it looked like this:
#
# /etc/fstab
# Created by anaconda on Sat May 16 03:24:18 2009
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or vol_id(8) for more info
#
UUID=c4710ace-faf2-4f0f-8609-cb0be82dce34 / ext3 defaults 1 1
#old home on 1TB drive
#UUID=ea558d02-2dc2-4c8e-ad82-12432050746b /home ext3 defaults,user_xattr 1 2
#new home on btrfs
UUID=89cfd56a-06c7-4805-9526-7be4d24a2872 /home btrfs defaults,subvol=home 0 2
UUID=6f51a1f6-4267-45eb-84eb-45ade094b037 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
UUID=45897b4e-f7d6-4ae5-bdf5-6f2a23c9a769 swap swap defaults 0 0
#added by Eric for share drive
babyluigi.mushroomkingdom:/fileshares /media/nfs/babyluigi nfs rsize=1024,wsize=1024,auto,users,soft,intr 0 0
babyluigi.mushroomkingdom:/media/xbmc /media/nfs/xbmc-mount nfs rsize=1024,wsize=1024,auto,users,soft,intr 0 0
#added by Eric for backup drive
UUID=8b1f32ff-edc5-48b8-8ec5-c711fb2083b3 /media/backup ext3 defaults 0 0
Time to reboot and see what happens before I put the hard drive into the system. SUCCESS! Now I just need to figure out which hard drive was the old home. I’ll disconnect one at a time and check to see which block ids are missing. I’ll be rebooting each time because even though SATA is plug and play, I’d rather not risk ruining things. Wrong on the first try. Luckily it’s only a 50% chance of being wrong as I knew which one had root. Time to try again. Success. Not sure why, but my Amarok music collection was lost although all the files are there; bummer. (It’s ok, after import this time it hadn’t lost the first/last played dates! yay!) My other KDE programs seemed to be OK. Interestingly, the computer’s running a bit faster. Maybe because of all the read errors on the old drive making things slower?
Now it’s time to create the snapshot directory to get things ready for the snapshots/backups.
# btrfs sub create /home/.snapshots
Create subvolume '/home/.snapshots'
OK, it’s time to use the snapshot/backup script I’ve created. You can get it at Github. It’s good for the daily level culling at this stage.
# ./snap-in-time.py
Create a readonly snapshot of '/home' in '/home/.snapshots/2014-03-13-2146'
# btrfs sub list /home
ID 275 gen 1653 top level 5 path home
ID 1021 gen 1653 top level 275 path .snapshots
ID 1023 gen 1653 top level 275 path .snapshots/2014-03-13-2146
OK, final step for now is to add it to cron to run every hour.
#btrfs snapshots of /home
0 * * * * /root/bin/snap-in-time.py
I had to change the script to give the full path of btrfs for it to work in cron.
Next time we go back to the VM to learn how to setup the btrfs RAID. See you then!