Category: Fedora

btrfs scrub complete

This was the status at the end of the scrub:

[root@supermario ~]# /usr/sbin/btrfs scrub start -Bd /media/Photos/
scrub device /dev/sdd1 (id 1) done
 scrub started at Tue Mar 21 17:18:13 2017 and finished after 05:49:29
 total bytes scrubbed: 2.31TiB with 0 errors
scrub device /dev/sda1 (id 2) done
 scrub started at Tue Mar 21 17:18:13 2017 and finished after 05:20:56
 total bytes scrubbed: 2.31TiB with 0 errors

I’m a bit perplexed at this information. Since this is a RAID1, I would expect it to be comparing info between disks – is this not so? If not, why? Because I would have expected both disks to end at the same time. Also, interesting to note that the 1TB/hr stopped being the case at some point.

Speed of btrfs scrub

Here’s the output of the status command:

[root@supermario ~]# btrfs scrub status /media/Photos/
scrub status for 27cc1330-c4e3-404f-98f6-f23becec76b5
 scrub started at Tue Mar 21 17:18:13 2017, running for 01:05:38
 total bytes scrubbed: 1.00TiB with 0 errors

So on Fedora 25 with an AMD-8323 (8 core, no hyperthreading) and 24GB of RAM with this hard drive and its 3TB brother in RAID1 , it takes about an hour per Terabyte to do a scrub. (Which seems about equal to what a coworker told me his system takes to do a zfs scrub – 40ish hours for about 40ish TB)

Finally have btrfs setup in RAID1

A little under 3 years ago, I started exploring btrfs for its ability to help me limit data loss. Since then I’ve implemented a snapshot script to take advantage of the Copy-on-Write features of btrfs. But I hadn’t yet had the funds and the PC case space to do RAID1. I finally was able to implement it for my photography hard drive. This means that, together with regular scrubs, I should have a near miniscule chance of bit rot ruining any photos it hasn’t already corrupted.

Here’s a documentation of some commands and how I got the drives into RAID1:

 

Before RAID:

# btrfs fi df -h /media/Photos
Data, single: total=2.31TiB, used=2.31TiB
System, DUP: total=8.00MiB, used=272.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=3.50GiB, used=2.68GiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=512.00MiB, used=0.00B

# btrfs fi usage /media/Photos
Overall:
    Device size:                   2.73TiB
    Device allocated:              2.32TiB
    Device unallocated:          423.48GiB
    Device missing:                  0.00B
    Used:                          2.31TiB
    Free (estimated):            425.29GiB      (min: 213.55GiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 5.64MiB)

Data,single: Size:2.31TiB, Used:2.31TiB
   /dev/sdd1       2.31TiB

Metadata,single: Size:8.00MiB, Used:0.00B
   /dev/sdd1       8.00MiB

Metadata,DUP: Size:3.50GiB, Used:2.68GiB
   /dev/sdd1       7.00GiB

System,single: Size:4.00MiB, Used:0.00B
   /dev/sdd1       4.00MiB

System,DUP: Size:8.00MiB, Used:272.00KiB
   /dev/sdd1      16.00MiB

Unallocated:
   /dev/sdd1     423.48GiB

   
[root@supermario ~]# btrfs device add /dev/sda1 /media/Photos/
/dev/sda1 appears to contain an existing filesystem (btrfs).
ERROR: use the -f option to force overwrite of /dev/sda1
[root@supermario ~]# btrfs device add /dev/sda1 /media/Photos/ -f

[root@supermario ~]# btrfs fi usage /media/Photos
Overall:
    Device size:                   6.37TiB
    Device allocated:              2.32TiB
    Device unallocated:            4.05TiB
    Device missing:                  0.00B
    Used:                          2.31TiB
    Free (estimated):              4.05TiB      (min: 2.03TiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)

Data,single: Size:2.31TiB, Used:2.31TiB
   /dev/sdd1       2.31TiB

Metadata,single: Size:8.00MiB, Used:0.00B
   /dev/sdd1       8.00MiB

Metadata,DUP: Size:3.50GiB, Used:2.68GiB
   /dev/sdd1       7.00GiB

System,single: Size:4.00MiB, Used:0.00B
   /dev/sdd1       4.00MiB

System,DUP: Size:8.00MiB, Used:272.00KiB
   /dev/sdd1      16.00MiB

Unallocated:
   /dev/sda1       3.64TiB
   /dev/sdd1     423.48GiB


[root@supermario ~]# btrfs balance start -dconvert=raid1 -mconvert=raid1 /media/Photos/

Done, had to relocate 2374 out of 2374 chunks

Post-RAID:

[root@supermario ~]# btrfs fi usage /media/Photos
Overall:
    Device size:                   6.37TiB
    Device allocated:              4.63TiB
    Device unallocated:            1.73TiB
    Device missing:                  0.00B
    Used:                          4.62TiB
    Free (estimated):            891.01GiB      (min: 891.01GiB)
    Data ratio:                       2.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)

Data,RAID1: Size:2.31TiB, Used:2.31TiB
   /dev/sda1       2.31TiB
   /dev/sdd1       2.31TiB

Metadata,RAID1: Size:7.00GiB, Used:2.56GiB
   /dev/sda1       7.00GiB
   /dev/sdd1       7.00GiB

System,RAID1: Size:64.00MiB, Used:368.00KiB
   /dev/sda1      64.00MiB
   /dev/sdd1      64.00MiB

Unallocated:
   /dev/sda1       1.32TiB
   /dev/sdd1     422.46GiB
   
   
[root@supermario ~]# btrfs fi df -h /media/Photos
Data, RAID1: total=2.31TiB, used=2.31TiB
System, RAID1: total=64.00MiB, used=368.00KiB
Metadata, RAID1: total=7.00GiB, used=2.56GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

And here’s the status of my first scub to test out the commands:

[root@supermario ~]# btrfs scrub status /media/Photos/
scrub status for 27cc1330-c4e3-404f-98f6-f23becec76b5
 scrub started at Tue Mar 21 17:18:13 2017, running for 00:09:10
 total bytes scrubbed: 145.57GiB with 0 errors

Using Flatpak to install LibreOffice on Fedora 24

After someone told me that a PDF I’d created in Calligra Office was illegible and having issues with spreadsheets loading slowly, I decided to install LibreOffice. However, rather than go with the version in the repos, I decided to go with Flatpak – which allows for a more advanced version via the usage of runtimes. First, I had to install Flatpak:

[code language=”bash”]

sudo dnf install flatpak

[/code]

Then I needed to install the runtimes. The LibreOffice page uses the –user tag, but I think that is just for installing it just to yourself rather than for the whole system. So I am omitting that.

[code language=”bash”]
wget https://sdk.gnome.org/keys/gnome-sdk.gpg
flatpak remote-add –gpg-import=gnome-sdk.gpg gnome https://sdk.gnome.org/repo/
flatpak install gnome org.gnome.Platform 3.20
[/code]

That took a bit and said things like “Writing Objects” on the terminal. Eventually that was done. Then it was time for LibreOffice. I grabbed the file from the website, then:

[code language=”bash”]

flatpak install –bundle LibreOffice.flatpak

[/code]

After doing that I did an alt-F2 to see if it could launch like a regularly installed application. It did not show up. Perhaps Flatpak only works well with Gnome for now?

[code language=”bash”]
flatpak run org.libreoffice.LibreOffice
[/code]

Worked, though.

In the future if I want to update it, I need to run:

[code language=”bash”]
flatpak update –user org.libreoffice.LibreOffice</pre>
[/code]

I do have to say that I’m disappointed it doesn’t appear in my alt-F2 menu.

SuperMario is at Fedora 24

My main computer is now on Fedora 24. This time around I only had to uninstall HDR Merge (which was from my COPR and I hadn’t built a Fedora 24 version yet) and OBS-Studio because there isn’t a Fedora 24 package for it yet. Not bad.

After rebooting, I didn’t have graphics. Then rebooting once more kicked the akmod into gear and now things appear to be working well. 2 more computers left to upgrade to Fedora 24 – the VM server and the Kodi living room box.

Fedora 24 is out!

Fedora 24 was released yesterday. I updated Daisy, my big laptop, first since it’s not critical. If the update broke something I wouldn’t care. The only hitch it had was that I had to reinstall the RPMFusion repos from the RPM for Fedora 24. Otherwise it was saying that one of the packages wasn’t signed and refused to do the upgrade. Probably has something to do with the fact that for the last release or two, RPMFusion wasn’t exactly in the best of conditions. I’m currently updating my netbook (Kuribo), but that’s more of an all-evening affair since it’s just running on an Intel Atom. There are three more Fedora machines in the house – SuperMario, TanukiMario, and BlueYoshi. I’ll probably save the livingroom Kodi (BY) for last since everyone in the house uses that to watch TV.

So far I haven’t noticed too many changes on KDE, but the icon indicating that I have updates is different. That seems to change with every release (or at least every other one).

A look at the many flavors of Korora

It’s been a long time since I looked at a new Linux distro. Long time readers know I used to review Linux distros a few years ago. But one of the maintainers of Korora posts to the Fedora Planet feed. (They may say constantly that RSS feeds are dead, but some of us still use them!) Korora (which used to be based on Gentoo) aims to create the ultimate desktop user’s Fedora setup. So they tweak Fedora a bit and add some repos like RPMFusion out of the box. Since I do a lot of this every time I install Fedora anyway, I may want to just install Korora next time I do an install. So I wanted to look at Korora and also use it as a chance to take a look at what’s going on nowadays in the non-KDE/Plasma desktops. Also, since Korora 23 should be coming out within the next month or so, it should help me see how Korora upgrades compare to Fedora (*should* be the same). First the install:

The only bad thing I could find with the install is not in that video. Certain desktops (XFCE was one, I’m not sure I think another was Mate?) have their “done” buttons obscured by the theming. Luckily I knew to click somewhere in the top left corner. Hopefully that’s fixed for the future.

For some reason, my XFCE video won’t embed. You can watch it here.

Upgrading to Korora 23 didn’t really change much – the desktop environments increased in version numbering, but I didn’t see anything drastically different.

I created my first RPM! And have a copr repo!

It’s the intersection of three of my hobbies – computers, Linux, and photography! Ever since I learned how to compile source code from the net, about a decade ago, I’ve wanted to create RPMs to help those who aren’t comfortable with compiling or simply don’t want to bother with compiling. But, for some reason, RPM creation was always something I struggled to get right. Nearly once a year I’d try and do it, but always failed. But recently, when reading the instructions on how to do it, it just clicked.

And there was software I wanted to use that didn’t have an RPM already. So I grabbed the source code to hdrmerge (which allows one to create high dynamic range photos – I plan on blogging about that some time in the near future) and worked to create an RPM. Then, to make it easy for people to update as the software updates, I created a copr repository!

You can find the repo here.

Also, it turned out to be more than RPM creation 101, it was a 200 level class as I had to figure out how to create a patch to make it work on Fedora. The next step might be for me to get it into the official Fedora repositories since there aren’t any licensing or patent reasons it couldn’t be in there.

Reinstalling Fedora 23 on my main Linux Computer

After daydreaming about running my Linux computer from an SSD for years (ever since I saw how fast it made my wife’s Linux install), I finally did it on the main Linux computer (SuperMario) as the root hard drive was getting old and was liable to die at any moment. The last time I’d installed my main Linux computer from scratch was in 2009 when I went from an old eMachine to the computer I’d built for my wife when I was going to ease her from Windows on her Dell computer over to Linux on a custom built computer. Lots had changed since then (like going to SystemD, for example) and I also wanted to move to a larger hard drive. So just copying everything over was out of the question. I figured a fresh install would get rid of a lot of the cruft I’d picked up in the last 7 years. Additionally, this time I would start with an install of the KDE spin of Fedora and I’m not planning on adding in other desktop environments or Window managers unless I am ready to ditch KDE. If I want to dabble in other desktops, I can always run a VM.

So I just wanted to document what I had to do for others who might want to do something similar in the future. First of all, key to this (and it also helps for when you have a growing digital library) is to have /home on its own hard drive. This way your personal settings travel with you easily and you don’t have to worry about copying your personal files back and forth. Before you do anything, go over the programs you have installed and figure out which ones you want for sure on the new system. Think about where they keep their settings. Again, most of them should be in your home directory, but when they’re meant to be shared with everyone, they may be in /usr/share. That’s the situation I found myself in with the flyers and datasheets for QMC2. Make note of the repositories you are using. Because dnf/yum check the repo keys you won’t be able to just copy them over and have it work, but note which you have and where to get them again.

Because I am using btrfs snapshots, I had to create a special /etc/updatedb.conf file for locate. Otherwise it indexed all the snapshots and filled the root partition. You’ll want to make a copy of /etc/fstab. You can’t just drop it in as a replacement because of the / drive  you’re installing, but for all the other drives, this will quickly get you back to a fully working computer. For me, I had entries for my home directory as well as a few other hard drives. You also might want to copy your user and root’s cronjobs. Some might be recreated, but others (especially if they were some clever little thing) will need to be copied. If you don’t use local DNS, you might want to copy your /etc/hosts file. Finally, copy over /etc/ssh to make sure you computer doesn’t seem foreign when you try to SSH to or from other computers.

I use Crashplan for backup, but they were almost the easiest part of the migration. Once I reinstalled and installed Crashplan, I was able to “adopt” the old computer (which was the same computer….just a new hard drive), it just verified that it could see all the files and all was good.

Once you’re installed, the first thing you might want to do is to log in as root and chown -R user:user (change to your username) on your /home directory. I’d been using Fedora so long that the first user had gone from being the traditional 500 to being 1000. So without doing this I wouldn’t have been able to access my own files. Then go about restoring the stuff you copied above like the cron jobs. Finally, if you use home DNS and/or static IP addresses, be sure to set that up as well.

That’s pretty much it. It took me a couple hours to get the initial setup and then a couple more hours of wrangling with video drivers. I hope this helps if you’re considering doing a reinstall, but quickly getting back to work with your settings in tact.

XCOM, Steam, and Linux

One of the great things about Steam and the internet is the cloud save. It means that you can go from computer to computer and, as long as it’s connected to the Internet, you can pick up your game where you left off. In my case, I have a Windows computer and a Linux computer both capable of running games from Steam. I wanted to see how well XCOM was ported to Linux so I launched it and didn’t have any of my saves! It turns out that the saves are in a folder called SaveData on Windows and savedata on Linux. (In the path: ~/.local/share/feral-interactive/XCOM/XEW – remove XEW if it’s not Enemy Within) Those aren’t the same thing in Linux. So you need to delete the folder savedata and then make a symlink to SaveData where the symlink is savedata. Once you do that it works perfectly. To give credit where it’s due, here’s the forum post where I fond the info.

Host to Guest Comms on KVM

So, the current solution for bridging with KVM/QEMU/Libvirt involves macvtap. This allows your VM to be seen by computers on the network which is key if you want to, say, run a server or DNS on a VM. However, there’s a catch – the host can’t reach the VM. Sometimes this doesn’t matter and sometimes it does. For example, if the guest is running DNS and you want the host to be able to use it for DNS, this is an issue. There is another way to do it, but it involves ditching Network Manager. I actually like Network Manager. So what am I to do?

Well, there are two solutions. One is to use a routed mode network. This causes the host to behave like a router and creates a subnet behind the host so if all your real computers are 192.168.1.x, the VMs would be 192.168.2.x. (Doesn’t have to be 2) For many scenarios this is the cleanest and most correct way to do this. However, it requires your real router to support static routes and it may break compatibility with consumer hardware that expects everything in a home to be on one subnet – eg networked printers, DLNA, etc.

The other solution is presented on the libvirt wiki as the “Less Painful Solution” for getting around macvtap. You create an isolated network (which you usually only do when you are testing and want to make sure that all the network traffic you’re observing comes from the VMs on said isolated network. Then you give your VM a second network interface that connects to this isolated network. You use THIS interface for host/guest communications. The part it doesn’t mention on the wiki that I plan to test as part of this blog post is to then add that entry to the Host’s /etc/hosts file so that it overrides DNS and the host is none-the-wiser that it’s not accessing it via the DNS which would be the address that the Host can’t use.

So, let’s give it a shot. I’m going to use the GUI for my first go-around.

Virt-manager - Isolated Network 1
Virt-manager – Isolated Network 1
Virt-manager - Isolated Network 2 - pick the subnet and if you're going to enable DHCP
Virt-manager – Isolated Network 2 – pick the subnet and if you’re going to enable DHCP

Step 3 is IPv6…skipping that….

Virt-manager - Isolated Network 4 - pick isolated
Virt-manager – Isolated Network 4 – pick isolated

Open the VM you want to add it to:

Virt-manager - Isolated Network -Adding the NIC
Virt-manager – Isolated Network -Adding the NIC

Run the VM and see if it took….

Virt-manager - Isolated Network - checking that it got a new IP address
Virt-manager – Isolated Network – checking that it got a new IP address

It appears that it did. So let’s see if I can SSH to it now:

Virt-manager - Isolated Network - checking SSH
Virt-manager – Isolated Network – checking SSH

OK, it all seems to be in order. Good, good. I also confirmed that adding it to the /etc/hosts file allows it to override what’s in DNS. So, there you go! That’s one solution if you want host to guest comms with KVM/QEMU/Libvirt.