Hitting alt-F2 then typing email (contact name – eg Danielle) and enter and then it presents me with an email window to send an email. No need to navigate to gmail.com or go over to the screen running Kmail (actually, usually Kontact).
It’s been a while since btrfs was first introduced to me via a Fedora version that had it as the default filesystem. At the time, it was especially brittle when it came to power outages. I ended up losing a system to one such use case. But a few years ago, I started using btrfs on my home directory. And even developed a program to manage snapshots. My two favorite features of btrfs are that Copy on Write (COW) allows me to make snapshots that only take up space when the file that was snapshot changes and the ability to dynamically set up and grow RAID levels. I was able to use this recently to get my photo hard drive on RAID1 without having to have an extra hard drive (because most RAID solutions destroy what’s on the drive).
However, btrfs has been plagued with some important issues – for example RAID5/6 is unstable, not recommended, and after many years still hasn’t solved the write hole. (Something the very similar ZFS has had solved for years) Look online and you’ll find scores of tales of people who have suffered unrecoverable data loss from btrfs.
A few years ago Red Hat deprecated btrfs on RHEL6. That makes sense given the long support times of the RHEL releases. The team at Red Hat has to backport kernel fixes and that gets complicated as time goes by. btrfs has grown by leaps and bounds since RHEL6. But a couple days ago (when I write this – 10 days before the blog post is going to appear), Red Hat announced it was getting deprecated on RHEL7. There was lots of speculation on the net and someone who used to hack on btrfs for RHEL mentioned that since he left, no one at Red Hat worked on it. Suse is the distro that employes btrfs hackers at this point. Then, yesterday, Stratis was announced. From the Phoronix article I read about Stratis in:
First a quote from the announcement of Stratis:
Stratis is a new tool that meets the needs of Red Hat Enterprise Linux (RHEL) users calling for an easily configured, tightly integrated solution for storage that works within the existing Red Hat storage management stack. To achieve this, Stratis prioritizes a straightforward command-line experience, a rich API, and a fully automated, externally-opaque approach to storage management. It builds upon elements of the existing storage stack as much as possible, to enable delivery within 1-2 years. Specifically, Stratis initially plans to use device-mapper and the XFS filesystem. Extending or building on SSM 2.1.1 or LVM 2.1.2 was carefully considered. SSM did not meet the design requirements, but building upon LVM may be possible with some development effort.
From the Wikipage describing that it’s going to land in Fedora 28:
a local storage system akin to Btrfs, ZFS, and LVM. Its goal is to enable easier setup and management of disks and SSDs, as well as enabling the use of advanced storage features — such as thin provisioning, snapshots, integrity, and a cache tier — without requiring expert-level storage administration knowledge. Furthermore, Stratis includes monitoring and repair capabilities, and a programmatic API, for better integration with higher levels of system management software.
Then from the author of the Phoronix article:
For Stratis 1.0 they hope to support snapshot management, file-system maintenance, and more. With Stratis 2.0 is where they plan to deal with RAID, write-through caching, quotas, etc. With Stratis 3.0 is where it should get interesting as they hope for “rough ZFS feature parity” and support send/receive, integrity checking, RAID scrubbing, compression, encryption, deduplication, and more. Only in the first half of 2018 is when they expect to reach Stratis 1.0. No word on when they anticipate getting to Stratis 3.0 with ZFS feature parity.
Interesting. I led me on a path of exploration of LVM and other tech. First of all, I don’t imagine btrfs is going to sit still, unworked on, while this happens. Maybe it finally reaches its stability goals. Maybe the threat of Stratis attracts more hackers to btrfs. Or, maybe Stratis catches up with, and surpasses, btrfs. I think if they can make the dynamic RAID work, and can get stability up to ZFS levels, I could move over to Stratis. If not, I’m still thinking about LVM and XFS or ext4 for my home-built NAS rather than btrfs (or together with btrfs if it doesn’t get too complex for snapshotting purposes) because that would (potentially) let me grow directories indefinitely in a way that works as my backup needs grow. This will require more knowledge and planning, though. I’ll keep documenting my research here.
With modern technology, here’s the pattern I’ve noticed since college. New tech comes out and I can see that it’s neat, but not how I can make use of it. A few years later, I finally come across the right article and it all makes sense to me. I first noticed this with VMs. I couldn’t see a reason to want to use it outside of a server context. Then I used it to review Linux distros. Then I used it to run my network’s services. The same happened with tablets, smart phones, and Docker.
When everyone kept hyping up Docker I couldn’t figure out why it’d be useful to me. It seemed overly complex compared to VMs. And if I wanted to have lots of isolated services running, Linux Containers (LXC) seemed a lot easier and closer to what I was used to. In fact in a Linux on Linux (host:hypervisor) situation, containers seem superior to VMs in every way.
But Red Hat supports Docker. Maybe it’s because Ubuntu was championing LXC and they seem to abandon stuff all the time like Google. (Unity being the latest casualty) And I was having some issues with the version of LXC on CentOS 7 having some issues – like freezing up while running yum or not running Apache. So I decided to explore Docker again.
Since the last time I came across Docker, I got into Flatpack and AppImage and suddenly Docker made sense again for someone outside of DevOps (where it always made sense to me). Using containers means I can run an app with a consistent set of libraries independent of what’s on the system or being used by other apps. So I used Docker to run phpIPAM and while it’s still a little more complicated than I’d like, but not too bad now that I have my head around the concept.
This was the status at the end of the scrub:
[root@supermario ~]# /usr/sbin/btrfs scrub start -Bd /media/Photos/ scrub device /dev/sdd1 (id 1) done scrub started at Tue Mar 21 17:18:13 2017 and finished after 05:49:29 total bytes scrubbed: 2.31TiB with 0 errors scrub device /dev/sda1 (id 2) done scrub started at Tue Mar 21 17:18:13 2017 and finished after 05:20:56 total bytes scrubbed: 2.31TiB with 0 errors
I’m a bit perplexed at this information. Since this is a RAID1, I would expect it to be comparing info between disks – is this not so? If not, why? Because I would have expected both disks to end at the same time. Also, interesting to note that the 1TB/hr stopped being the case at some point.
Here’s the output of the status command:
[root@supermario ~]# btrfs scrub status /media/Photos/ scrub status for 27cc1330-c4e3-404f-98f6-f23becec76b5 scrub started at Tue Mar 21 17:18:13 2017, running for 01:05:38 total bytes scrubbed: 1.00TiB with 0 errors
So on Fedora 25 with an AMD-8323 (8 core, no hyperthreading) and 24GB of RAM with this hard drive and its 3TB brother in RAID1 , it takes about an hour per Terabyte to do a scrub. (Which seems about equal to what a coworker told me his system takes to do a zfs scrub – 40ish hours for about 40ish TB)
A little under 3 years ago, I started exploring btrfs for its ability to help me limit data loss. Since then I’ve implemented a snapshot script to take advantage of the Copy-on-Write features of btrfs. But I hadn’t yet had the funds and the PC case space to do RAID1. I finally was able to implement it for my photography hard drive. This means that, together with regular scrubs, I should have a near miniscule chance of bit rot ruining any photos it hasn’t already corrupted.
Here’s a documentation of some commands and how I got the drives into RAID1:
Before RAID: # btrfs fi df -h /media/Photos Data, single: total=2.31TiB, used=2.31TiB System, DUP: total=8.00MiB, used=272.00KiB System, single: total=4.00MiB, used=0.00B Metadata, DUP: total=3.50GiB, used=2.68GiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=512.00MiB, used=0.00B # btrfs fi usage /media/Photos Overall: Device size: 2.73TiB Device allocated: 2.32TiB Device unallocated: 423.48GiB Device missing: 0.00B Used: 2.31TiB Free (estimated): 425.29GiB (min: 213.55GiB) Data ratio: 1.00 Metadata ratio: 2.00 Global reserve: 512.00MiB (used: 5.64MiB) Data,single: Size:2.31TiB, Used:2.31TiB /dev/sdd1 2.31TiB Metadata,single: Size:8.00MiB, Used:0.00B /dev/sdd1 8.00MiB Metadata,DUP: Size:3.50GiB, Used:2.68GiB /dev/sdd1 7.00GiB System,single: Size:4.00MiB, Used:0.00B /dev/sdd1 4.00MiB System,DUP: Size:8.00MiB, Used:272.00KiB /dev/sdd1 16.00MiB Unallocated: /dev/sdd1 423.48GiB [root@supermario ~]# btrfs device add /dev/sda1 /media/Photos/ /dev/sda1 appears to contain an existing filesystem (btrfs). ERROR: use the -f option to force overwrite of /dev/sda1 [root@supermario ~]# btrfs device add /dev/sda1 /media/Photos/ -f [root@supermario ~]# btrfs fi usage /media/Photos Overall: Device size: 6.37TiB Device allocated: 2.32TiB Device unallocated: 4.05TiB Device missing: 0.00B Used: 2.31TiB Free (estimated): 4.05TiB (min: 2.03TiB) Data ratio: 1.00 Metadata ratio: 2.00 Global reserve: 512.00MiB (used: 0.00B) Data,single: Size:2.31TiB, Used:2.31TiB /dev/sdd1 2.31TiB Metadata,single: Size:8.00MiB, Used:0.00B /dev/sdd1 8.00MiB Metadata,DUP: Size:3.50GiB, Used:2.68GiB /dev/sdd1 7.00GiB System,single: Size:4.00MiB, Used:0.00B /dev/sdd1 4.00MiB System,DUP: Size:8.00MiB, Used:272.00KiB /dev/sdd1 16.00MiB Unallocated: /dev/sda1 3.64TiB /dev/sdd1 423.48GiB [root@supermario ~]# btrfs balance start -dconvert=raid1 -mconvert=raid1 /media/Photos/ Done, had to relocate 2374 out of 2374 chunks Post-RAID: [root@supermario ~]# btrfs fi usage /media/Photos Overall: Device size: 6.37TiB Device allocated: 4.63TiB Device unallocated: 1.73TiB Device missing: 0.00B Used: 4.62TiB Free (estimated): 891.01GiB (min: 891.01GiB) Data ratio: 2.00 Metadata ratio: 2.00 Global reserve: 512.00MiB (used: 0.00B) Data,RAID1: Size:2.31TiB, Used:2.31TiB /dev/sda1 2.31TiB /dev/sdd1 2.31TiB Metadata,RAID1: Size:7.00GiB, Used:2.56GiB /dev/sda1 7.00GiB /dev/sdd1 7.00GiB System,RAID1: Size:64.00MiB, Used:368.00KiB /dev/sda1 64.00MiB /dev/sdd1 64.00MiB Unallocated: /dev/sda1 1.32TiB /dev/sdd1 422.46GiB [root@supermario ~]# btrfs fi df -h /media/Photos Data, RAID1: total=2.31TiB, used=2.31TiB System, RAID1: total=64.00MiB, used=368.00KiB Metadata, RAID1: total=7.00GiB, used=2.56GiB GlobalReserve, single: total=512.00MiB, used=0.00B
And here’s the status of my first scub to test out the commands:
[root@supermario ~]# btrfs scrub status /media/Photos/ scrub status for 27cc1330-c4e3-404f-98f6-f23becec76b5 scrub started at Tue Mar 21 17:18:13 2017, running for 00:09:10 total bytes scrubbed: 145.57GiB with 0 errors
Continuing my LXC project, let’s list the installed containers:
That just shows the name of the container – lemmy. For completion’s sake, I’m going to start it as a daemon in the background rather than being sent straight into the console:
lxc-start -n lemmy -d
As per usual Linux SOP, it produced no output. Now to jump in:
lxc-console -n lemmy
That told me I was connected to tty1, but did not present a login. Quitting out via Ctrl-a q let me go back to the VM’s tty, but trying again did not get me login. There’s some weird issue that doesn’t allow it to work, however, this did:
lxc-attach -n lemmy
I’m not 100% sure why it works and console doesn’t, but there seems to be discussion about systemd causing issues. At any rate, the only limitation of lxc-attach is that the user doing it has to also exist on the container. However, given that these are server boxes, root is fine and so it works.
Unfortunately, networking does not work. That’ll be for next time.
I’m continuing on from yesterday’s post to get the VM ready to host LXC. I’m starting with Centos 7 so the first thing I had to do was enable the epel repos:
yum install epel-release
Then, according to the guide I was following, I had to also install these package:
yum install debootstrap perl libvirt
That installed a bunch of stuff. I also get that they’re trying to break out what they’re doing, but they probably could have installed both that and the LXC stuff below in one blow:
yum install lxc lxc-templates
Then start the services we just installed:
systemctl start lxc.service systemctl start libvirtd
Then, a good thing to do to make sure everything’s working correctly is to run the following:
If you get all “enabled” (in Centos 7 it’s also green), then you’re in good shape. You can see which templates you have installed with the following command:
ls -alh /usr/share/lxc/templates/
When I did that, I had alpine, altlinux, busybox, centos, cirros, debian, fedora, gentoo, openmandriva, opensuse, oracle, ubuntu, and ubuntu-cloud.
As my last act of this post, I’ll create my first container:
lxc-create -n lemmy -t centos
This is going to run Cockpit to keep an eye on servers on my network. After running that command, it looked like a yum or dnf install was happening. Then it did some more stuff and then told me what the root password was. It also told me how to change it without having to start the container. So I did that. Next time…starting and running a container.
As I mentioned before, I’m looking at Linux Containers (LXC) to have a higher density virtualization. To get ready for that, I had to create a network bridge to allow the containers to be accessible on the network.
First I installed bridge-utils:
yum install bridge-utils -y
After that, I had to create the network script:
In there I placed:
DEVICE="virbr0" BOOTPROTO="static" IPADDR="192.168.1.35" #IP address of the VM NETMASK="255.255.255.0" GATEWAY="192.168.1.1" DNS1="192.168.1.7" ONBOOT="yes" TYPE="Bridge"
Then, since my ethernet on this machine is eth0
and after a
systemctl restart network
it was supposedly working. I was able to ping www.google.com. We’ll see what happens when I start installing LXC Containers.
I updated Rawhide and ended up with this login screen. I like it – I think mostly because of the font.
Back when I first was working on replacing my Pogoplug (the original BabyLuigi), I was looking at potentially using it to learn about Docker in addition to creating virtual machines that were actually useful instead of just playing around with VMs for looking at Linux distros. The benefit of Docker was to have the isolation of VMs without the overhead of VMs. Also, since it was trending pretty hard, I figured it’d be good for my career to have some experience with it. So I spent a few weeks researching Docker and playing around with some of the online demos. I read lots about how it was used and how to avoid the usual pitfalls. But in the end I went with a VM that did a bit more than I wanted; I’d wanted to separate services so that updating one thing wouldn’t cause me to lose everything. However, the more I looked into it, the more it looked like unnecessary headache without enough of a benefit. Dockers were SO isolated that if you wanted to run a LAMP stack you had to run at least 3 Docker containers and find a way to string them together and have a separate pool of storage they could all access.
Recently I’ve been hanging around in the Home Lab subreddit. There are a lot of people in there like me who enjoy learning about computing and using it to make things easier (if a bit more complex) at home. It pairs well with the Self Hosted subreddit, another thing that is important to me because of how many services have been changed or dropped (see Google Reader) from under me. I prefer to be in charge of things on my own. In that subreddit I heard about LXC – Linux Containers. I was intrigued and after I came across it again, I did some research. LXC is what I wanted Docker to be (which makes sense since Docker is forked from LXC). It allows you to run what is essentially a VM without the overhead of simulating hardware. I’ve seen some webpages that claim you can get 41 LXC containers where you’d be able to fit about 2 VMs. I haven’t seen anything that high in the Home Lab subreddit, but I HAVE seen some pretty impressive densities. Since I don’t have the money for a computer that could hold the number of VMs I’d love to run, I’m going to be exploring LXC containers. I’ll blog about my progress so you can learn along with me. I’m pretty excited about learning this new tech.
I created this video to help people learn how easy it is with Libvirt, KVM, and QEMU to have multiple monitors in your virtual machines.
Going to do some summer cleaning on my VMs, so I wanted to document peak KVM as a reminder of how many I had running at this time:
After someone told me that a PDF I’d created in Calligra Office was illegible and having issues with spreadsheets loading slowly, I decided to install LibreOffice. However, rather than go with the version in the repos, I decided to go with Flatpak – which allows for a more advanced version via the usage of runtimes. First, I had to install Flatpak:
sudo dnf install flatpak
Then I needed to install the runtimes. The LibreOffice page uses the –user tag, but I think that is just for installing it just to yourself rather than for the whole system. So I am omitting that.
flatpak remote-add –gpg-import=gnome-sdk.gpg gnome https://sdk.gnome.org/repo/
flatpak install gnome org.gnome.Platform 3.20
That took a bit and said things like “Writing Objects” on the terminal. Eventually that was done. Then it was time for LibreOffice. I grabbed the file from the website, then:
flatpak install –bundle LibreOffice.flatpak
After doing that I did an alt-F2 to see if it could launch like a regularly installed application. It did not show up. Perhaps Flatpak only works well with Gnome for now?
flatpak run org.libreoffice.LibreOffice
In the future if I want to update it, I need to run:
flatpak update –user org.libreoffice.LibreOffice</pre>
I do have to say that I’m disappointed it doesn’t appear in my alt-F2 menu.
My main computer is now on Fedora 24. This time around I only had to uninstall HDR Merge (which was from my COPR and I hadn’t built a Fedora 24 version yet) and OBS-Studio because there isn’t a Fedora 24 package for it yet. Not bad.
After rebooting, I didn’t have graphics. Then rebooting once more kicked the akmod into gear and now things appear to be working well. 2 more computers left to upgrade to Fedora 24 – the VM server and the Kodi living room box.