Programming Update for April 2021

I had a lot less variety in my programming month, but still had a lot of fun doing it. In fact, Programming consumed most of my leisure thoughts. More about why I was doing it below, but I’ve been reading Programming Perl as well as skimming through Introducing Go and Learn You A Haskell for Great Good!. Ever since some folks used Haskell during last year’s Advent of Code and this guy’s videos that I mentioned in an early 2021 blog post, I’ve been very curious about the language. In fact, at this point I’ve decided that Go and Haskell will be the next two languages I learn. 

Python

I continued working on the Django app I’m making for a friend. I finally started to get the hang of how Django works, especially when passing information around in the app. I got to a minimal viable product (MVP) in April, but I’m hoping to finally get a 1.0 that meets all his needs within the first couple weeks of May.

Advent of Code 2015 Problem Set

So the activity that gave me the most fun in April was working through the 2015 Advent of Code problem set. I chose to use it to cement my understanding of Ruby, which I learned in 2020. But that wasn’t enough for me on its own. I decided to also solve each problem with Perl. I tried learning Perl for the second time about a decade ago. I ended up abandoning it for Python when I wanted to contribute to GRAMPS, which was written in Python. So I had some understanding of the strange sigils before the different types of variables, but I didn’t have a great understanding of the more complicated things that could be done with Perl. There’s also something I haven’t mentioned in this blog in a long time, but I think sometimes we come across knowledge before we’re ready for it. This isn’t some mystical woo-woo thing. It’s just a fact, sometimes you don’t have the underlying knowledge necessary to take advantage of new knowledge that you’re being presented with. A great example is hashes. I just couldn’t truly get my mind around it at the time (10 years or so ago). But now that I’ve done a ton of work with Python dictionaries, I realize they’re the same thing and hashes make sense to me now.

I’ve been extra happy in the fact that, by the end of April I had reached Advent of Code Day 10 and hadn’t had to skip any days. Yes, there were a couple days during the 2020 event where I skipped an entire problem or part 2 of a problem because I ran out of time, but I also had to skip a few problems because I had no idea how to solve them. So far that hasn’t happened with the 2015 problem set. I wanted to draw attention to some of the things I learned so far thanks to AoC 2015.

require "../../input_parsing/parse_input" 
 
def parse_directions(parenthesis) 
    floor = 0 
    parenthesis.each do |paren| 
    if paren == "(" 
        floor += 1 
    elsif paren == ")" 
        floor -= 1 
    end 
    end 
    return floor 
end 
 
 
 
input_text = input_per_line('../input.txt') 
puts parse_directions(input_text[0].split(//))

For my Day 1 entry to Ruby, I just practiced going over arrays with each. I also used string.split(//) to split the text into an array of each character. I made a comment on Reddit and someone suggested a more Rubyist solution:

require "../../input_parsing/parse_input" 
 
def parse_directions(parenthesis) 
    parenthesis.reduce(0) do | floor, paren | 
        if paren == "(" 
            floor + 1 
        elsif paren == ")" 
            floor - 1 
        end 
    end 
end 
 
 
 
input_text = input_per_line('../input.txt') 
puts parse_directions(input_text[0].split(//))

This simplifies the code by eliminating the need to initialize the floor variable to sum things up. Instead it becomes the input for an array that is reduced. (I would make more use of reduce later in 2015)

Day 2 

Just last year I started learning about what map and reduce mean in computer science. Such is the situation for someone who is primarily self-taught. Turns out that a pretty big chunk of Advent of Code (and other programming challenges) solutions can be summed up as map, filter, reduce. Of course, that hides the fact that figuring out the map and filter functions is often the whole point of the challenge. Day 2’s Ruby code was the first time I explicitly used map (see line 5 where I convert all the array members to integers) in any language (although a Python list comprehension is almost always an implicit map or filter)

require "../../input_parsing/parse_input" 
 
def get_dimensions(dimension_line) 
    dimensions = dimension_line.split('x') 
    dimensions.map(&:to_i) 
end 
 
 
def calculate_box_area(dimensions) 
    2*dimensions[0]*dimensions[1] + 2*dimensions[1]*dimensions[2] + 2*dimensions[0]*dimensio
ns[2]  
end 
 
def calculate_small_area(dimensions) 
    sorted_dimensions = dimensions.sort 
    small_area = sorted_dimensions[0] * sorted_dimensions[1] 
end 
     
 
input_text = input_per_line('../input.txt') 
all_box_areas = [] 
all_small_areas = [] 
input_text.each do |box| 
    all_box_areas.append(calculate_box_area(get_dimensions(box))) 
    all_small_areas.append(calculate_small_area(get_dimensions(box))) 
end 
 
summed_box_areas = all_box_areas.reduce(0) {|sum, num| sum + num} 
summed_small_areas = all_small_areas.reduce(0) {|sum, num| sum + num} 
 
puts "The Elves need #{summed_box_areas+summed_small_areas} square feet of wrapping paper!"

Day 2’s Perl code was the beginning of learning how to use groups in regular expressions to get values for variables in line 13. (This would become important in many future 2015 puzzles) Also the first time I used sorting on line 17

#!/usr/bin/perl 
 
use v5.14; 
 
open(PRESENTDIMENSIONS, , "../input.txt") || die "Can't open input.txt: $!\n"; 
 
my @present_dimension_list = <PRESENTDIMENSIONS>; 
 
my @accumulated_areas; 
 
for my $present_dimension (@present_dimension_list){ 
 
    my($length, $width, $height) = $present_dimension =~ /(\d+)x(\d+)x(\d+)/; 
     
    push (@accumulated_areas, 2*$length*$width+2*$width*$height+2*$height*$length); 
     
    my @temp_dimensions = sort{ $a <=> $b }($length, $width, $height); 
     
    push (@accumulated_areas, $temp_dimensions[0] * $temp_dimensions[1]); 
} 
 
# reduce is not in the standard Perl library 
 
my $total_area = 0; 
for my $area (@accumulated_areas){ 
    $total_area += $area; 
} 
 
say "The Elves need $total_area square feet of wrapping paper"

Day 4

Day 4 was pretty interesting for showing how a regular expression could make code very simple. Compare my Python and Perl code for that day:

import hashlib 
 
 
def calculate_hash(text_to_hash): 
    return hashlib.md5(text_to_hash.encode('utf-8')).hexdigest() 
 
 
def does_it_lead_with_five_zeroes(hash_string): 
    zero_count = 0 
    for character in hash_string: 
        if character == "0": 
            zero_count += 1 
        if character != "0": 
            break 
    return zero_count >= 5 
 
 
def find_special_number(puzzle_text): 
    decimal_number = 1 
    found_it = False 
    while not found_it: 
        decimal_number += 1 
        calculated_hash = calculate_hash(puzzle_text + str(decimal_number)) 
        found_it = does_it_lead_with_five_zeroes(calculated_hash) 
    return decimal_number 
 
 
if __name__ == "__main__": 
    puzzle_input = "iwrupvqb" 
    answer = find_special_number(puzzle_input) 
    print(f"The number to add is {answer}")
#!/usr/bin/perl 
 
use v5.14; 
 
use Digest::MD5 qw(md5_hex); 
 
my $puzzle_input = "iwrupvqb"; 
 
my $number = 0; 
 
my $hex_test = md5_hex("$puzzle_input$number"); 
 
until ($hex_test =~ /^00000/){ 
 
        $number++; 
        $hex_test = md5_hex("$puzzle_input$number"); 
} 
 
say "Santa's magic number is $number";

It’s pretty incredible how much simpler that is, right? It’s not that I couldn’t have done that with Python, it’s just that I didn’t have the idea until I was thinking about how I’d do the problem with Perl.

Day 7

Last year during the 2020 AoC I learned about memoization, but had trouble implementing it. This time I finally figured it out. It was built-in to Python via lrucache, but I had to implement it manually in Ruby. That really helped me prove to myself that I had learned how to do it.

from datetime import datetime 
from functools import lru_cache 
import re 
import sys 
sys.path.insert(0, '../../input_parsing') 
import parse_input 
 
all_wires = {} 
 
 
def create_dictionary(instructions): 
    """Take in instructions and place the connections into a dictionary entry for the wire. 
    eg: 123 -> x would lead to a key of x and a value of 123. 
    """ 
    wires = {} 
    for instruction in instructions: 
        pattern = re.compile(r'(.*) -> (\w+)') 
        regex = re.findall(pattern, instruction) 
        connection = regex[0][0] 
        wire = regex[0][1] 
        wires[wire] = connection 
    return wires 
 
 
def break_up_equation(equation): 
    """Figure out the operand(s) and operation and return them.""" 
    two_operand_regex = re.compile(r'(\w+) ([A-Z]*) (\w+)') 
    one_operand_regex = re.compile('([A-Z]*) ([a-z]+)') 
    if re.match(two_operand_regex, equation): 
        result = re.findall(two_operand_regex, equation) 
        return result[0][0], result[0][1], result[0][2] 
    elif re.match(one_operand_regex, equation): 
        result = re.findall(one_operand_regex, equation) 
        return result[0][0], result[0][1] 
    else: 
        print(f"{equation=}") 
        return [equation] 
 
 
@lru_cache() 
def find_value_on_line(wire_to_find): 
    """Figure out what the final value is on a wire.""" 
    if all_wires[wire_to_find].isnumeric(): 
        return int(all_wires[wire_to_find]) 
    equation = break_up_equation(all_wires[wire_to_find]) 
    if len(equation) == 3: 
        if equation[0].isnumeric(): 
            operand_left = int(equation[0]) 
        else: 
            operand_left = find_value_on_line(equation[0]) 
        if equation[2].isnumeric(): 
            operand_right = int(equation[2]) 
        else: 
            operand_right = find_value_on_line(equation[2]) 
        operation = equation[1] 
        if operation == "AND": 
            return operand_left & operand_right 
        elif operation == "LSHIFT": 
            return operand_left << operand_right 
        elif operation == "OR": 
            return operand_left | operand_right 
        elif operation == "RSHIFT": 
            return operand_left >> operand_right 
    elif len(equation) == 2: 
        return find_value_on_line(equation[1]) ^ 65535 
    else: 
        print("one went straight through") 
        return find_value_on_line(equation[0]) 
 
 
if __name__ == "__main__": 
    print(f"Starting at {datetime.now().strftime('%d-%b-%Y %H:%M:%S')}") 
    bobby_instructions = parse_input.input_per_line('../input.txt') 
    all_wires = create_dictionary(bobby_instructions) 
    wire_a = find_value_on_line('a') 
    print(f"The value on wire a is {wire_a}") 
    print(f"Ended at {datetime.now().strftime('%d-%b-%Y %H:%M:%S')}")
require "../../input_parsing/parse_input" 
 
def create_dictionary(instructions) 
    wires = Hash.new 
    instructions.each do |instruction| 
        match_results = instruction.scan(/(.*) -> (\w+)/) 
        connection = match_results[0][0] 
        wire = match_results[0][1] 
        wires[wire] = connection 
    end 
    wires 
end 
 
def break_up_equation(equation) 
    if equation.match?(/(\w+) ([A-Z]*) (\w+)/) 
        broken_equation = equation.scan(/(\w+) ([A-Z]*) (\w+)/) 
        #puts "A 3 way match: #{broken_equation[0][0]}, #{broken_equation[0][1]}, #{broken_e
quation[0][2]}" 
        return broken_equation[0][0], broken_equation[0][1], broken_equation[0][2] 
    elsif equation.match?(/([A-Z]*) ([a-z]+)/) 
        broken_equation = equation.scan(/([A-Z]*) ([a-z]+)/) 
        #puts "A 2 way match: #{broken_equation[0][0]}, #{broken_equation[0][1]}" 
        return broken_equation[0][0], broken_equation[0][1] 
    else 
        return [equation] 
    end 
end 
 
@all_wires = Hash.new 
@cache = Hash.new 
 
def for_cache(wire_to_find) 
#def find_value_on_line(wire_to_find) 
    int = Integer(@all_wires[wire_to_find], exception: false) 
    return int if int 
    #puts "#{@all_wires[wire_to_find]}" 
    equation = break_up_equation(@all_wires[wire_to_find]) 
    #puts "equation is #{equation[0]} #{equation[1]} #{equation[2]}" 
    if equation.length() == 3 
        operand_left = Integer(equation[0], exception: false) 
        if operand_left 
            #puts "L: #{operand_left} is a number" 
        else 
            #puts "L: #{equation[0]} is not a number" 
            operand_left = find_value_on_line(equation[0]) 
            #puts "Now operand_left is #{operand_left}" 
        end 
        operand_right = Integer(equation[2], exception: false) 
        if operand_right 
            #puts "R: #{operand_right} is a number" 
        else 
            #puts "R: #{equation[2]} is not a number" 
            operand_right = find_value_on_line(equation[2]) 
            #puts "Now operand_right is #{operand_right}" 
        end 
        operation = equation[1] 
        case operation 
        when "AND" 
            return operand_left & operand_right 
        when "LSHIFT" 
            return operand_left << operand_right 
        when "OR" 
            return operand_left | operand_right 
        when "RSHIFT" 
            return operand_left >> operand_right 
        end 
    elsif equation.length() == 2 
        return find_value_on_line(equation[1]) ^ 65535 
    else 
        return find_value_on_line(equation[0]) 
    end 
end 
 
def find_value_on_line(wire_to_find) 
    #puts "Cache value is #{@cache[wire_to_find]}" 
    @cache[wire_to_find] ||= for_cache(wire_to_find) 
end 
 
 
if $PROGRAM_NAME == __FILE__ 
    instructions = input_per_line('../input.txt') 
    #instructions = ['123 -> x', '456 -> y', 'x AND y -> d', 'x OR y -> e', 'x LSHIFT 2 -> f
', 'y RSHIFT 2 -> g','NOT x -> h', 'NOT y -> i'] 
    @all_wires = create_dictionary(instructions) 
    answer = find_value_on_line('a') 
    #answer = for_cache('a') 
    puts answer 
end

Day 9

Solving Day 9 was an oddessy that took me a couple days to fully solve. When I first read the problem, it reminded me of something I remembered from undergrad. I couldn’t remember if it was from the second semester of programming or discrete math, but I knew that once upon a time I’d been taught the algorithm to solve this problem. Some searching revealed that it was Djikstra’s Algorithm that I’d been thinking of. That turned out not quite to be what I wanted because Djikstra is for when you have a specified start and end node. So I did a little more sleuthing and thought perhaps what I wanted to solve for was a Hamiltonian Walk. That tells you if there’s a path that will reach every node and what the length is. So that got me most of the way there, but what I wanted was the shortest path. So that could be accomplished via the Traveling Salesman solution except that the Traveling Salesman comes back home and in this problem, Santa was only supposed to visit each city once. The solution in that case is to create a fake city that has 0 distance to all other cities. (basically, take the matrix that defines the city distances and add an extra row and column of zeroes to the bottom and right side).

My AoC code for all years can be found at this Github repo.

Microsoft MakeCode

Stella wanted a little wheeled bot of her own so I got her some assorted pieces to build a similar bot to Sam’s without it being exactly the same. I also wanted to take advantage of the Circuit Playground Express and Cricket we already had. Unfortunately, I haven’t been able to figure out the ultrasonic sensor yet. But I hope to have her robot working in May.

Sam’s Thought Processes

As I’ve mentioned on this blog before, I always find it fascinating how the kids interpret the world. Here are a couple examples of how Sam is seeing things at this point.

LEGO Ban

Last weekend I was sitting at the computer, working on some code when my wife came into the room. “Did you tell Sam that he couldn’t play LEGOs?”

“Of course not,” I retorted.

“Did you tell him he couldn’t be in the basement, then?”

“Nope”

After a couple more questions, she cuts to the chase. “What exactly did you tell Sam when he came in to see you?”

“I told him no more screen time.” Sam had been playing video games for a while as well as watching some TV.  Well, now that LEGO doesn’t include instruction manuals for their large sets, you have to look up the instructions online. So he interpreted no screens as meaning he couldn’t use the tablet to look up the instructions for his LEGOs. Instead of asking if the tablet was OK for LEGOs, he was just complaining that I was banning him from LEGOs, leading to the confusion.

Peep and Egg

Now that the twins can read, they’ve been re-reading some of the books we’d previously read to them. One of the ones they’ve been returning to is called Peep and Egg: I’m not Hatching. It has a series of pages like these:

The takeaway is supposed to be that the egg is comfortable with the way things are, but could be having so much more fun if they’d come out of their comfort zone. Sam’s told us that the lesson is that Peep should learn to be more patient with Egg and let him do things at his own pace.

Mozilla’s Legacy

A few days ago I read this article over at Tech Republic about how, Mozilla’s greatest achievement is not Firefox, but the Rust programming language. They point to Firefox’s declining numbers in the face of Chrome and Chromium-based browsers and I’m inclined to agree with the author. There is, of course, a kind of poetry to this. Although Netscape was one of the first dot-com companies and beat Microsoft to the punch at creating the first mainstream web browser, it’s not Netscape Navigator which is its greatest legacy. Instead it’s spinning off into Mozilla and, the most poetic part, the creation of the Javascript programming language. (Javascript was written in just a week and a half and this episode of Red Hat’s Commandline Heroes podcast does an excellent job documenting it)

It’s also not unprecedented in the tech world – Bell Labs, the research arm of AT&T’s greatest contributions to tech have nothing to do with telephones. Their researchers invented transistors (basis of modern computers), the C programming language (pretty much every operating system and most video games until recently were written in C), the UNIX operating system (between UNIX and its kinda-descendent Linux – a good portion of the research computers and the Internet run on these OSes), and many other technologies. So it wouldn’t be the worst legacy for Mozilla to have. 

Rust, in case you’re unfamiliar, is a low-level language like C but it has safety mechanisms that should help reduce the number of bugs that can be exploited by the bad guys – at least at the operating system and driver level. Google has indicated it will begin using Rust in Android. Linus Torvalds has mentioned he’s not opposed to it in the Linux kernel. I believe I’ve also seen articles saying that Microsoft is considering using it for future versions of Windows. 

I don’t think it would be a good thing for us to end up back in a place where there’s only one browser – Chromium and its clones, but if Firefox dies, at least Mozilla will have given the world Rust. (Also, Firefox was originally called Phoenix… just as it rose from Netscape’s ashes, maybe something else would rise from Firefox)

OMG: All Your Base is 20 Years Old

As I was going through my feed reader recently, I came across an article from Ars Technica, that I’d skipped over when it first came out, which announced that the All Your Base Meme is now 20 years old. I couldn’t believe it. It was my first meme, a few years before Numa Numa would be the meme that crossed over into regular pop culture. It led me to The Laziest Men on Mars’ page on Mp3.com, which was a kind of proto-Bandcamp in the early 2000s where indie bands (and some commercial bands) would put up MP3s to gain followers. (This ended up being my favorite song from The Laziest Men on Mars) Here is the video that took over all of us on the Internet 20 years ago:

Yeah, I’m sure you’re wondering how in the world that was a thing anyone cared about. But, much in the same way that I wonder how anyone could ever enjoy most of early cinema, it was a different time on the Internet then. A time when putting together an amateur video and getting it to everyone was just starting to be feasible. And it got so embedded into my brain that about six so years ago I got the following shirt when I went to the Defcon conference:

all your base shirt_
all your base shirt

Yeah, there’s a lot of geekiness contained in that one shirt.

Anyway, hope you enjoyed that trip down memory lane or, if you’re a young millenial or Gen Z are shaking your head at what we late Gen X/early millenials found entertaining.

Programming Projects: March 2021

I started off the month thinking it was going to be Python heavy and ended up doing a lot more micro-controller programming. To be fair, I was mostly programming in CircuitPython, but it definitely came out of nowhere.

Python

Civilization VI Webhook in FastAPI

Last month I created a webhook program in Flask to generate notifications when it was someone’s turn in our multiplayer Civilization games. When I posted about it on reddit, someone suggested that I would be better off using FastAPI instead of Flask. I’d been hearing about FastAPI for months on Talk Python to Me and Python Bytes, but I always got the impression that it would only be useful if I was writing my website to be async. 

Instead what I got was a much cleaner interface, an automated documentation system, and better use of types to validate data. I’m only using a fraction of what’s nice about FastAPI and I find it much nicer to use for this use case than Flask. After I got it working as well as when I was using Flask, I started working on enhancements and bug fixes. 

The work that drove me nuts until I figured it out was how to clean up my code (that is, refactor to reduce code reuse) and still maintain the ability to not have duplicate notifications from the Civilization Play By Cloud service. It turned out that I needed to be faster than 0.01 seconds to handle the code. So I had to write the new turn to the dictionary that’s functioning as a sort of bootleg database BEFORE posting to Matrix. It was quite satisfying to figure that out.

I then focused on moving some items (like usernames) out to config files so that it would be easy for someone to take my code and use it for themselves. I’m almost done with that. Afterwards I’ll create a PyPi package.

The github repo is here.

Django 2 by Example and Prophecy Practicum Django Rewrite

On the other end of the Python web frameworks, I also did some work with Django. When I first tried to learn Django a few years ago it was just too complex for me to understand. I was trying to learn too many concepts at once. Now that I understand decorators and, thanks to my work with Flask and FastAPI, the concepts of routes, it’s a lot easier for me to learn Django. The MVC framework still separates things a lot, causing a higher cognitive load. However, I can see the beauty in how it separates things and makes it easier for teams to work on the code. 

Once I had enough of the basics under my belt, I took another look at the Prophecy Practicum app I was writing for a friend. My work on creating it with Flask ground to a halt under the weight of all the extra work I was doing to develop things like an admin view. Eventually, I realized that Django may be a heavy framework for the way a lot of web development is done nowadays (feed APIs to phone apps and Javascript libraries), but for what I wanted to do, it would reduce a lot of the work I was doing because it came with those libraries built in. So I’ve been able to almost get to a working first stab at things with only a couple hours’ work. It’s a lot nicer to focus on what I want the app to do rather than having to build up a bunch of scaffolding. Also, I love the way it handles creating databases from classes.

CircuitPython

Play Dough Piano

When reading through the Adafruit Python on Microcontrollers mailing list, I found out about a Play Dough Piano (unfortunately I couldn’t figure out how to download that presentation). I copied their CircuitPython code since I don’t have any experience with CircuitPython HID. But I developed the Scratch code on my own thanks to all the experience I gained last year with Scratch. I also used my recent experience with soldering to add the pins to the QTPy. If you decide to do this yourself, it’s important to note that you cannot use a breadboard because the capacitive touch pins on the QTPy reads as a key being pressed. The kids had a lot of fun and it finally gave me an idea of what to make with a QTPy. I bought 2 of them when they were half off during John Park’s show on Adafruit. The code is available here.

QTPy Streamdeck

I don’t need a slick Streamdeck, just the ability to be able to take a few actions while in-game.

Working on the Circuit Python Piano with Sam combined with seeing someone make a Streamdeck using Cherry MX keys make me consider that perhaps I could use my QTPy to create a Streamdeck. I’d previously been focused on recreating the ElGato Streamdeck, so I was going to use a PyPortal. However, a PyPortal is $50-ish and I don’t have a 3D printer to make an enclosure. Meanwhile I already had the QTPy and some buttons. I didn’t need the buttons to change or have images on them, so I went ahead and created a nice, simple Streamdeck. My first version only worked if I had OBS in focus which defeated the purpose. I was using key.send. Afterwards speaking to someone in the Adafruit Discord, I learned that I had to do press and release. Now it works awesomely and I can start recording and pause/unpause the recording without leaving the game.

The repo is available here.

Microsoft MakeCode

Sam’s car

I got tired of waiting for the BBC Micro:Bit V2 to be in stock at Adafruit. It’s the same price as the BBC Micro:Bit V1, but with more features, so it felt silly to get the earlier version. But I got tired of not being able to build the CuteBot with Sam, so I bit the bullet and bought the version 1. It was very easy to program in Microsoft MakeCode because there are special blocks for programming the car. We were able to program the car to “dance like a bee”, avoid obstacles, and follow a path. The latter was particularly impressive to the wife. The kids were amused that their stuffed penguins were invisible to the car’s eyes. That is, when in obstacle avoidance mode, it would still crash into the penguins. I thought that was a little odd myself. But then I did some research and realized the “eyes” were an ultrasonic sensor. Since it’s using echo-location, it doesn’t work on furry objects – the same reason that padding is used to improve the acoustics in a room by dampening the noise. The object needed to be reasonably solid to let the car know it was there. I still have some more tinkering to do with Sam to add some more options to the examples the car’s manufacturer printed in the manual.

C#

GameDev.Tv Multiplayer class

I finished up the online multiplayer video game class I was taking through GameDev.Tv. As I’ve said in previous updates – I now truly appreciate the work that goes into an online multiplayer game. It’ll be a while before I add multiplayer to the game I’ve been working on, but I’ll definitely be studying what I did in this class when the time comes. I highly recommend it if you’re considering making an online multiplayer game for the first time – especially if you want to do it over Steam.

My Extra Life Donation Tracker reaches v6.0 (feature complete)

This is my seventh year raising money for Johns Hopkins Children’s Center via the Extra Life gaming charity. When I started back then I was brand new to streaming or recording video game play. In fact, just a few years before that I hadn’t even understood the point. I found out that you could somehow display your progress towards fundraising on your screen while you played. I had no idea how to do that and, at the time, I’d only done commandline programs. So I found bfinleyui’s web-based program. I set up XSplit (which I was using at the time) to capture the web pages and was able to have a great first year. The following year he created an app with Adobe Air that was even better. The GUI for this app would become the inspiration for ElDonationTracker; what I aspired to eventually build. My first commit to the repo for ElDonationTracker was that year. I was exploring how to get the API data via the commandline, but I wasn’t too hurried because I had the Adobe Air app. Unfortunately, the following year Adobe killed Adobe Air. So it was now time for me to try and make my own app. Unfortunately, I’d never done GUI programming before and my experiment with the TKinter GUI framework didn’t quite work.

I spent a year or two perfecting the API part of the code. I would run it on the commandline and use the output for my streams. It worked on both Linux and Windows and things were pretty good. But I still didn’t have the GUI. This meant I still didn’t have the “you got a donation” notification with an image and sound file. This was what I missed most from bfinleyui’s GUI app.

Extra Life Donation Tracker Donation Alert window
Extra Life Donation Tracker Donation Alert window

Finally, a few years ago I discovered how to easily code with the QT GUI framework thanks to a book I’d read and the discovery of the QT Designer program. All I had to do was create a GUI with drag and drop and then use pyuic to turn it into Python code. From there I could add my logic and the first GUI was born.

I then spent the next few years perfecting the code. There are a few issues in the Github repo documenting various bugs I’ve had to iron out. I’ve also added unit tests to try and capture regressions and other issues.

With 6.0 I have finally met all the goals I set out to reach 6 years ago. It’s a great-working GUI that provides everything I need for streaming for Extra Life. I have other users (evidenced by bug reports) and that really makes my day. During the 2020 Extra Life Day, the Streamlabs integration (which has displaced the need for programs like this) failed, but my program continued to work. Huzzah!

Extra Life Donation Tracker main GUI window
Extra Life Donation Tracker main GUI window

One thing I made a key goal for 6.0 was structuring the project so that others could use the Donor Drive API I’ve built to create other Python Projects. In order to really drive this home, I may end up separating that code out into another package, but I hadn’t decided at this point.

Now that I’ve finally reached my goal, it may be a while before I reach 7.0 or even anything past 6.1. I will focus on improving the unit tests and, if anyone requests it, new outputs. It’s been incredible to work on a project for 6 years and finally reach a finishing point. It’s my longest running, most complex, and most used project and it’s the one I’m most proud of.

Looking back at a Year of COVID-19

It seems that it’s time to look back at a year of COVID-19. Scalzi did it. Ars Technica did it. I’m not usually a huge bandwagon jumper, but I thought, “why not?” This has been a huge, disruptive event. It might be therapeutic to write about it.


It was 11 March (I believe) when the WHO officially declared COVID-19 to be a pandemic. One of the first things I remember hearing about COVID-19 was the cruise ship that was stuck offshore because President Trump was so afraid of the numbers rising that he couldn’t let the folks off the ship. Turns out that was going to be a foreshadowing the likes of which we couldn’t quite understand at the time. My first inkling that maybe this was starting to become an issue in the US was in late February or early March when we went to visit relatives in New York City and they were against eating out because they were scared of getting sick. At the time, we thought they were just being a bit germaphobic. In fact, at the end of March we went to a restaurant for what we didn’t know at the time would be our last time to go out to eat for a year. (To date we still haven’t eaten out, not even for outdoor dining)

After that, of course, was the infamous “masks aren’t necessary” phase of the pandemic. So we just tried to limit our trips to the stores, but didn’t really worry about masks. Then work had us wear masks in public areas, but not at our desks. Eventually, it was all masks all the time. Which is where we remain today.

I got something around 1-2 months off of work during peak pandemic. Also, from not going anywhere on vacation, I ended the year with a ton of vacation days I couldn’t carry over. So I was able to dedicate a few days to programming projects and electronics projects. By being able to work continuously for many hours rather than coming to it at the end of a work day and spending a chunk of time remembering what I was working on, I got a lot further with my projects. I also spent a lot of time playing video games with the kids. I didn’t get as much cooking done as I would have thought at first because in the early pandemic there were constant food panics that made finding exactly what I wanted very hard. During the sour dough bread boom it was hard to find flour, for example. We also tackled some time-consuming house tasks like staining the deck or painting the porch columns.

I don’t understand the folks who have been complaining of boredom; of finishing Netflix, etc. Am I the only person in the world who has hobbies?

For me, the biggest negative from the pandemic was not getting to see my family. I usually travel to see my direct family 2-3 times a year and my in-laws another 2-3 times. And they usually each head this way a time or three. That’s basically it. I’m mostly an introvert. I’ll have fun at a work picnic or a family get-together, but I don’t need or crave human interaction. Between work and the wife and kids, I don’t have any needs outside of that. As for restaurants, I go out to eat when the wife desires a different atmosphere, but I don’t really enjoy eating out. Unless we’re spending a ton of money, the food is never better than what either my wife or I could cook. It’s also inevitably going to be less healthy. The only place worth going out to IMHO is Korean because making a ton of ban chan (the little side dishes) is more trouble than it’s worth for just one meal.

I have lots of sympathy for those who have lost their businesses. Marketplace has been featuring interviews with folks who happened to think January 2021 was a great time to open the business they’ve been dying to open their entire lives. I have sympathy for the folks who work in the restaurant and bar industries that have lost their jobs and income. If we put those folks aside, I don’t understand the folks who have been complaining of boredom; of finishing Netflix, etc. Am I the only person in the world who has hobbies? Even with the aforementioned 2 months off of work, I’ve still got TONS I could do if someone were to pay me my current salary and tell me to stay at home.

From various Humble Bundle purchases, I’ve got about 350 video games to play. (If you don’t know – via Humble Bundle you can usually get 10-20 games for about $20) Most of them I’ve never played. Many, like Civilization or Spelunky, are infinitely replayable. I have over 1000 digital books and magazines to read. I have a bunch of programming projects. I have programming languages I’d like to learn. I have a backlog of photos to edit and/or tag. I have electronics projects to work on. There are over 200 new recipes I’d like to try. There are somewhere between 200 and 300 comic book issues I digitally have that I haven’t read yet. I have a bunch of classes to learn how to program video games. Scarlett and I started taking a drawing class. I have TV shows and movies that I wouldn’t mind seeing. Just those things together would probably be a full-time job for a couple years. On top of that, I have kids to raise and play with. A wife to be a husband to. And I know lots of folks are doing the “bubble” thing with their parents – if I were doing that, too then I’d have parents and 5 younger siblings to grow my relationship with. I don’t get it. Are most Americans that caught up in our Puritan heritage that they can’t have fun without work? I see especially see major fulfillment when I program because I’m creating things that others find useful. It’s very interesting.

They always say that folks die soon after retirement because they don’t have anything to do and they almost literally die of boredom. I don’t see that being a problem with me.

Well, a year out it looks like we may be turning the corner on the pandemic phase of COVID-19. Some scientists are saying it may be here to stay with us. COVID-19 may become like the common cold or the flu. Every winter we get some shots to keep us from either getting it or getting a mild case if we do get it. And that might be the best we can do. I’m fine with that if it means I can feel safe traveling to see my parents and my in-laws. It was kinda crappy to see the country fall apart in a time where we should have banded together. I’m hoping (perhaps stupidly) that this was merely a failure of leadership and not a true semi-permanent state of being here in the US. The scientists say these pandemics may become more frequent as climate change pushes different animals into contact with us and zoonotic transfers become more common. I’d like to hope that it doesn’t matter if we have a Republican or Democratic president – next time we’ll do a better job.

We’ll just have to see.

Programming Jan/Feb 2021

I was pretty busy programming at the start of 2021 across a few different languages. Let’s jump right in!

C#

I’m nearing the end of the GameDev.tv online RTS course, and it’s been a lot of fun. Since last time we added player colors to the units, a minimap that can be used to move around the screen, new units, and a Lobby UI. I’m a few lessons away from being able to create binaries I can use to play online with others or via Steam.

MS MakeCode

I wanted to try a new electronic project with Stella, so I fired up the tutorial for the Circuit Playground chairs swing ride. I had her cut out the figures that would go on the ride. Then I used the glue gun to put everything together. After all the parts were ready, I had Stella do the programming since the kids are used to programming in Scratch and MS MakeCode is basically Scratch plus the ability to do Javascript. 

Python

Python Morsels

I completed my final assignment for the Python Morsels exercises. This time we had to take in a list of timestamps and sum them together. This time it wasn’t only a great exercise to learn and practice with Python, but it’s also a utility I can use in various projects. I immediately realized it could have made summing up the times for my end of year video games blog post a lot easier. Here’s the solution I came up with:

import math 
 
 
def figure_out_base_sixty(number: int) -> (int, int): 
    """Figure out the next number up if I have more than 59 seconds or minutes.""" 
    if number > 59: 
        return math.floor(number/60), number % 60 
    else: 
        return 0, number 
 
 
def sum_timestamps(timestamps: list) -> str: 
    """Accept a list of timestamps (in the format of MM:SS or HH:MM:SS) and return a timestamp that is the sum of all 
    given times. 
 
    :param timestamps: A list of timestamps. 
    :type timestamps: list 
    :returns: Timestamp sum of all times in the list. 
    """ 
    split_time = [timestamp.split(':') for timestamp in timestamps] 
    hours = [] 
    minutes = [] 
    seconds = [] 
    for timestamp in split_time: 
        if len(timestamp) == 2: 
            minutes.append(int(timestamp[0])) 
            seconds.append(int(timestamp[1])) 
        else: 
            hours.append((int(timestamp[0]))) 
            minutes.append(int(timestamp[1])) 
            seconds.append(int(timestamp[2])) 
    total_hours_step_1 = sum(hours) 
    total_minutes_step_1 = sum(minutes) 
    total_seconds_step_1 = sum(seconds) 
    minutes_to_add, total_seconds_final = figure_out_base_sixty(total_seconds_step_1) 
    total_minutes_step2 = total_minutes_step_1 + minutes_to_add 
    if total_minutes_step2 > 59 or hours: 
        calculated_hours, total_minutes = figure_out_base_sixty(total_minutes_step2) 
        final_hours = calculated_hours + total_hours_step_1 
        return f"{final_hours}:{total_minutes:02d}:{total_seconds_final:02d}" 
    else: 
        total_minutes = total_minutes_step2 
        return f"{total_minutes}:{total_seconds_final:02d}"

After all this time of learning, my solution was actually pretty close to Trey’s solution, and that really made me feel great.

Pybites

In the same Humble Bundle where I got the Python Morsels exercise credits, I also got access to Pybites. I’ve done about 7 or 8 of them so far. The platform is nice, with a built-in Python interpreter and a good community. But if I compare it to Python Morsels, I don’t like it as much for learning. They give you a problem and provide unit tests to make sure your code works. But, unlike Trey – if you need help the solution is simply stated. Trey’s solutions, on the other hand, contain a lot of commentary that help to explain why it’s the best answer. He also covers answers that work, but aren’t as pythonic. His bonus questions build on the base problem in a way that really encourages learning. Pybites isn’t bad, but after doing Python Morsels – if I were going to pay for one of them, I’d pay for Python Morsels.

Civilization VI Play by Cloud Webhook

I’m a little late to this, but in the final weekend of February 2021 my brothers and I started playing Civilization VI using Play by Cloud. Previously, the only way to play asynchronously was to use an external service like Play Your Damn Turn. (We used to use another one with Civ V, but Play your Damn Turn works with Civ V, Civ VI, and Beyond Earth). Play Your Damn Turn (PYDT) has a neat little app that runs on your computer to let you know when there’s another turn. Play By Cloud, which Firaxis integrated into Civilization VI in a February 2019 update, just uses Steam updates. Dan and Dave have been using Play by Cloud for their newest games, but this weekend they invited me to start a game on there, too. For some reason, Steam updates didn’t quite work all that well to let me know when it was time to take my turn, but I found out you can set up webhooks to get informed by Civilization when there’s a new turn available. Almost all the examples I found online involved setting up the webhooks to push alerts to Discord. I’ve got a Matrix server, so I spent the weekend writing up a program to handle it for us.

When I was working on it, I didn’t create a perfectly sanitized program without any info I wouldn’t want out there, so I’m not ready to put it on Github, but I can share some sanitized code here. 

When I was trying to develop, it took some searching to find out what data is sent by Civilization VI. It sends JSON:

{
"value1": "the name of your game",
"value2": "the player's Steam name",
"value3": "the turn number"
}

While working on the code, I found out that you can also set up webhooks with PYDT. They actually send even more data (with actually useful names!):

{
"gameName": "the name of your game",
"userName": "the user's Steam username",
"round": the round number (int),
"civName": "the name of your civilization",
"leaderName": "the name of your civ leader (and for some of them the attribute)"
}

The first thing you need when dealing with webhooks, is a place that can handle a POST command on the net. This was absolutely perfect with Flask as Django would be way heavier than we need to just accept some POST requests.

from flask import Flask, request, Response, jsonify 
import json 
import logging 
 
import matrix_bot 
 
app = Flask(__name__) 
flask_matrix_bot = matrix_bot.MatrixBot() 
most_recent_games = dict() 
logging.basicConfig(level=logging.DEBUG, format='%(levelname)s- %(asctime)s - %(message)s') 
 
try: 
    with open('most_recent_games.json', 'r') as file: 
        most_recent_games = json.load(file) 
        logging.debug("JSON file loaded.") 
except FileNotFoundError: 
    logging.warning("Prior JSON file not found. If this is your first run, this is OK.") 
 
 
def player_name_to_matrix_name(player_name: str) -> str: 
    if player_name == "One Oh Eight": 
        return "Dan" 
    elif player_name == "TheDJOtaku": 
        return "Eric" 
    elif player_name == "Wedge": 
        return "David" 
    else: 
        return player_name 
 
 
@app.route('/webhook', methods=['POST']) 
def respond(): 
    logging.debug(f'JSON from Play By Cloud: {request.json}') 
    game_name = request.json.get('value1') 
    player_name = player_name_to_matrix_name(request.json.get('value2')) 
    turn_number = request.json.get('value3') 
    if game_name in most_recent_games.keys(): 
        if most_recent_games[game_name]['player_name'] != player_name: 
            logging.debug("Game exists, but this is not a duplicate") 
            message = f"Hey, {player_name}, it's your turn in {game_name}. The game is on turn {turn_number}" 
            flask_matrix_bot.main(message) 
            most_recent_games[game_name] = {'player_name': player_name, 'turn_number': turn_number} 
        else: 
            logging.debug("Game exists and this is a duplicate entry.") 
    else: 
        most_recent_games[game_name] = {'player_name': player_name, 'turn_number': turn_number} 
        logging.debug("New game.") 
        message = f"Hey, {player_name}, it's your turn in {game_name}. The game is on turn {turn_number}" 
        flask_matrix_bot.main(message) 
    with open('most_recent_games.json', 'w') as file: 
        json.dump(most_recent_games, file) 
    return Response(status=200) 
 
 
@app.route('/pydt', methods=['POST']) 
def respond_pydt(): 
    logging.debug(f'JSON from PYDT: {request.json}') 
    game_name = request.json.get('gameName') 
    player_name = player_name_to_matrix_name(request.json.get('userName')) 
    turn_number = request.json.get('round') 
    civ_name = request.json.get('civName') 
    leader_name = request.json.get('leaderName') 
    message = f"Hey, {player_name}, {leader_name} is waiting for you to command {civ_name} in {game_name}. " \ 
              f"The game is on turn {turn_number}" 
    flask_matrix_bot.main(message) 
    most_recent_games[game_name] = {'player_name': player_name, 'turn_number': turn_number} 
    with open('most_recent_games.json', 'w') as file: 
        json.dump(most_recent_games, file) 
    return Response(status=200) 
 
 
@app.route('/recent_games', methods=['GET']) 
def return_recent_games(): 
    return jsonify(most_recent_games)

First, lines 12-17, I check for the most recent games that I’ve saved out to a JSON file. This allows for persistence if I have to stop the program or it crashes or whatever. Then, lines 20-28, I also created a function to convert from Steam screennames to Matrix names so that when the Matrix bot pushes the data, it’ll come out as a mention to the user and draw their attention to the turn they need to play. Here’s what it looks like:

Hey, Eric, Montezuma is waiting for you to command Aztec in Gathering Storm! 1. The game is on turn 59

Hey, Eric, Gilgamesh is waiting for you to command Sumeria in Mesas Play . The game is on turn 161

Hey, Eric, it's your turn in Eric and Dan Duel. The game is on turn 9

You can see in the first two I can make use of the extra data PYDT gives us. The last one is Play By Cloud, giving us less to work with.

I create two endpoints – one for Civilization VI (starting at line 31) and one for PYDT (line 55).  The Civilization VI endpoint (webhook) has deduplication code because if each person has it set to push out the JSON on all turns, it would end up repeating the alert.

I also added the recent_games endpoints so that I could create another matrix bot that would listen for commands and let users check up on their games. 

"""Provide ability to listen for commands from Matrix.""" 
 
import asyncio 
import json 
import logging 
from nio import AsyncClient 
import requests 
 
# testing 
# URL = "http://localhost:5000/recent_games" 
# production 
URL = "<YOUR SERVER URL>" 
 
logging.basicConfig(level=logging.INFO, format='%(levelname)s - %(asctime)s - %(message)s') 
 
 
class ListenerMatrixBot: 
    """A bot to send alerts about the game to Matrix""" 
    def __init__(self): 
        try: 
            with open('matrix.conf') as file: 
                self.config = json.load(file) 
                logging.debug("Listener Matrix Config loaded.") 
                file.close() 
        except FileNotFoundError: 
            logging.warning(f"Settings not found.") 
 
    async def login(self): 
        client = AsyncClient(self.config.get('server'), self.config.get('username')) 
        response = await client.login(password=self.config.get("password")) 
        logging.info(f"Listener Login response: {response}") 
        logging.debug(f"Listener Room would be: {self.config.get('room')}") 
        return client 
 
    @staticmethod 
    def get_current_games(): 
        """Get the list of games from the recent games endpoint. 
 
        :returns: A dictionary of the games, next player, and turn number. 
        """ 
        response = requests.get(URL) 
        return dict(response.json()) 
 
    def format_current_games(self): 
        """Format the list of current games for display in Matrix server.""" 
        return_text = "Here is a list of the games currently known about on the server:\n" 
        response_dictionary = self.get_current_games() 
        for key in response_dictionary: 
            game = key 
            player = response_dictionary[key].get('player_name') 
            turn_number = response_dictionary[key].get('turn_number') 
            return_text += f"{game} awaiting turn {turn_number} by {player}\n" 
            logging.debug(return_text) 
        return return_text 
 
    def format_blame_games(self, player_name: str) -> str: 
        number_of_games = 0 
        return_text = "" 
        response_dictionary = self.get_current_games() 
        for key in response_dictionary: 
            game = key 
            turn_number = response_dictionary[key].get('turn_number') 
            if response_dictionary[key].get('player_name') == player_name: 
                return_text += f"{game} awaiting turn {turn_number} by {player_name}\n" 
                number_of_games += 1 
        if number_of_games > 0: 
            return f"There are {number_of_games} games waiting for {player_name} to take their turn:\n" + return_text 
        else: 
            return f"There aren't any games waiting for {player_name}. Great job!" 
 
    async def main(self): 
        my_client = await self.login() 
        with open('next_batch', 'r') as next_batch_token: 
            my_client.next_batch = next_batch_token.read() 
        while True: 
            sync_response = await my_client.sync(30000) 
            with open('next_batch', 'w') as next_batch_token: 
                next_batch_token.write(sync_response.next_batch) 
            if len(sync_response.rooms.join) > 0: 
                joins = sync_response.rooms.join 
                for room_id in joins: 
                    for event in joins[room_id].timeline.events: 
                        if hasattr(event, 'body'): 
                            if event.body.startswith("!Civ_Bot current games"): 
                                data_to_send = self.format_current_games() 
                                logging.debug(data_to_send) 
                                content = {"body": data_to_send, "msgtype": "m.text"} 
                                await my_client.room_send(room_id, 'm.room.message', content) 
                            elif event.body.startswith("!Civ_Bot help"): 
                                data_to_send = ("""Current Commands: 
                            !Civ_Bot help - this message 
                            !Civ_Bot current games - the list of games Civ_Bot currently knows about. 
                            !Civ_Bot blame <Matrix Username> - list of games waiting on that person.""") 
                                content = {"body": data_to_send, "msgtype": "m.text"} 
                                await my_client.room_send(room_id, 'm.room.message', content) 
                            elif event.body.startswith("!Civ_Bot blame"): 
                                player = event.body.lstrip("!Civ_Bot blame ") 
                                data_to_send = self.format_blame_games(player) 
                                logging.debug(data_to_send) 
                                content = {"body": data_to_send, "msgtype": "m.text"} 
                                await my_client.room_send(room_id, 'm.room.message', content) 
 
 
my_matrix_bot = ListenerMatrixBot() 
asyncio.run(my_matrix_bot.main())
 

For the weekend or evening, I could get to the game soon after getting notified. But on weekdays, my brothers might play their turns while I’m at work and then by the time I get home (or even a few days later) I might forget whether there are games waiting for me. So I set up the listener to allow me to either ask it about all the games outstanding or just my own by using !Civ_Bot blame Eric.

Overall, I was pretty darned happy with how it came out for just a weekend’s worth of work.

Impractical Python Chapter Projects

I did a few more chapters of the Impractical Python book. Specifically we worked on Genetic Algorithms for breeding mice and cracking safes; Haiku in which we implemented syllable counting and Natural Language Processing with NLTK. You can see the code I wrote and what I learned in these chapters by going to my Github repo.

ELDonation Tracker

I refactored all the Python files into submodules to better separate what each file does. It is much cleaner now. Plus I finally released 6.0 – there will be a separate blog post covering that.

NASA Background Photo Downloader

Sometimes there’s more than one photo per day on the NASA feed so I changed the code to grab the last 3 images.

Scratch Jr

I also finished the Scratch Jr book with Stella. I enjoyed the book. It appears Goodreads is having some issues now, but I’ll try to remember to add a link to my review when I’m able to.

How Desktop Environment Tweaking Helps Me Be More Productive

A few months ago, someone asked about whether the rices*/modifications/tweaks people displayed on reddit.com/r/unixporn (where people show off their desktops, not human pornography) were actually useful. Someone commented they’d like to see a post on how someone uses their mods. So I decided to write this up.

*I know the term ricing could be considered racist or insensitive. In this context, it’s simply the term of art used on the subreddit.

To start off, I’m using KDE Plasma on Fedora. In the past I’ve been a huge fan of Fluxbox and XFCE. On my netbook I’m also using Qtile to great effect because the monitor is sub-SD resolution. I’ve been using KDE for something like 10 years, ever since around the 4.5 release. What I really like about KDE is the ability to use both Virtual Desktops and Activities. Think of activities as collections of virtual desktops. Or, if virtual desktops extend your desktop in 2 dimensions, activities push it out to the third dimension. What benefit do they give me? There are two main benefits: First of all, it allows me to keep my virtual desktops to a minimum so I don’t have to flick and flick to find where some program is. I have my video editing suite in the video editing activity. VERY easy to find. Second, with KDE apps (and limited success with non-KDE apps), activities can keep track of what you had open in each. So I can have some programs open to certain files whenever I open up the Programming activity rather than having to open it up each time.

I’m using Latte Dock in place of KDE’s default panel. It essentially works the same as the normal panel except I can have different launchers for different activities. I also added a Latte Dock panel at the top to provide an activity switcher .I find that generally faster to use than pulling up the native switcher. It also gives me an overview of what I’ve got open on each one.

So, I’ll take you activity by activity into how my setup helps me get more work done. One thing you’ll notice is that I try to use themed background images so that I immediately know what activity I’m in.

Audio Activity

Audio Activity (20210215)
Audio Activity (20210215)

The main thing I’ve done here is to add 2 Folder View widgets and 1 “now playing” widget. (I lost the now playing widget on an update and forgot to re-add it so you wont’ see it in the above image) The now playing widget is just for fun/looks. Most of the time I’m going to either use my multimedia keys on my keyboard or the now playing widget I put into my top Latte Dock. The folder view widgets facilitate my Audio workflow. I’m one of those folks who still believes in having your own music collection. Sometimes (although rarer than in the TV/movies world) there are rights issues that keep music off of Spotify (or other music streamers). Sometimes the band just hasn’t put it up or there are a few other reasons why the music has to be learned. Also, sometimes you just don’t have access to the Internet. So the lower folder view is where I put audio files that need some work. Maybe I’ve just recorded from a vinyl album or maybe I bought, ripped, or downloaded a new album that doesn’t quite have all the tags I want it to have. So I make sure everything is nice and fixed up with either Picard or (if the music isn’t in MusicBrainz) Kid3. The upper folder view allows me to quickly jump to an artist in fewer clicks than first launching my file viewer and then navigating to the artist.

Books Activity

Books Activity (20210215)
Books Activity (20210215)

Here I have 3 folder views. The top left is an Unimported folder for eBooks and magazines I haven’t imported into Calibre yet. It has folders for various magazines I’ve subscribed to or book series that I’ve supported on Kickstarter. This allows me to use Calibre’s “import from folder” setting that combines all the files in one folder into 1 Calibre entry. So I put the PDF, EPUB, and Mobi entries into the appropriate folder and then import. The bottom left folder is a folderview to my Downloads folder with a filter on it to only show book file types. I use this when I download books from Humble Bundle. I grab them all from Downloads and put them into a folder in the upper folderview. Then I import them as stated before. On the bottom right is my Dropbox ebooks folder. For some places that I buy my eBooks, I have to download them with a Windows app. So I download them and then pop them into this folder. From there I can follow the previous workflow to get the files into Calibre.

Comics Activity

Comics Activity (20210215)
Comics Activity (20210215)

Once again, my flow is around folder views as well as a launcher for two of the programs I use most often in this activity. I use Tellico to keep track of the comics I have, their ratings, URIs to find them on the system, etc .QComicbook is my comic reader of choice if I’m reading a CBZ/CBR. For PDFs, I just use Okular. I think the folders on view here are pretty self-explanatory. The revolve around the fact that, to remind myself of what I have yet to read, I do not move my comics into the Publisher-based folders until I’ve read them. Mostly I read Image and Dynamite published comics, so those are featured with their own folder views. There’s another for all the Humble Bundle comics I’ve bought over the years. I’ve still got a lot of those to read.

Comms Activity

Comms Activity (20210215)
Comms Activity (20210215)

I actually only use this for communications programs, so there isn’t really anything I changed with this activity. I did make use of Latte Dock’s ability to have different launchers per activity – this may result in me eventually getting rid of the launcher I have on the desktop of the Comics activity. This activity usually has Kontact (KDE’s version of MS Outlook) and Choqok (twitter/GNU Social/Mastodon client) on one desktop. The other usually runs Discord, Konversation (IRC), and Element. The fourth screen usually has a web browser.

Main Activity

Main activity (20210214)
Main activity (20210214)

I use this activity when I’m running programs that don’t fit into any of the other categories. Since I usually don’t have as many long-running apps on this desktop, this is the one where I have my system monitors. Yeah, back in the day I played with Conky and lots of other similar system monitors. But these are all about functionality for me. When I’m having issues with the system I take a look at these and try to figure out what’s going on. I’ve been able to pinpoint lots of issues where something had a memory leak and was eating all the RAM as well as when something was going wrong with my network. It’s not perfect since, if the system is that borked it won’t update them, but it’s helped me a lot. The weather widget is nice for a quick view vs a few clicks on my smartphone. The older one was a little better as it showed the forecast a little further out. But the weather isn’t that reliable around here, so I guess it doesn’t matter. The drawing tablet widget is something I’m using for the first time now. If you have a Wacom tablet (or compatible with the Linux Wacom software) you can select various profiles on there as well as its orientation – making it quite useful.

Photography Activity

Photography Activity (20210215)
Photography Activity (20210215)

This one is pretty self-explanatory – a few folder views to get me exactly to where I want to go when I’m loading photos to the computer.

Programming Activity

Programming Activity (20210215)
Programming Activity (20210215)

In the past I’ve had widgets here for selecting KDevleop or Konsole profiles, but I didn’t really find that all that useful. I’ve usually got all four screens full of full-screen programs, so I don’t have too much use for widgets. For example, when I’m coding in Python I usually have (from left to right) Firefox, Pycharm, Konsole, and GitQlient open – each fullscreen on a monitor. 

Video Editing

Video Editing activity (20210214)
Video Editing activity (20210214)

Once again, same theme as usual – Folder views that go to the folders I use often.

Video Games Activity

Video Games Activity (20210215)
Video Games Activity (20210215)

To be honest, I don’t use this activity too often. I’m usually gaming on my Windows computer. But sometimes I need to run some games or test something and don’t want it mucking up my work on the other screens, so I open them up in here.

Videos Activity

Videos Activity (20210215)
Videos Activity (20210215)

A folderview for an easy way to reach the NFS shares that hold the videos I’ve painstakingly ripped from DVDs and Blurays or recorded via MythTV. 

Virtual Machines Activity

Virtual Machines Activity (20210214)
Virtual Machines Activity (20210214)

This one usually has Virt-manager running as well as fullscreen views into the various VMs I’m running. They might be views into local VMs or remote VMs. If you use libvirt/KVM – I’d like to make a plug for remote viewer which is awesome as it lets you run multiple virtual monitors

Web Activity

Web activity (20210214)
Web activity (20210214)

Finally, there’s Web Activity. Since I usually like to run my browser on the fourth monitor and shared across all activities, you might wonder what I do in the web activity? This is for the KDE File Downloader – KGet – which is really nice at downloading if you feed it a text file with a list of URLs to download. Also KTorrent where I literally download my Linux ISOs. I try to be a good Linux user and always download via torrent and seed until the version is outdated. I usually get some pretty good ratios on my Fedora, CentOS, and Raspberry Pi torrents – 5 at a minimum and usually double-digits. Here I make use of a launcher widget to remind me of all the web browsers I have. Since I don’t use the equivalent of the bottom-left icon used in Windows and vanilla KDE to launch programs (I just use alt-F2 and type the name of the program), I often forget some of the lesser-used programs. I installed all these browsers last year as I started exploring what they had to offer. Currently I’m starting to get more and more in love with Vivaldi and its features. (It also launches a lot faster on my resource-constrained laptop) Qutebrowser (which has vim bindings built-in) also works very well on my netbook with Qtile because I’m able to do a lot without using the mouse. 

So, there you go. MOSTLY at this point in time my mods revolve around the folder view for greater productivity. I also use KDE’s activities to spread my programs around so it’s very easy to alt-tab between them and reduce clutter. 

One tiny hitch with the Fedora 33 upgrade

It messed with DNS resolution for my local network, at least temporarily. I couldn’t resolve any websites that needed to hit our local DNS server. I did some Googling and saw that the resolver tech was changed from Fedora 32 to Fedora 33. I change a setting for my NIC and then changed it back and either that fixed it or (some websites mentioned just needing to give it some time). Either way, that was it. Relatively smooth.

Upgrading Supermario to Fedora 33

While new versions of Fedora have been pretty darned stable for a few years now, I usually wait a while after a release to upgrade. This year waiting ended up turning into just leaving it alone. But with February half-way done, we’re actually starting to get kind of close to the Fedora 34 release date. So I figured President’s Day was as good a day as any to do the upgrade.

Years ago, I became interested in a music player called Tomahawk. The latest package I had was a Fedora 31 package. This was causing the upgrade to fail because it needed an older library to stick around. Since I haven’t used Tomahawk since a month or so after writing that article (about 5 years ago), I just did a dnf remove tomahawk. That also got rid of some dependencies that were probably just sticking around for that package.

The more complicated issue was summarized by dnf as: package python3-pytest-relaxed-1.1.5-10.fc33.noarch requires (python3.9dist(pytest) < 5 with python3.9dist(pytest) >= 3), but none of the providers can be installed. For that one I tried it with –allowerasing. That allowed the upgrade to go on.

After that it was 6GB of packages to update (thankfully I’m not on a metered connection) and then a reboot. If nothing goes awry, this’ll be the only post related to upgrading my main computer to Fedora 33. The only holdout now is my oldest’s laptop, but I think I’ll do that one today, too.

Vivaldi Part 4, Brave Part 2, Qutebrowser Part 1

This post continues a series on exploring new browsers:

Vivaldi vs Brave on Windows

Brave didn’t last very long for me on Windows. I just don’t do enough on there to make use of their ad blocking and ad replacing tech. On the other hand, I’ve been using more and more of Vivaldi’s features. I just started using their Web panels – this allows you to have a web page on the left that loads up in a small section – great for pages that you would like to reference here and there without cluttering up your tabs. I’m using it on my Windows computer to keep some Web panels with notes on what I’d like to do in some games – like Civ VI, Darkest Dungeon, and Cities Skylines.

Qutebrowser

Meanwhile, on Kuribo, one of my netbooks, I’ve moved to using Qutebrowser. This is the same netbook where I’m using Qtile as the Window Manager. Qtile is working very well for this since it maximizes all windows and allows me to be keyboard based. Qutebrowser (like QTile is also programmed in Python) is also keyboard-based with lots of vim keybindings. I’ve only just started using it, but with how annoying it it to use the mouse on this netbook, it’s great using a keyboard based web browser.

Five Iron Frenzy – Until This Shakes Apart

Over its long tenure as a band, Five Iron Frenzy has had some really silly songs. Songs like “Arnold, Willis, and Mr. Drummond”, “Oh, Canada”, “Blue Comb ‘78”, “The Untimely Death of Brad”, “Where is Micah?”, and many others. But they’ve also always been a really political band. On their first album, Upbeats and Beatdowns, they had the song “Anthem” which spoke about politicians wrapping themselves in themes of nationalism. “Milestone” also dealt with racism and prejudice. “Beautiful America” tackled both politics and another constant theme across FIF’s albums – rampant capitalism. Over the rest of the discography there was:

  • Banner Year – mistreatment of Native Americans
  • Most Likely to Succeed – doing anything to become rich
  • Get your Riot Gear – about police abuse
  • Giants – rampant capitalism
  • The Day We Killed – mistreatment of Native Americans
  • American Kryptonite – rampant capitalism
  • Zen and the Art of Xenophobia – racism, nativism
  • Someone Else’s Problem – looking the other way at how capitalism means exploitation
Five Iron Frenzy - Golf Courses of America
Five Iron Frenzy – Golf Courses of America

Until recently, I didn’t know this was a historical trend in Ska music. At the time I discovered Five Iron Frenzy, I didn’t listen to secular music. The Supertones and The Insyderz were more concerned with themes of Christianity. When I did start listening to all music, I didn’t see much political in any of the No Doubt songs that made it to the radio. It wasn’t until years later that I heard all of The Mighty Mighty Bosstones’ Let’s Face It album which has a song against racism and two songs against drug use. 

FIF Concert (Nov 2013)
FIF Concert (Nov 2013)

All of that preamble is to say that Until This Shakes Apart is a deeply political album. I’m not sure Reese, Dennis, and Scott could have written anything else in the shadow of the Trump presidency. Overall, the first impression I got from listening to the album was that it was definitely a darker album with lyrics that would sometimes make me a bit sad that we were still dealing with these issues in our country. As I continued to listen, I’ve definitely come away feeling that some of these songs will end up being in my top 10 Five Iron Frenzy songs. I was a Kickstarter backer on this album and so I’ve been listening to it for nearly a week now. Here are my thoughts and impressions on the individual tracks:

  1. In Through the Out Door – A song about xenophobia. Contains a great reference to Gil Scott Heron’s “The Revolution Will Not be Televised”. More towards the rock with horns  sound that Five Iron Frenzy (and many other 90s ska bands – like the Rx Bandits) has mostly moved towards.
  2. Lonesome for Her Heroes – A song about a lack of regard for the environment and rampant capitalism, including gentrification. After the previous song being such a strong rock with horns song – this one is a return to a classic ska sound. I don’t know enough about the history of ska to place its era, but it definitely has a different sound than the Third Wave Ska of the 90s. It seems to mix a bit with an older style.
  3. So We Sing – This was the song to advertise the Kickstarter. It’s a real banger that I’d love to hear live when we start being able to go to concerts again. Lyrically, it’s somewhat of a spiritual successor to “See the Flames Begin to Crawl”. Love the reference to Peter Pan/Captain Kirk at the end of the chorus. Back to rock with horns.
  4. Bullfight for an Empty Ring – another political one that tackles lots of issues, but mostly the hypocrisy of Christians who are aligned with the political right in America. Ska-punk with a bit more emphasis on the punk part. 
  5. Renegades – this time about the politics of gun control as well as once again excoriating Christians who support the party of guns all over the place. Once again a journey to a more traditional ska sound, but with some interesting effects on the track.
  6. Tyrannis – A song against the continued existence of those who build monuments to the losers of the American Civil War. Very strong, fast-paced rock with horns.
  7. Auld Lanxiety – A spiritual successor to songs like “At Least I’m Not like all those Other Old Guys” and “Battle Dancing Unicorns with Glitter”. It’s about getting older and nostalgia. Unlike those songs, the theme is a little less in your face. I really like this song a lot with its music falling halfway between ska and rock with horns. It’s got great backup vocals that work so well.
  8. Homelessly Devoted to You – the first purely fun song on the album. It reminds me a bit of “Vendetta!” from Reese’s side band, Roper. Another good mix of ska and pop music sensibilities. 
  9. One Heart Hypnosis – About being obsessed with social media and getting likes (hearts). Reminds me a bit of “Beautiful America” from Upbeats and Beatdowns. Once again contains a mix of ska and pop/rock music.
  10. While Supplies Last – This one has to have been written during the COVID lockdown as it mentions hoarding of Lysol and diapers. Another song about the hypocrisy of the Religious Right and the fact that they aren’t actually acting as Jesus would. Strongly ska – almost throwback to 1st or 2nd wave. (I don’t know enough to be able to nail it down – but there’s definitely a stronger reggae sound to it.) That said, it does go a bit more into hardcore territory near the end (A la “Fistful of Sand” in Out Newest Album Ever!)
  11. Wildcat – a song about blue collar workers. Definitely back to rock with horns.
  12. Like Something I Missed – I can’t tell just from the lyrics if this is meant to be a couple that’s gone beyond being able to come back together or are just at a very low point in their relationship, but I *love* the chorus with “I need a low dose of you believing in me/ I need a sheet cake made of victory”. It’s sung with such emotion and it’s definitely how I feel sometimes, whether that’s at work or with various relationships. Once again rock with horns. Musically it reminds me of what they were doing on The End is Near, particularly “It was Beautiful”. 
  13. Huerfano – A spiritual sequel to “Suckerpunch” from Our Newest Album Ever!. It’s about being bullied and the chorus is a call to come together with others and be hopeful. Sonically – rock with horns.

Listening to it again as I wrote this, I have to say that my favorite song is “So We Sing” followed by “Homelessly Devoted To You” and “Auld Lanxiety”. I like a lot of the other songs, like “Like Something I Missed”, “Bullfight for an Empty Ring” and “Lonesome for her Heroes”. But this is probably not an album I’d listen to a lot in order because it’s just so depressing to remember where we were at in 2020. The older I get (and I think Roper’s around 10 years older) – the more frustrated I get at how long it takes to move society in the more progressive direction. We’re still fighting forces that would take rights away from women or folks who don’t look like them. We’re still so far from finding a post-scarcity society like Star Trek. Almost everything I hate about social media has to do with putting short term profits over the health of the world. Shoot, even the police situation is still a mess.

So, it’s a good album. No, it’s a great album. If it’s not their best, it’s in the top three. I’m glad Reese turned these thoughts into music. Like the political songs he (and the others in FIF) wrote before, they’re important. And they may use the power of music to move folks to action. But some days I need the music to be an escape. So I’ll listen to my favorites and save the whole album listen for when I’ve got the stamina for it. May the next album find us all in a better place.

FIF
FIF