Coming Full Circle: Contributing to the Python Project Matrix-Nio

Somewhere around 15 years ago, I started learning Python because I’d gotten deep into genealogy and started using the free and open source project Gramps. As I was also pretty deep into free software (somewhere around that time I became a supporting member of the FSF for a few years – and I think my still donates to them) I got interested in the idea of helping projects by contributing code. Python seemed like it wasn’t too hard to learn (compared to C++), so I started learning. I never did end up contributing to Gramps as I found GUI programming incredibly hard. In fact, it was something like 6 years ago before I created some useful GUIs (that weren’t copied out of tutorials) and only last year that I started making good, competent GUIs using the QT toolkit.

Slowly I created more complex Python programs that solved various problems I had. But it was in 2019 that I started making complex projects with pytest, flake8, learning about decorators, etc and started reaching a certain level of mastery in the language. (Which is not to say mastery of the language, but certainly rising above casual tinkerer beginner level)

Also, long-time readers of the blog know that I support self-hosting so that you’re not the product and also so that services can’t be yanked away from you, so recently I started running my own server, using the Synapse reference implementation. (In case you don’t know, Matrix is a federated version of a Slack-like app – basically IRC on steroids). As part of my garage IOT project, I wanted to have it be able to send a message into a Matrix room to let me know if the door has opened or closed. So I needed a Python library to interact with Matrix (rather than speaking at the level of http GET and POST urls). That led me to matrix-nio, the library recommended by the Matrix devs since they’re unable to maintain their own Python SDK.

I wanted to be able to have the user try and register their bot if they didn’t already have an account for it on their matrix server, but matrix-nio lacked both registration and deactivation methods. I put in a feature request for those features. When poljar, the project maintainer, marked it as a good issue for a beginner, I decided to jump in and try and implement it myself. It wasn’t easy – it involved Python’s async library, which I know nothing about, and a way of implementing the repo that was very different from the way I would have done it (not wrong, just different – and probably more like how I should do it given poljar’s experience programming). But I’m now the 3rd biggest contributor to the code base and I still haven’t implemented my deactivation code!

It’s pretty awesome to finally be able to give back to an open source project! I hope many others will be able to benefit from my code.

Milestones reached in two of my Python Projects

For my btrfs snapshot program, snapintime, I have reached version 1.1.1. About six years after starting this project, it now covers all the cases I wanted to back when I started. I’m very happy with where it’s now at.

For my Extra Life Donation Tracker (PyPi page here), I’ve reached my v4.2.0 release. This release fixes some of my oldest issues (#s 13 and 14 – while the current highest issue is #95), and allows users maximum flexibility in the way they want to set up their tracker window. They can change the font face and color of the text that displays the latest donation and they can change the background color if chromakey green doesn’t work for their use case.

A Tip for Reading Manga on the Kobo Clara HD

Recently I got a bunch of manga through a Humble Bundle sale. Having read “authentic” style manga before, the Kobo Clara HD seemed to be about the right size to read manga without having to do any zooming. So, naturally I uploaded the .epub for the Kobo. It was HORRIBLE. It cut off part of the image and made it impossible to read. After a bit of Googling around, I found the suggestion to use the .CBZ file, as the Kobo was capable of reading it. That worked very well. The only real bummer is that it has a margin around the page that makes it just SLIGHTLY reduced in size. It’s not a problem for the text, but some of the fine details can be a little hard to make out. Overall, it worked VERY well and I recommend it as a way to read manga in ebook form (allowing you to carry all of a modest run on your device). (But might not fit all of, say, One Piece or Dragon Ball.)

Review: Surrija by Surrija (formerly Jane Lui)

Like, perhaps, many of my readers, I knew Jane Lui’s music mostly from her cover songs. But I really liked her voice and so I decided to back the Kickstarter for her new album, Surrija. By the time this review is published, I’ll have had the album for about 2 months. For this review I’ll first focus on my initial impressions and then my impressions after having listened for a while.

My house is not full of Sonos speakers, Amazon Echos, or Google Home devices. So when I want to listen to music, it’s either on the living room sound system, or (more often now that I have kids), playing from my phone wherever I happen to be. Because of aforementioned kids, I rarely listen to music with headphones on while at home. That whole preamble is to say that I’m often listening to music, whether it’s Taylor Swift, Louis Jordan, Audioslave, or Billie Holiday, through my Pixel’s speakers. I would strongly advise you not to do that on your first listen of Surrija. While I normally don’t have issues with music through my speakers, Surrija (as we’ll refer to Jane Lui since that’s her new stage name with the album in italics) is trying to push boundaries with this album and until you get to know it, it just sounds extremely discordant coming out of speakers. This is particularly true with the first track, Nothing Love. The percussion that opens up the album is halting, seems to fail to maintain a tempo, is full of glitches – like glitch pop or a badly encoded MP3 and seems to challenge the user. Barcelona also starts off with some strange beats and Surrija singing with a very eerie voice and purposely out of key, before she moves into a beautiful rendition of a bridge that shows her range and that she’s purposely pushing her voice to put out a certain atmosphere. So, on that first listen on my phone, I wasn’t happy with my Kickstarter backing.

Then, the next day, I threw the album on at work (we have an open office, so headphones are the only way to stay sane) with some headphones. Suddenly I was able to hear the full range of what Surrija was trying to do with this album and I went 180 on my opinion. This was a great album that was worth waiting for! Surrija has put a LOT into this album and I would love to hear her do an episode on Song Exploder about any of them and how she decided what to layer into each track. So, without further ado, my thoughts on each track.

  1. Nothing Love – Interestingly, the song that at first listen (again, from my phone’s speakers) turned me off from the album ended up being my favorite once I gave it a proper listen with headphones on. Musically, my favorite part is the chopped up horns that accompany the chorus. Surrija’s haunting voice on the bridge is my favorite vocal part of the song.
  2. Barcelona – This song also has a beautiful chorus that shows off a lot of the strength of Surrija’s voice.
  3. Sylvette – This is the first song with a bit more of a pop sensibility. If I were introducing someone to Surrija, I’d start with this song. It has a much more common baseline where the percussion is not fighting the song. It’s also a great jam that I find stuck in my head for days after listening to the album.
  4. Minotaur – I’m not sure what the title has to do with the song, but this is a very interesting song. There’s a certain tension that builds musically and vocally that gets released with the chorus, right as you hear “She changes everything.”. I’m not entirely sure if it’s meant to be a hopeful song, but it certainly elicits that feeling in me.
  5. She learned not to be scared – This is a very haunting piano piece that sounds very familiar. Is a reference to a famous piece? Is it actually a famous piece? Either way, it’s a great, short track.
  6. Gilot – we return to some out of tune instruments in the melody, combined with sparse beats, and Surrija’s beautiful voice. I think this is my second favorite song on the album.
  7. Turnstile Hostile – while not as straightforward as Sylvette, it’s definitely a song that could be a single that helps take this album up the charts. It progresses to a crescendo in the end that really pays off.
  8. Dora – This one is in conversation with Minotaur. It’s also beautiful and appears to contain a duet, although it may just be Surrija singing with herself thanks to recording and editing technology.
  9. Mercy Street – A third song that could be a single, but this one a very slow, emotional one. Funny thing is that I’m pretty sure I’ve heard similar lyrics in Christian Rock. In fact this song reminds me in style of Plumb. Of course, don’t take that to say I’m accusing Surrija of copying anything, more just an observation of how certain themes appear in certain contexts.
  10. Semibelieve – Another slow song.
  11. h u m – doesn’t really function on its own. It’s more of an interlude into the next song.
  12. Almost Time – A good closing song. Powerful, if a bit melancholy sounding.

This is definitely an album worth buying. If you’re no longer buying music, definitely worth streaming. You get more out of it with every listen. Check it out!

Hoodie Sisters

There was something about this image that reminded me of older snapshots. I think it’s the fact that it’s a candid moment, not perfectly sharp as Scarlett moved out of the focal plane; but it’s not out of focus – you can see that from the shirts. So when I was processing the shot, I went for an older look. I think I ended up applying a Kodachrome effect.

2020 Cherry Blossom 10 mile Virtual Race

a post-race selfie

Ever since fall 2019, I’ve been training for the Cherry Blossom 10 mile race, scheduled for 5 April 2020. Unfortunately, like everything else involving more than one person, it’s been scuttled by COVID-19. After all that training, it was disappointing. (I will take a moment to acknowledge that it’s objectively ridiculous to be disappointed by a missed race when people are dying. However, there’s always someone worse off than any other person. eg. I can feel that I’ve had a bad day at work, even if my day wasn’t as bad as someone who found out they had cancer It doesn’t delegitimize their feelings of disappointment as long as they keep it in perspective.) The organizers of the Cherry Blossom Festival race had a solution – a virtual race! Run wherever you can and record your time and it’ll all go on the website. It’s not perfect, but it’s certainly a way to use all that training you’ve done. So today I went ahead and did the race since the weather’s actually good today. It was an AWESOME run. It felt great to do it and, given the insane hills in my neighborhood, I finished feeling that I could definitely kick butt in the Cherry Blossom race next year because their course is not very hilly. It also renewed my confidence in being able to do well in the Baltimore 10-miler this summer (if it’s not cancelled), renowned for being hilly. So how did I do? (if you paid attention, it was spoiled at the top…hehe)

Well, first of all, check out these achievements I earned during the race:

Fastest mile! 6:54
Fastest 5k! 22 min 30 seconds!
Fastest 10k! 46 min 40 seconds – need to do this at a 10k race!

Finally, here’s the time!

10 miles in 1 hour 14 minutes 48 seconds
hard to run slowly to this music

I looked at the website and, based on the training they’d been having me do, they predicted a 7:52/mile race pace. So I set that in my Garmin and headed out. But I was listening to my techno album Y2K: Beat the Clock and to Lionize. And so I just couldn’t go that slowly. I keep going faster than the pace I was supposed to go. I was worried that would wear me out. But I just kept up that pace, slowing on the long uphill stretches, but overall keeping a good pace. In face, even though miles 5-7 were close to that pace, even they were faster. And I did a sprint in the 0.25 miles, responsible for my best mile ever recorded by the Garmin. And so, as you can see above, I averaged a 7:29/mile pace. Very, very exciting. I cannot wait to run this race next year with others around me (and after having had a full year of training)

Throughout this training, for the past few weeks, has had me running 12 miles every weekend as my long run. I’d been having doubts about being able to do a half marathon before this training, but now it’s right there within reach and I’m definitely also trying to figure out which half marathons might be available for sign up in the future. I’ll leave you with some amusing stats from the Garmin.

I guess I should take it easy tomorrow

My Photostats Python Program

11 years ago (HOLY MOLEY WHERE DOES THE TIME GO?!?) I wrote about Jeffrey Friedl’s LR Stats plugin. As you know if you’ve been around my blog for a while, I love stats! It’s the biggest reason I do my end of year posts with my music, reading, and video game stats. Well, I haven’t used Lightroom for a few years now, and for some reason I was inspired this past weekend to recreate this ability. So I started on photostats, which will present a lot of the same stats that his Lightroom plugin does. On the plus side, my Python program is app-agnostic!

Here are some graphs representing 2020:

How many photos I’ve taken with each lens in 2020
How many photos I’ve taken in 2020 at a given focal length

I still have lots of mods to make, like labeling the axis and so forth. But I’m pretty excited already! I’ll end this post with my 2019 lens model stats:

PyCharm 1 week Update

I’ve been using PyCharm for about a week now across a couple projects. I’d like to note a few things I’m really enjoying:

  • PyCharm is aware of your venv config and you can use it to pip install new packages. Ok, so pip is easy to use for installing. But it also provides a quick at-a-glance view of the packages you’ve installed and their versions – and I find it easier to view than a pip freeze.
  • I know KDevelop has this for C++, but it doesn’t really have the same support for Python – being able to run my Python files from within PyCharm as well as being able to run the debugger from within PyCharm is like how new cars unlock for you when you get close. It’s not necessary. It’s not even that huge of an improvement. But it’s a quality of life thing that you miss when it’s not there.
  • PyCharm is very smart about git. With KDevelop, I could add files withink the GUI (which is nice for making sure the project is aware of the file and provides a quick visual cue). But then I had to remember to git add it (either in the GUI or on the commandline). With PyCharm, every time I add a file through the GUI, it asks me if I want to add it to version control.
PyCharm asking if I want to add the new file to source control
  • I love being able to do a commit/push at once through the PyCharm GUI.
  • It also noticed on a new project that I hadn’t setup a .gitignore yet. AND when I said I wanted to set one it, it did just like GitKraken and automatically created an awesome file for me that already had all the Python stuff you’d want to ignore.
PyCharm noticing I don’t have a .gitignore set up
  • Finally, I also figured out the one thing I was strongly missing from KDevelop – the ability to highlight your variables so that it’s quick and easy to find them throughout your source code without having to first highlight them. I asked on reddit and was told to enable semantic highlighting.
PyCharm with semantic highlighting enabled.

So, I think I’m pretty strongly hooked. I guess there’s still something that could come about that would make me turn away from PyCharm, but right now I’m thinking it’s basically the BEST way to code in Python on Windows and Linux.

Reviving and Revamping my btrfs backup program Snap-In-Time

If you’ve been following my blog for a long time, you know that back in 2014 I was working on a Python program to create hourly btrfs snapshots and cull them according to a certain algorithm. (See all the related posts here: 1, 2, 3, 4, 5, 6, 7, 8, 9) The furthest I ever got was weekly culling. Frankly, life and school contributed a good excuse not to keep going because I had created a huge headache for myself by attempting to figure out the date and cover all the possible corner cases with unit tests. This is what my code looked like in 2014.

As you know, I’ve recently been on a Python programming binge due to reading a few books that taught me advanced topics and how to properly structure Python code folders. One of the books also had a reminder to check the Python modules (built-in and others) before implementing something that has already been done and well-tested. I’ve also gotten way better at reading and understanding the official Python documentation. So I discovered the datetime module that alleviates almost all the pain I was having when working on Snap-In-Time and spent a week or two on a code sprint. I’m now almost back to the same functionality I had before, but with MUCH cleaner and more maintainable code! Also, it’s become my third package on PyPi.

Snap-in-Time on PyPi

Now I’m jazzed to keep working on it and making it better and better.