Below you will find pages that utilize the taxonomy term “Python”
2024 Programming EOY
This year was a bit light on programming. I was focused a lot on gaming, cooking, and Tabletop Gaming. (No surprise if you’ve been reading my December blog posts) I constantly found myself desiring to code and I definitely had a few ideas - like a utility to extract EXIF data out of my images for use in Hugo. But when I would sit down at my computer I often found myself preferring to play Cities: Skylines II or Against the Storm. (More on that when I post my EOY video game post) As per usual, let’s go month-by-month (after the first quarter that is)
Advent 2024 Day 03
Time once again to review my Advent of Code solutions! Also the latest die or dice from the Dice Envy Advent Calendar; scroll below the AoC code to see today’s die.
Advent of Code
As the years have piled on, Eric Wasl has thrown in references to past years’ problems. However, as of day 3 we now have 2 references (perhaps that’s what the historian story line is about). Day 02 had you visit the location of 2015 Day 19 and today visited the location of 2020 Day 2 (my first year doing Advent of Code!) The only thing I’m slightly worried about is if this signals that this year will be a grand finale for the annual event.
Advent 2024 Day 02
Time once again to review my Advent of Code solutions! Also the latest die or dice from the Dice Envy Advent Calendar; scroll below the AoC code to see today’s die.
Talk Python Bits
Michael Kennedy from the podcasts Python Bytes and Talk Python has recently written a couple blog posts that I wanted to highlight here. First up is his post about rewriting the Talk Python site in Quart (aka async Flask). Three and a half years ago I wrote about realizing that it’s not about Django vs Flask vs FastAPI, but actually about finding the right framework for your workload. I didn’t completely state my current thought process in that blog post, but I was pretty close to the advice I currently give. If you want a Python-based site that needs CMS features and you don’t want to have to reinvent the world, Django is your framework. If you are making websites where you want to be in control of every puzzle piece, then you want flask. You only bring in what you need. And if you’re writing REST API based site (controversy about whether REST truly is REST aside), you want FastAPI. Well, I respect Michael Kennedy’s opinion on this matter for a few reasons:
Advent 2024 Day 01
December has finally arrived and with it my favorite activity of the month - Advent of Code! I also wanted to give myself a fun little present this year, so I bought the Dice Envy Advent Calendar. Every day I’ll get a fun new die (or dice!). Scroll below the AoC code to see today’s die.
Advent of Code
Let’s start off with Advent of Code. This year I haven’t done any of the Advent of Code problems ahead of December so I was a bit rusty. One is not often using the same algorithms or libraries for Advent of Code as one does for everyday programming. I started off with Python since that’s my most comfortable language. First you can head over to today’s page to see the prompt. If you’re not solving the problems yourself, you won’t be able to see the prompt for part 2, so in summary:
Wordpress to Hugo Migration Process
As there are many people who are currently looking for alternatives to Wordpress in light of a little…. instability… I decided I would document my migration process. I figure my case is one of the more extreme cases, as I ran a self-hosted Wordpress instance for the past 19 years (since Feb 2005) and have ~4000 posts that needed to be migrated. I also have lots of photos, videos, and other media. Finally, I have made heavy use of many Wordpress features.
Programming Jan-April 2024
This year started off pretty light when it came to programming because I’ve been addicted to the video game Against the Storm since last winter. But I eventually started working again on various projects - some old and some new. I didn’t do any programming in January, so we’ll start in February.
February and March
Over these two months I worked on my replacement for web access to my Taskwarrior TODO list because Inthe.am had shut down. In February I got the podman containers set up - one to run the taskd server and one to run the website I’d coded up in Flask. In March I had to write some rudimentary Javascript to get the website to highlight the selected tab (Overdue, Today, This Month, etc). The rest of the interactivity on the site works using HTMX, letting me focus on Python instead of Javascript, but I just wasn’t able to get that part of the site to work without a tiny bit of Javascript. I also added some fixes because the date/time widget assumes UTC. Of course, now that I have it all working correctly and get lots of use for it (especially when I’m at work and I want to quickly get something out of my brain’s short-term buffer), Taskwarrior went to 3.0 which completely changes the way the program works, the API, and the way syncing works. I think in the end it’ll be for the best, but it’s annoying that I need to figure this out. That may involve finally learning how to use PyO3 to interact with Rust or re-writing part of my backend in Rust. We’ll have to see where that goes.
A few thoughts on Programming languages
Just a few thoughts on programming languages that have been rattling around in my head this week, but which don’t each merit a full blog post. The main theme is that the culture behind each programming language leads to some interesting choices, as is the case with spoken languages.
This week I started learning how to program in Rust. Even though I’m using the project-based Command-Line Rust to learn, the author still went with the traditional “Hello, world!” project for the first intro to the language. I was also working on a Go project last week and so it immediately stood out to me that (at least as taught by this author) Rust has the print! macro that allows you to succinctly print to the command line. By contrast, Go requires importing fmt before you can print. This was the first topic that was swirling around in my head this week. What makes language designers choose whether printing output (one of the most basic things a program can do) is built-in or requires an import. I even remember back when I was learning Java in undergrad (I think it was Java 1.8, but I don’t remember) we had to use the savitch library just to get program input (another very basic computer program concept). As I thought about it, I wondered if it has to do with thoughts around compilation and whether the language designers think you’re mostly making user-interactive programs or libraries? It makes sense to me that scripting languages like Python, Ruby, and Perl would have print built-in since you always have to have the interpreter along with you, so all the basics should be there. (The original Batteries Included Python promise, for example) But perhaps the Go developers thought you wouldn’t always be printing to the command line so a more efficient binary could be compiled by forcing you to import the functionality? I’m not entirely sure.
My 2023 Programming Progress
In 2023 I just didn’t have the urge to do as much programming as in years past. I felt more of a tug towards video games, reading, and baking/cooking. So this recap will be quite a bit shorter than usual. A couple upfront themes and ideas: The programming was essentially Go and Python. I started reading a book about Julia, but didn’t do any new programming in Julia. I mostly worked on new code vs maintaining older code.
Programming Update: March 2023
In March I continued to work on my programming projects as well as a little more Advent of Code.
Things started off with me finishing up my dreamhostapi and Dreamhost_DNS_Go projects in Golang. I got the API working under all the conditions that I need and I worked on both the output of the program and the logging. It now functions quite well for me to keep my dynamic DNS URLs up to date. My next step is something I’m going to do in a few of my cron scripts - have a separate JSON log that could be interpreted so that I could either have a daily email to let me know that the scripts all ran correctly or so that I can load them into Prometheus/Grafana to see on a dashboard how all my scripts are doing.
Programming Update: Jan 2023 and Feb 2023
January
January was a relatively light programming month for me. I was focused on finishing up end of year blog posts and other tasks. Since Lastfmeoystats is used to generate the stats I need for my end of year music post, I worked on it a little to make some fixes. The biggest fix was to change the chart titles not to be hardcoded. I didn’t realize it until I was reviewing my blog post, but I had hard-coded the year when I first wrote the code a couple years ago. I also changed the limits on some of the data I was collecting so that I could do more expansive trending for my overall stats.
2022 In Programming
I started off the year mostly working on Python projects. For January and February I finally started making some great progress with understanding modern web frameworks and use of CSS frameworks rather than rolling my own. This not only helped with the Prophecy Practicum project, but would form the backbone to a lot of coding this year. I also wrote a utility to tally up my videos for my end of year video game roundup, saving me literal hours of time. As you can read from following the link above, I also worked on my long-running Extra Life Donation Tracker code.
Programming Update: December 2022
December was wholly dedicated to solving the Advent of Code 2022 problem set. Our job was to help the elves trek through the jungle to get magical starfruit. It was a lot of fun to do it live once again! This year I also had experience from the previous years (both live and on my own pace) that gave me the skills to solve some of the puzzles, including some puzzles that were variations on themes I’d seen before. In the end, I collected 36 stars, near my average for number of stars collected during AoC. Between work, family commitments, my programming and problem solving skill levels, that’s about the most I can usually do during the live period in December.
Programming Update: Nov 2022
November was not a huge programming month for me. On the weekends I was a little more focused with family stuff and videogames. But I did manage work on a couple projects.
Ever since 2010, I’ve been been using Python to automatically post my top 3 artists to Twitter. In 2020, it even became my first package on Pypi. With activity picking up on Mastodon, I ported the code over to that site. I also used the opportunity to clean up the code bit to my more modern coding standards. I haven’t created a package yet, but may soon enough.
I also did some work on my Extra Life Donation Tracker. I’d pulled out the API so that others could build on it, but, as a result, I was creating a Team object from the API rather than the inheritance version. I did a workaround to make it work, but I’m going to have to work at the code a bit more to make it less clunky.
Review: Data Visualization with Python and JavaScript: Scrape, Clean, Explore & Transform Your Data
Data Visualization with Python and JavaScript: Scrape, Clean, Explore & Transform Your Data by Kyran Dale
My rating: 4 of 5 stars
While a book about web technologies is undoubtablely going to get out of date (especially when Javascript is involved), I would definitely recommend this book if you want to do some data visualization either as part of your job or for an undergrad, grad, or PhD project. While I would probably use FastAPI rather than Flask, I heard recently that the Javascript library the author uses, D3, is still one of the best in class libraries for this kind of work.
Programming Update: October 2022
As October came around, it was time to get ready for Hacktoberfest. I’ve been participating for the past few years and I love the fact that DigitalOcean supports this project which gets more people to contribute to free and open source software.
In the past, I’ve often contributed to my Extra Life Donation Tracker. Since the program is pretty mature at this point, most of the issues I have would have taken me too long, especially with how busy I’ve been with non-programming projects lately. So I dedicated my efforts to helping with some quality of life issues for the Prophecy Practicum (Django Version) that I’ve coded up for my friend.
Programming Update: Aug
August was a programming-filled month for me. It focused entirely on Python and I mostly continued working on established projects. Let’s jump in!
Amortization
I wanted to re-calculate the amortization table for my home loan for the first time in about a year. As a refresher, I created this program (vs using Excel or an online form) because we are not consistent in the amount of extra principal payments we make. For example, if I get a bonus at work, I might throw all of that bonus into the loan payment. So this program takes variable extra payments into account when creating the amortization table.
Programming Update: March/April
In March I went back to Advent of Code 2016. In an effort not to get stuck, I decided to go as far as I can in Python before going back around and working on the other languages. So I did days 6 and 7 in Python. Overall, relatively easy Regex problems.
And that’s it. It was not a big month for programming. I mostly focused on playing with the kids and cooking. May/June are looking like they’re going to contain quite a bit more programming, at least relatively speaking. I’m hoping to both get back to work on some of my projects and also get back to Advent of Code.
Programming Update: January/February 2022
I started off the year not expecting to do much programming. Compared to some months in 2021, I barely programmed, but I did end up programming much more than I expected. Let’s take a look at what I worked on in the first sixth of the year.
Python Programs
End of Year Video Games Helper
Coming off of last year in which I finally used Python to pull my last.fm data and create graphs, I decided to do the same for my End of Year Video Games blog post. I haven’t posted the code to Github, but here it is:
My Programming Projects and Progress in 2021
As I did last year, I’d like to take a look at how well my predictions matched up to what ended up happening:
- Working on my Extra Life Donation Tracker: Yes! I made a bunch of releases last year to fix various bugs for my users. I also finally broke out the Donor Drive Code into its own project so that my code could be used as the basis of non-Extra Life Projects
- Moving Prophecy Practicum to Django: Yes! I did this and my colleague has been using it for about 6 months now. I have some quality of life issues to fix that will help me get better at Django and maybe CSS.
- Redoing flickr views project: Nope. Completely forgot about this.
- Progress on my Unity Game - Eric’s Comet Cleaners - None.
- Learning new programming languages: Yes! Haskell and Go, through Advent of Code problem solving. Also got better at Ruby and Perl.
- Electronics: Some Adaboxes, but no work on my BBQ ThermostatKids: A little more Scratch with the twins. No “real” programming languages with Scarlett.
Compared to last year I had 10 more commits to Github. Pretty consistent!
Programming Update: November/December 2021
In these last two months of the year I only worked on Advent of Code.
November
In November I worked through part of the 2016 problem set. I didn’t get too far because of how many languages I was doing at this point. Eventually I decided to allow myself to get a bit further in Python and then catch up with the other languages. Whenever I’d get stuck I’d go back to the other languages. Overall, once I’d figured out Python - Ruby, Perl, and Golang would be pretty easy. Haskell would still be hard, but I started getting the hang of it near the end of the month.
Programming Update: September / October 2021
Across both September and October I got a decent amount of programming done. I just about finished the Learn You Haskell for Great Good book (got up to just before Monads) and started Head First Go. Once I had enough Go language to get started on AoC, I knew I’d be ready for the 2016 problem set. Let’s get into the nitty gritty!
Python (not Advent of Code)
Impractical Python
I finished chapter 11 which contained my old friend the Monty Hall Paradox. I’ve had a version up on my site for a very long time based on a PHP tutorial in Linux Format Magazine. This time I was able to code it in Python as well as creating a GUI version. Chapter 11 also touched upon The Birthday Paradox.
Completing Advent of Code 2015 with 3 Programming languages
Throughout the spring and summer of 2021 a few of the times that I mentioned on the Advent of Code subreddit that I was doing the 2015 problem set in all 3 languages, some folks said they’d be interested in a writeup on the experience. Now that I’ve finally finished 2015 (my first set of 50 stars!) it’s time for that writeup. Before I continue, I’d like to thank everyone on the subreddit who has helped me. I have a README.md for each day’s problem and you’ll find my thanks to those who helped me within those READMEs here in my repo.
Programming Update for June 2021
June was mostly Python, although I did do chapter 1 of Scratch 3 Games for Kids with Sam. He really, really enjoyed it and I anticipate doing the challenge problems and maybe chapter 2 in July or August.
Books
I read the intro and first couple chapters of both Flask Web Development, 2nd Edition and Data Visualization with Python and Javascript, both from a recent Humble Bundle. The Flask book may be useful for learning more about creating a non-Django site and, even if I mostly stick with FastAPI, it should provide some concepts that are applicable across both frameworks. With the data visualization book, I would love to use that to better visualize my annual Last.fm stats.
Programming Update for April 2021
I had a lot less variety in my programming month, but still had a lot of fun doing it. In fact, Programming consumed most of my leisure thoughts. More about why I was doing it below, but I’ve been reading Programming Perl as well as skimming through Introducing Go and Learn You A Haskell for Great Good!. Ever since some folks used Haskell during last year’s Advent of Code and this guy’s videos that I mentioned in an early 2021 blog post, I’ve been very curious about the language. In fact, at this point I’ve decided that Go and Haskell will be the next two languages I learn.
Programming Projects: March 2021
I started off the month thinking it was going to be Python heavy and ended up doing a lot more micro-controller programming. To be fair, I was mostly programming in CircuitPython, but it definitely came out of nowhere.
Python
Civilization VI Webhook in FastAPI
Last month I created a webhook program in Flask to generate notifications when it was someone’s turn in our multiplayer Civilization games. When I posted about it on reddit, someone suggested that I would be better off using FastAPI instead of Flask. I’d been hearing about FastAPI for months on Talk Python to Me and Python Bytes, but I always got the impression that it would only be useful if I was writing my website to be async.
My Extra Life Donation Tracker reaches v6.0 (feature complete)
This is my seventh year raising money for Johns Hopkins Children’s Center via the Extra Life gaming charity. When I started back then I was brand new to streaming or recording video game play. In fact, just a few years before that I hadn’t even understood the point. I found out that you could somehow display your progress towards fundraising on your screen while you played. I had no idea how to do that and, at the time, I’d only done commandline programs. So I found bfinleyui’s web-based program. I set up XSplit (which I was using at the time) to capture the web pages and was able to have a great first year. The following year he created an app with Adobe Air that was even better. The GUI for this app would become the inspiration for ElDonationTracker; what I aspired to eventually build. My first commit to the repo for ElDonationTracker was that year. I was exploring how to get the API data via the commandline, but I wasn’t too hurried because I had the Adobe Air app. Unfortunately, the following year Adobe killed Adobe Air. So it was now time for me to try and make my own app. Unfortunately, I’d never done GUI programming before and my experiment with the TKinter GUI framework didn’t quite work.
Programming Jan/Feb 2021
I was pretty busy programming at the start of 2021 across a few different languages. Let’s jump right in!
C#
I’m nearing the end of the GameDev.tv online RTS course, and it’s been a lot of fun. Since last time we added player colors to the units, a minimap that can be used to move around the screen, new units, and a Lobby UI. I’m a few lessons away from being able to create binaries I can use to play online with others or via Steam.
My Programming Projects and Progress in 2020
Back in 2019, when I did my programming retrospective I made a few predictions. How did those go?
- Work on my Extra Life Donation Tracker? Yup! See below!
- Write more C++ thanks to Arduino? Not so much.
- C# thanks to Unity? Yes, but not in the way I thought. I only did minor work on my game, but I did start a new GameDev.tv class.
- Learning Ruby? Well, I wouldn’t necessarily say I learned Ruby. I did finish the book Ruby Wizardry and I took copious notes. But until I do some practice - maybe via some code katas, I don’t think I’ll have solidified it in my mind.
- 3D Game Dev? Nope, not really.
- Rust and Go? Not even close. Although I did make sure to get some books on the languages.
So, what happened? On the programming front, I wanted to continue my journey to truly master Python after having used it at a surface level for the past 15ish years. I dedicated myself to doing the Python Morsels challenges (more on that below) and working through various Python development exercises. Outside of programming, the time I had off from COVID was used to play with my kids and they wanted to play lots of video games. So we took advantage of having way more time than usual to do that. So a lot of my goals slipped. We’ll get to 2021 predictions at the end, so let’s take a look at 2020!
PyGame 2.0 is out!
I just found out today that PyGame 2.0 was released yesterday, on the 20th anniversary of the software. One of the first steps I took with Python was a series of games I made from tutorials in Linux Format Magazine. On Github I’ve got my shifter game and my Space Invaders clone. The shifter game is a little wonky since I haven’t touched the code in over a decade. But I was able to make the one shift needed to make the Space Invaders clone work on Python 3. So you can enjoy those and celebrate that PyGame development has picked up steam again.
Last few weeks in Programming: Python, Ruby
You may notice it’s been a while since I last wrote about programming. Well, a lot of the programming I was doing with the kids was due to having more time on my hands thanks to COVID. With Scarlett back in school full-time, there’s a lot less time for that as we focus on making sure she gets the most out of her education as well as teaching the twins to make up for not being in preschool this year. This has left me with reduced time for all my projects and entertainment (games, TV, and reading). Up until now that has meant the programming was put off to the side - kind of. Because I’ve been loving my exercises from the Python Morsels website and I’ll be talking about that when I get to the Python section. But first up, Ruby!
Last Few Days in Programming: Python and Arduino
Python
Spent the last few days finalizing the 5.0 release of my Extra Life Donation Tracker and then pushed ahead to get version 5.1 out. Here’s the PyPi page if you want to use it for your Extra Life live streams.
Arduino
I got back to my BBQ Themostat project and did some minor programming while trying to figure out how to run a computer case fan. So far I’m still working on the wiring aspect of this part of the project, but some folks on reddit did point out that part of my problem was getting a pair of my BJT connections mixed up.
Coming Full Circle: Contributing to the Python Project Matrix-Nio
Somewhere around 15 years ago, I started learning Python because I’d gotten deep into genealogy and started using the free and open source project Gramps. As I was also pretty deep into free software (somewhere around that time I became a supporting member of the FSF for a few years - and I think my smile.amazon.com still donates to them) I got interested in the idea of helping projects by contributing code. Python seemed like it wasn’t too hard to learn (compared to C++), so I started learning. I never did end up contributing to Gramps as I found GUI programming incredibly hard. In fact, it was something like 6 years ago before I created some useful GUIs (that weren’t copied out of tutorials) and only last year that I started making good, competent GUIs using the QT toolkit.
Milestones reached in two of my Python Projects
For my btrfs snapshot program, snapintime, I have reached version 1.1.1. About six years after starting this project, it now covers all the cases I wanted to back when I started. I’m very happy with where it’s now at.
For my Extra Life Donation Tracker ( PyPi page here), I’ve reached my v4.2.0 release. This release fixes some of my oldest issues (#s 13 and 14 - while the current highest issue is #95), and allows users maximum flexibility in the way they want to set up their tracker window. They can change the font face and color of the text that displays the latest donation and they can change the background color if chromakey green doesn’t work for their use case.
My Photostats Python Program
11 years ago (HOLY MOLEY WHERE DOES THE TIME GO?!?) I wrote about Jeffrey Friedl’s LR Stats plugin. As you know if you’ve been around my blog for a while, I love stats! It’s the biggest reason I do my end of year posts with my music, reading, and video game stats. Well, I haven’t used Lightroom for a few years now, and for some reason I was inspired this past weekend to recreate this ability. So I started on photostats, which will present a lot of the same stats that his Lightroom plugin does. On the plus side, my Python program is app-agnostic!
Hey, Pycharm, hey.
Back in September I wrote about moving to KDevelop for my larger Python projects and also staying with Kate for my smaller projects. I’ve REALLY been enjoying all the features as I work on more and more complex packages involving lots of files. But for a few episodes of Python Bytes now (and/or maybe Talk Python to Me - they share a host) they’ve been talking about how awesome it is to use an IDE that has a Python debugger built in. It may be that KDevelop has that ability (I’m pretty sure it does for C++), but I’m not sure. I am nearly certain that Pycharm does have a debugger built-in. A while ago I’d set up the Pycharm COPR repo on my laptop. I tried playing with it a little at the time, but I was coding on a very small project without a venv, so things didn’t work out well. (Pycharm is pretty opinionated about the way a Python project should work)
My first PyPi Package!
I was reading Serious Python by Julien Danjou as well as looking through other Python repos when I realized that my Extra Life Donation Tracker repo was a mess. (It’ll be different by the time you read this, but here’s how it looked at the time) After learning about how things should be structured and reading Mr. Danjou’s book about how I could use Sphinx to auto-document my code, I decided to undertake the challenge. However, there was a lot to learn and this code is used by many others for their Extra Life campaigns. I wanted to start off with much simpler code, so I started off by packetizing my lastfm-twitter code. It was a fun journey over the last week or so as I learned how to restructure my code and auto-document. (I also learned a lot about how to use Drone CI, which I’m using for my home projects that aren’t on GitHub) As the final lesson, I taught myself how to create a package for PyPi via Drone CI and so, I now have a PyPi package that anyone can import with pip!
2019 In Programming
This was a VERY busy year for my coding. This year I had over 769 commits to GitHub; my previous best was 58 in 2014. The commits were very unevenly distributed. If you look at the graph of my commits, outside of a busy week in January and another in May, I didn’t really start coding until August.
GitHub activity graph
But perhaps commits to GithHub isn’t the best metric. If you look at what I accomplished in the second half of 2019, I worked on an incredible number of projects.
New Release (v1.9.3) of my Extra Life Donation Tracker Software
I’m getting really close to finally getting this program to where I wanted it to be 4 years ago. My most recent release adds in the Team info to the GUI (and the previous release, which I didn’t blog about added team info in general). I’m getting feature requests and bug reports and all those kinds of encouraging things that make people want to provide programs for others. Also new as of a couple releases ago is the fact that I’m using Github’s CI (called Github Actions) to produce binaries for me that I can attach to releases to make it easier for folks to use the software.
Moving to KDevelop for my larger Python Projects
When I first started programming it was just at the prompt of my Tandy computer. Then it was in the QBasic text editor on the IBM computers at school. But when I started programming again with “real” languages, I ended up going with Emacs. Although I was first annoyed at the way commands like save were prefixed, it became my favorite editor. I think that was partially because of diving headfirst into the free software movement and partially because I didn’t like vi’s different modes and how annoying that made things if you didn’t realize you weren’t in the text entry mode. Eventually, I moved on to Kate because I love KDE (been running it as my main desktop for over 10 years now) and I loved the features it enabled. Also, since vi (or vim) is found EVERYWHERE while Emacs usually requires installation, I ended up switching to vi whenever I’m ssh’d into a computer. If you add plugins like powerline, it can be pretty awesome to use. Even on my desktop if I’m editing /etc/fstab I’m more likely to pull up vi than the weight of KDE (not that it takes up THAT much RAM).
New Release of my Extra Life Donations Tracker Software
After a bit more work since my 21 July post, I’ve finally reached what I can consider to be beta status for the GUI. Things have matured to the point where it works well on both Linux and Windows and I’m now using it every time I game. I created a bunch of issues on Github to track new features I want to add in order to get to what I think will be a 1.0 release. I’m excited at the progress I’ve made and that the software continues to have interest. As always, the code is available on Github. Here are videos on how to use the GUI:
New Release of my Extra Life Donations Tracker Software
If you follow the blog, you remember I’ve been working on a GUI for the donation tracker. Part of making that work involved changing the code to be object oriented. Also, it required moving the user-defined configs to be moved out of the main code. I’ve done that and finally tested that it works (I had a silly bug involving a typo that I had to correct) and merged the code into master over on the github repo. The new code required a change in the directions, so I made new instructional videos to go along with those changes:
Exploring btrfs for backups Part 6: Backup Drives and changing RAID levels VM
Hard drives are relatively cheap, especially nowadays. But I still want to stay within my budget as I setup my backups and system redundancies. So, ideally, for my backup RAID I’d take advantage of btrs’ ability to change RAID types on the fly and start off with one drive. Then I’d add another and go to RAID1. Then another and RAID5. Finally, the fourth drive and RAID6. At that point I’d have to be under some sort of Job-like God/Devil curse if all my drives failed at once, negating the point of the RAID. The best thinking right now is that you want to have backups, but want to try not to have to use them because of both offline time and the fact that a restore is never as clean as you hope it’ll be.
Exploring btrfs for backups Part 5: RAID1 on the Main Disks in the VM
So, back when I started this project, I laid out that one of the reasons I wanted to use btrfs on my home directory (don’t think it’s ready for / just yet) is that with RAID1, btrfs is self-healing. Obviously, magic can’t be done, but a checksum is stored as part of the data’s metadata and if the file doesn’t match the checksum on one disk, but does on the other, the file can be fixed. This can help protect against bitrot, which is the biggest thing that’s going to keep our children’s digital photos from lasting as long as the ones printed on archival paper. So, like I did the first time, I’ll first be trying it out in a Fedora VM that mostly matches my version, kernel, and btrfs-progs version. So, I went and added another virtual hard drive of the same size to my VM.
Exploring btrfs for backups Part 4: Weekly Culls and Unit Testing
Back in August I finally had some time to do some things I’d been wanting to do with my Snap-in-Time btrfs program for a while now. First of all, I finally added the weekly code. So now my snapshots are cleaned up every three days and then every other week. Next on the docket is quarterly cleanups followed up yearly cleanups. Second, the big thing I’d wanted to do for a while now: come up with unit tests! Much more robust than my debug code and testing scripts, it helped me find corner cases. If you look at my git logs you can see that it helped me little-by-little figure out just what I needed to do as well as when my “fixes” broke other things. Yay! My first personal project with regression testing!
Exploring btrfs for backups Part 3: The Script in Practice
Night of the second day:
# btrfs sub list /home
ID 275 gen 3201 top level 5 path home
ID 1021 gen 3193 top level 275 path .snapshots
ID 1023 gen 1653 top level 275 path .snapshots/2014-03-13-2146
ID 1024 gen 1697 top level 275 path .snapshots/2014-03-13-2210
ID 1025 gen 1775 top level 275 path .snapshots/2014-03-13-2300
ID 1027 gen 1876 top level 275 path .snapshots/2014-03-14-0000
ID 1028 gen 1961 top level 275 path .snapshots/2014-03-14-0100
ID 1029 gen 2032 top level 275 path .snapshots/2014-03-14-0200
ID 1030 gen 2105 top level 275 path .snapshots/2014-03-14-0300
ID 1031 gen 2211 top level 275 path .snapshots/2014-03-14-0400
ID 1032 gen 2284 top level 275 path .snapshots/2014-03-14-0500
ID 1033 gen 2357 top level 275 path .snapshots/2014-03-14-0600
ID 1035 gen 2430 top level 275 path .snapshots/2014-03-14-0700
ID 1036 gen 2506 top level 275 path .snapshots/2014-03-14-0800
ID 1037 gen 2587 top level 275 path .snapshots/2014-03-14-0900
ID 1038 gen 2667 top level 275 path .snapshots/2014-03-14-1700
ID 1039 gen 2774 top level 275 path .snapshots/2014-03-14-1800
ID 1040 gen 2879 top level 275 path .snapshots/2014-03-14-1900
ID 1041 gen 2982 top level 275 path .snapshots/2014-03-14-2000
ID 1042 gen 3088 top level 275 path .snapshots/2014-03-14-2100
ID 1043 gen 3193 top level 275 path .snapshots/2014-03-14-2200
Morning of the third day:
Exploring btrfs for backups Part 2: Installing on My /Home Directory and using my new Python Script
I got my new hard drive that would replace my old, aging /home hard drive. As you read in part 1, I wanted to put btrfs on it. This is my journey to get it up and running. Plugged it into my hard drive toaster and ran gparted.
[caption id=“attachment_7889” align=“aligncenter” width=“421”] Gparted for new drive[/caption]
[caption id=“attachment_7890” align=“aligncenter” width=“500”] Gparted for new drive1[/caption]
My Second Ever Useful GUI Program
A while ago I wrote about my first ever useful GUI program. And in one of the series of posts that followed I explained that the reason I hadn’t made a useful GUI program before now is because all the typical stuff has already been made over and over. We don’t need any more tetris clones, word processors, or music programs. Recently I had reason to create my second program. A while ago I created a an amortization calculator in python which you can find here.
Developing my first plasmoid: The Data Engine (in python)
I figured it’d be neat to show you how my plasmoid works so you could use it when developing your own plasmoids. Here’s the main.py of my data engine. The indentation is off, in case you try to copy and past this in.
Here are the imports:
from PyQt4.QtCore import * from PyKDE4.kdecore import * from PyKDE4 import plasmascript #for flickr import views
Those are pretty standard. The last one is the part of my engine that interacts with flickr. Right now I have some work to do to get that presentable, but all you need to know is that it outputs XML to this main part of the data engine.
Programming to the Rescue: Amortization Automation
What I love about programming is the instant feedback. In most programming languages, after you set up a framework for the barest bones of a program you can then run it at every step of the way to confirm that you are moving towards your goal. What I love second-most about programming is the fact that I am using my computer to solve a problem and free myself from tedium.
Philosophizing on the Move can lead to trouble
The plot twist at the end is awesome, but so is the programmer’s debate going on, especially since we were talking about this at work this week. Another great xkcd from Randall Monroe:
More Flickr Hacking
Over the last two days, between playing Portal and Age of Empires III (both of which are…awesome!), I’ve been doing a little more flickr hacking. This is very closely related to my previous Flickr program where I checked if my pictures were ready to be moved into the next views groups.
This time I wanted to see which of my photos were not in any groups. So in the case that I’ve missed putting a photo into the 25 views groups because it’s a few pages back and I really don’t have time to go back through all of my photos to find the pictures to put into the groups. I tried that once and it was excrutiatingly slow - even slower than figuring out if it was time to graduate my pictures out of the views groups. So here’s the output of my current program:
Hacking Flickr
So, thanks to a problem I had with rigging up Nick in " Sugar", I’ve temporarily lost interest in animation. This happens to me all the time, and I think I’ve mentioned it before in previous blog posts that I’m too lazy to look up before that I tend to gain and lose interest in my hobbies. For example, I haven’t touched the Wii since I beat Mario Galaxy during Christmas and until this recent problem with “Sugar” happened, I hadn’t done any programming in about a year. Even my interest in photography has ebbed so much that I let my subscription to Digital Camera Magazine lapse.
Introducing my SECOND CGI program: Eric's BlogRoll
You can find it here or on the right-hand side under “My pages”. This is a program that’s been in my head for about four days and which I spent most of yesterday working on until I realized I was going down the wrong path. Ah, but I’m getting ahead of myself. This program is my attempt to bridge the borders which artifically divide us in the blog world, while also providing myself with a convinience. The page, if you haven’t visited it yet, displays blogs from my blogroll all on one page. Live Journal, a blogging site, allows you to do this, but only with other LJ users. Similarly, Tripod blogs allow you to do it only with other Tripod users. However, they failed to appreciate the power of hacker bloggers, such as myself.
Two Programming Firsts! (for me)
It took me the better part of about 2 hours to figure out how to get everything to work correctly, but I am finally done with my program, GNaughtyNice. It figures out how naughty or nice a word is by using the Google search API and searching once with the porn filter on and once with it off. You can access this great feat of programming at http://server.ericsbinaryworld.com/GNaughtyNice.html
I think it’ll probably make a pretty god party game. You also need a Google search API key because I only get 1000 searches which could get used up quite quickly if this were to become popular enough.