I’ve been hearing about Podman for a while now – at Red Hat Summit and at various local Red Hat presentations. I’ve seen the slides where the RHEL presenter (it’s always the same guy, but I’m terrible with names – after a bit of research, I think it’s Dan Walsh) asks you to pledge to call them container images, not Docker images, etc. But up until now, even though I’m a huge Red Hat fan, I’ve continued to use Docker as my container engine because I am just running a few containers for myself. I don’t even use a one-machine Docker Swarm. I use docker-compose. And that’s just not something that Podman is ever going to officially support. This makes sense because Red Hat is thinking enterprise. And in the enterprise there are two scenarios: 1) Orchestration – vanilla Kubernetes, OpenShift, etc – and 2) are devs running docker run (or podman run) to test the images before putting them into the orchestrator. I’m an anti-pattern, even if I’m not the only one doing things this way.
Recently I’ve been thinking of converting over to Podman from Docker. There are a few reasons. First of all, Docker requires running a daemon. Not only does that use more resources, but it provides a target for exploitation. In fact, there’s currently a crypto-miner worm workings its way through vulnerable Docker servers. Also, the daemon runs as root and that makes it dangerous if there’s an escape. Podman doesn’t have a daemon. And it is built to be able to be run by users. Again, remember that the use case for Podman that Red Hat is targeting is the dev on his own computer or laptop that’s testing something that will eventually be put onto an orchestrator. So they want them to be able to run Podman as a regular user.
But there’s another reason and it’s something so subtle that it’s been escaping me until this week. If you look at all of Red Hat’s material convincing folks to use Podman, it’s all around how it’s a drop-in replacement for Docker. But recently it’s dawned on me, as I’ve read a bit more about podman and reflected on various talks I’ve heard about it, there’s something hidden right there in the name. Why is it “pod” man and not “container” man? Well, besides containerman being a much longer command to type, it’s because while on the surface podman is about RHEL’s Docker replacement, under the hood it’s related to Kubernetes. In Kubernetes (from now on referred to as k8s) the smallest unit of management is the pod (which can have 1 to many containers). So when you run Podman as a drop-in for Docker (Red Hat even mentions using alias to help with muscle memory), it’s just creating 1-container pods. But you could actually use podman to create multi-container pods. And, in the same way that docker-compose.yml is used both for Docker Compose and Docker Swarm, the yaml that you get from Podman can be used for Kubernetes distros.
So, then comes the reason for the question that is the title of this blog post. If I’m going to transition from Docker to Podman, I’m pretty sure it’s not going to happen perfectly without any issues. After all, the Podman as drop-in replacement for Docker works in the simplest of use-cases. But some of the containers I’m using – like Calibre-web – may be making use of Docker-isms rather than standard OCI container features. So I’d like to be able to do a phased transition so that I don’t have to either take an entire day off from work or spend a weekend trying to get everything working. A weekend in which the family is upset that things aren’t working because my homelab runs the house’s tech.
To test this, I fired up a Fedora 30 server edition VM and an isolated network where I would install Docker and make sure that was working and then try and install Podman at the same time and see if that would work. Why Fedora 30 when 31 is already out? Because another reason I’ve become interested in Podman is because Fedora 31 moves on to cgroupsv2 and Docker doesn’t support that yet. There’s a command that can be used to turn it off, but if I have any issues, I’d rather not have it be because of an extra variable. So Fedora 30 it is.
I went to Docker, downloaded their repo and installed via instructions here. And containers were up and running.
I went ahead and picked PHP IPAM.
And I went through the config scripts. Here we are:
OK, now that I know I have a working Docker VM, it’s time to set up Podman. I’ve found one easy way to set it up AND have it working in Cockpit is to install the package cockpit-podman. This leads to the following Cockpit screen for Podman:
This is as far as I’d gotten on my production system. I was afraid that turning on the service would wreck Docker. ALSO, what service? I thought Podman ran without a daemon? As this page explains, it’s basically using systemd to do your container management. So I clicked on Start Podman.
I clicked around in my Docker PHP admin website and it still worked. So running Podman didn’t kill Docker. Huzzah! That’s great to know if you’re doing migrations. You can see that the images aren’t there. This is because Podman stores images in a different directory than Docker does. I’m going to try and create a pod with phpIPAM and MYSQL that can run on a different set of ports in parallel.
To start off, I ran
podman run -dt --pod new:phpIPAM -e MYSQL_ROOT_PASSWORD=my-secret-pw -v /root/phpipam-podman:/var/lib/mysql -d mysql:5.6
This would have put it into a pod named phpIPAM and saved /var/lib/mysql into the directory /root/phpipam-podman.
And if we do:
# podman pod ps POD ID NAME STATUS CREATED # OF CONTAINERS INFRA ID 3040956968bd phpIPAM Running About a minute ago 2 ca94fe7c5a5e
I’m *slightly* concerned at this point that they both supposedly expose the same port, but I didn’t explicitly expose it to the box, so maybe it’ll be OK. Now let’s add the phpIPAM part to that pod. Unlike with the Docker example, I couldn’t run the exact command from Docker Hub because the –link command was unrecognized. I’m hoping that having them in the same pod mitigates that. Command was:
podman run -dt --pod phpIPAM -p 8080:80 -e MYSQL_ENV_MYSQL_ROOT_PASSWORD=my-secret-pw pierrecdn/phpipam
I notice that putting containers into pods takes slightly longer than starting up Docker containers.
I got the error:
Error: cannot set port bindings on an existing container network namespace
So maybe I needed to set the port when first creating the pod. A quick search on the net seemed to suggest this was true. I couldn’t figure out how to remove the pod because it complained about having containers inside it. So for now, since this is a VM I can just throw away, I’m going to make a phpIPAM2 pod.
# podman pod create --name phpIPAM2 -p 8080 # podman run -dt --pod phpIPAM2 -e MYSQL_ROOT_PASSWORD=my-secret-pw -v /root/phpipam-podman:/var/lib/mysql -d mysql:5.6 # podman run -dt --pod phpIPAM2 -e MYSQL_ENV_MYSQL_ROOT_PASSWORD=my-secret-pw pierrecdn/phpipam
# podman pod ps POD ID NAME STATUS CREATED # OF CONTAINERS INFRA ID 5387ffc281ae phpIPAM2 Running 4 minutes ago 3 e2c6c36682a0 3040956968bd phpIPAM Running 18 minutes ago 3 ca94fe7c5a5e
But I wasn’t QUITE where I needed to be as you can see here:
A few things to note here:
- It’s doing 8080->8080
- There’s a “pause” container represented in here per each pod. But otherwise the Podman view in Cockpit is kind of unhelpful for creating pods. It is fine if you’re just doing things the Docker way.
- At least it’s a decent list of your images.
OK, let’s try this ONE MORE TIME!
# podman pod create --name phpIPAM3 -p 8081:80 # podman run -dt --pod phpIPAM3 -e MYSQL_ROOT_PASSWORD=my-secret-pw -v /root/phpipam-podman:/var/lib/mysql -d mysql:5.6 # podman run -dt --pod phpIPAM3 -e MYSQL_ENV_MYSQL_ROOT_PASSWORD=my-secret-pw pierrecdn/phpipam # podman pod ps POD ID NAME STATUS CREATED # OF CONTAINERS INFRA ID 9a83c0ea0089 phpIPAM3 Running 2 minutes ago 3 ddc4589ba911 5387ffc281ae phpIPAM2 Running 11 minutes ago 3 e2c6c36682a0 3040956968bd phpIPAM Running 26 minutes ago 3 ca94fe7c5a5e
I’m closer, but it doesn’t answer on 8081.
Ah, it turns out that, unlike Docker, Podman does not punch hole through the firewall. I had to open up Port 8081. Now the only problem is that the mysql container in that pod exited. So now what? Hmm….this time it was SELinux causing problems. I couldn’t tried just a little to figure it out, but for the sake of getting things running and the fact that this VM is on on isolated network, I just turned SELinux off. (But it’s good to know that SELinux helps protect the system when using Podman). After all this it still wasn’t working because the phpIPAM container was complaining it couldn’t get to the SQL database. So I figured I’d try one more thing – name sure I use the “name” directive in the container creation.
# podman pod create --name phpIPAM4 -p 8081:80 # podman run -dt --pod phpIPAM4 --name phpipam-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -v /root/phpipam-podman:/var/lib/mysql -d mysql:5.6 # podman run -dt --pod phpIPAM4 -e MYSQL_ENV_MYSQL_ROOT_PASSWORD=my-secret-pw --name ipam pierrecdn/phpipam
When I issued the following command:
podman generate kube phpIPAM4 > phpIPAM4.yaml
I think I may have figured out the issue. Because it looks like:
image: docker.io/library/mysql:5.6 name: phpipam-mysql ports: - containerPort: 80 hostPort: 8081 protocol: TCP
And I’m pretty sure that should belong to the phpIPAM container, not the MySQL container. I wonder if creation order matters?
# podman pod create --name phpIPAM5 -p 8081:80 # podman run -dt --pod phpIPAM5 -e MYSQL_ENV_MYSQL_ROOT_PASSWORD=my-secret-pw pierrecdn/phpipam # podman run -dt --pod phpIPAM5 -e MYSQL_ROOT_PASSWORD=my-secret-pw -v /root/phpipam-podman:/var/lib/mysql -d mysql:5.6
And then…….IT WORKED. IT FREAKIN’ WORKED! So the port that’s exposed should go to the first container you add. Hmm… could be annoying for complex pods.
Well, it didn’t 100% work because it’s expecting a certain name for the MySQL container and since we couldn’t provide the –link, that didn’t happen. But I could probably fix that by giving the MySQL container the right name. I’ll try that tomorrow. It’s been an interesting 2 hours already.