this post was submitted on 22 Feb 2024
106 points (92.1% liked)

Linux

48182 readers
1130 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
 

I am moving from docker to podman and selinux because I thought that podman is more secure and hence, the future. I thought the transition will be somewhat seamless. I even prepaired containers but once I migrated I still ran into issues.

minor issue: it's podman-compose instead of podman compose. The hyphen feels like a step back because we moved from docker-compose to docker compose. But thT's not a real issue.

podman does not autostart containers after boot. You have to manually start them, or write a start script. Or create a systemd unit for each of them.

Spinning up fresh services works most of the time but using old services that worked great with docker are a pain. I am wasting minutes after minutes because I struggle with permissions and other weird issues.

podman can't use lower number ports such that you have to map the ports outside of the machine and forward them properly.

Documentation and tutorials are "all" for docker. Github issues are "all" for docker. There isn't a lot of information floating around.

I'm still not done and I really wonder why I should move forward and not go back to docker. Painful experience so far. https://linuxhandbook.com/docker-vs-podman/ and following pages helped me a lot to get rid of my frustration with podman.

all 50 comments
sorted by: hot top controversial new old
[–] [email protected] 50 points 8 months ago* (last edited 8 months ago) (2 children)

Your issues stem from going rootless. Podman Compose creates rootless containers and that may or may not be what you want. A lot more configuration needs to be done to get rootless containers working well for persistent services that use low ports, like enabling linger for specific users or enabling low ports for non-root users.

If you want the traditional Docker experience (which is rootful) and figure out the migration towards rootless later, I'd recommend the following:

  1. Install podman-docker. This provides a seamless Docker compatibility layer for podman, allowing you to even use regular docker commands that get translated behind the scenes into Podman.
  2. Install regular docker-compose. This will work via podman-docker and gives you the native docker compose experience.
  3. Enable podman.socket and podman-restart.service. First one socket-activates the central Podman daemon, second one restarts any podman containers with a restart-policy of always on boot.
  4. Run your docker-compose commands using sudo, so sudo docker-compose up -d etc. You can run this with sudo podman compose as well if you're allergic to hyphenation. Podman allows both rootful and rootless containers and the way you choose is by running the commands with sudo or not.

This gets you to a very Docker-like experience and is what I am currently using to host my services. I do plan on getting familiar with rootless and systemd services and Kubernetes files, but I honestly haven't had the time to figure all that out yet.

[–] [email protected] 7 points 8 months ago
[–] [email protected] 2 points 8 months ago* (last edited 8 months ago) (1 children)

Enable podman.socket and podman-restart.service. First one socket-activates the central Podman daemon, second one restarts any podman containers with a restart-policy of always on boot.

Thanks, the last time I checked I was told that creating individual systemd services was the only viable solution and I ended up ditching podman because I didn’t think it was worth the hassle. I might try it again with your tips.

[–] [email protected] 2 points 8 months ago* (last edited 8 months ago)

Definitely not necessary. If that was the case, it wouldn't live up to it's claims of being a transparent Docker replacement at all. I think you do need to use systemd if you want to go full rootless, but I haven't tried it enough to make a solid call on that.

But yeah, with the above steps, I've moved seamlessly over to Podman for my self hosting stack and I've never looked back. It's also great because I can take literally any Docker Compose I find on the Internet and it will most likely just work.

[–] [email protected] 28 points 8 months ago (2 children)

Podman is purposefully built to rely on systemd for running containers at startup. It ties in with the daemonless and rootless conventions. It's also nice because systemd is already highly integrated with the rest of the OS, so doing things like making a container start up after a drive is mounted is trivial.

Podman has a command to generate systemd files for your containers, which you can then use immediately or make some minor tweaks to your liking.

I use podman for my homelab and enjoy it. I like the extra security and that it relies on standard linux systems like systemd and user permissions. It forces me to learn more about linux and things that apply to more than just podman. You can avoid a lot of trouble by running the containers as root and using network=host, but that takes away security and the fun of learning.

[–] [email protected] 5 points 8 months ago

Ooh I didn't know about the systemd integration, that actually sounds like a really smart approach.

To be honest, until right now I'd pretty much written off podman as docker 2

[–] [email protected] 3 points 8 months ago

You can avoid a lot of trouble by running the containers as root and using network=host

Root yes, but you can avoid network=host most of the time pretty easily. I am still struggling with going rootless myself tbh.

[–] [email protected] 28 points 8 months ago (1 children)

Regarding the low port number thing, that’s just a consequence of not running as root. By default, regular users can’t listen on ports below 1000.

[–] [email protected] 20 points 8 months ago (1 children)
[–] [email protected] 41 points 8 months ago (1 children)

You are correct. I’m as bad as hard drive manufacturers.

[–] [email protected] 0 points 8 months ago* (last edited 8 months ago) (1 children)

Well hard drive manufacturers are actually correct. A gigabyte (GB) is in base 10 and thus 1000 megabytes, not 1024. Gibibytes (GiB) are base 2 (hence “bi”) and thus 1024 mebibytes.

[–] [email protected] 2 points 8 months ago

I know it's technically correct but it still hurts a little inside to admit it each time.

I know the reason is because giga is an SI prefix but all the way through my education, 1 GB was taught to be 1024 MB, so I always want to use this instead of what is correct.

To be fair, the tech industry has been naughty with things like this. I know of two. I wonder how many others there are?

I believe that:

  • The style of characters a user can choose is called a typeface. I think every piece of software calls it a font. I remember hearing it came from Apple/Steve Jobs.

  • I believe the use of setup is incorrect. Setup is a noun, so it refers to an existing configuration. It tends to be used when running an OS or program for the first time though, which I believe set up is the correct term. Set up is an adjective and refers to the act of creating the configuration.

I've wondered if these were done due to screen space constraints or aesthetics.

[–] [email protected] 24 points 8 months ago

Writing systemd services for your containers is something yoully have to get used to with podman, pretty much. It's actually very easy with the built in command "podman generate systemd", so you can just do something like " podman generate systemd --name my-container > /etc/systemd/system". I much prefer managing my containers with systemd over the docker daemon. It's nice!

Also, podman can use privileged ports as root, right?

[–] [email protected] 16 points 8 months ago (1 children)

minor issue: it’s podman-compose instead of podman compose. The hyphen feels like a step back because we moved from docker-compose to docker compose. But thT’s not a real issue.

podman does not autostart containers after boot. You have to manually start them, or write a start script. Or create a systemd unit for each of them.

I'm also currently migrating all of my self-hosted services from docker to podman. Look into using Quadlet and systemd rather than podman-compose: https://www.redhat.com/sysadmin/quadlet-podman

Your Quadlet .container files will end up looking very similar to your docker compose files. Podman will automatically generate a systemd service unit for you if you drop the .container file in your user systemd unit directory ($HOME/.config/containers/systemd/) and run systemctl --user daemon-reload. Then starting the container on boot is as simple as systemctl --user enable --now containername.service.

This will not solve your rootful vs. rootless issues, as others have pointed out, but Quadlet/systemd is nice replacement for the service/container management layer instead of docker-compose/podman-compose

[–] [email protected] 8 points 8 months ago

+1 for quadlet. It's another file format to learn, but it's worth it, particularly if you want your containers to auto-update. Also check out podlet to help mitigate some of the compose to .container issues.

[–] [email protected] 10 points 8 months ago

podman does not autostart containers after boot. You have to manually start them, or write a start script. Or create a systemd unit for each of them.

I have not yet tried podman, but I know that podman-compose used to have an option to generate systemd units for your pods: https://docs.podman.io/en/latest/markdown/podman-generate-systemd.1.html

Still, that option has been deprecated in favour of Podman Quadlet https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html

[–] [email protected] 9 points 8 months ago (1 children)

Podman is great, but a lot of confusion arise from the rapid development the last ~year and the fact that different distros have relatively old versions in their repos.

I recommend using the latest Fedora Server and defining your containers as quadlets. Also, on Fedora, yoi can install Cockpit (and cockpit-podman) and get a decent webgui to manage your host and container.

I should just write a blog post about this instead of typing this up on my phone in bed 😆

[–] [email protected] 3 points 8 months ago

Yes please!!!

[–] [email protected] 9 points 8 months ago (1 children)

Almost all of your problems are because you aren't running as root. These aren't bugs. They seem like a pain because you're transitioning from Docker which runs as root (which is ABSOLUTELY INCORRIGIBLE in my opinion).

SELinux is a different story though. Now that's a hard to tame beast. Things go wrong easily if you don't know what you're doing.

I suggest researching more before jumping off into a new technology, you seem like you weren't anticipating some of these problems which adds to the frustration.

[–] [email protected] 1 points 8 months ago

You can run docker rootless too. On local machines running docker with root is a risk that for many is acceptable. On servers and publicly exposed hosts, rootless.

[–] [email protected] 7 points 8 months ago

For the port thing, you can set the net.ipv4.ip_unprivileged_port_start sysctl to a lower value like 80 (may need to go lower if you also do email). It also applies to IPv6.

The default of 1024 is for security, but the actual security granted by it is not really that relevant nowadays. It stems from a time where ports < 1024 were used by machines to trust other machines using stuff like rsh & telnet, and before we considered man-in-the-middle attacks to be practical and relevant. Around the start of this millennium, we learned better. Nowadays we use SSH and everything is encrypted & authenticated.

The only particularly relevant risk is that if you lower it enough to also include SSH's default port 22, some rogue process at startup might make a fake SSH server. That would come along with the scary version of the "host key changed" banner so the risk is not that high. Not very relevant if you're following proper SSH security practices.

[–] [email protected] 6 points 8 months ago

You must have been expecting philadelphia

[–] [email protected] 6 points 8 months ago

it’s podman-compose instead of podman compose

Don't use it, it's not a full replacement. The script is barely maintained and not really "official".

I think before switching from Docker to Podman you should first get proficient in Docker, because Podman is not for beginners (yet).

[–] possiblylinux127 5 points 8 months ago (1 children)

Podman isn't a replacement for docker. Its very similar syntax wise but its not a replacement.

The only thing I use podman for is Jellyfin and distrobox

[–] [email protected] 2 points 8 months ago

Have you tried it with podman-docker? I've basically switched my entire self-hosting stack onto podman without much issue using that compatibility layer.

[–] [email protected] 5 points 8 months ago* (last edited 8 months ago)

You can make actual docker compose use podman by running a user podman docker socket and setting that as an environment variable (export DOCKER_HOST=unix:///run/user/$UID/podman/podman.sock)

https://brandonrozek.com/blog/rootless-docker-compose-podman/

[–] [email protected] 4 points 8 months ago* (last edited 8 months ago) (2 children)

podman does not autostart containers after boot. You have to manually start them, or write a start script. Or create a systemd unit for each of them.

FWIW, I'm on Bluefin-dx (one of uBlue^[1]^'s images) and I've noticed that my containers autostart at boot since I've rebased from Silverblue to Bluefin-dx. Mind you; I'm not necessarily advocating for you to make the switch to Bluefin-dx, but it's at least worth finding out how they've been able to achieve that and perhaps implement their ways for your own benefit.


  1. Which are mostly Fedora Atomic images with some QoL and thus SELinux, Podman (etc.) are just baked in as you would expect.
[–] [email protected] 6 points 8 months ago (1 children)

We use quadlets to manage those containers: https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html

As others in the thread have pointed out just having systemd manage them is the way to go, it's a nice combo!

[–] [email protected] 2 points 8 months ago

Thank you Jorge for chiming in!

[–] possiblylinux127 3 points 8 months ago

Podman compose isn't really usable for me

[–] [email protected] 3 points 8 months ago

For the low-port issue, maybe try something like how K8S tends to handle it:

  • One container that is either rootful or allowed to use low ports. Run a reverse proxy like HAProxy or Envoy in this.
  • All other containers for services, run on high ports, pointing to them in the reverse proxy container's config.
  • Don't use bare http, unless required. Getting valid TLS certs is dead easy and free with LetsEncrypt.
[–] [email protected] 1 points 8 months ago (1 children)

I know this isn't the answer you want, but consider switching away from compose entirely
A local kubernetes instance handles all the routing for me, and since i was using that anyway podman was legitimately a drop in replacement for docker.

Podman is just the tool that creates the container for me, running it gets handled by something else entirely.

Also, i can run podman compose up just fine, no hyphen needed. https://docs.podman.io/en/latest/markdown/podman-compose.1.html

[–] [email protected] 1 points 8 months ago

thx! maybe I'll switch to it in a year or so. For now, I am good in learning new containerization technology

[–] [email protected] 1 points 8 months ago (2 children)

podman does not autostart containers after boot.

Does docker do this? I wrote a systemd unit for my docker container because I thought that there is no way for docker to autostart containers?

[–] [email protected] 11 points 8 months ago (1 children)

It does. You probably did not enable docker.service to start on boot.

[–] [email protected] 1 points 8 months ago

Ok, i need to check this later. Thanks!

[–] [email protected] 5 points 8 months ago

Not sure about straight docker run commands, but if you're using docker compose you can add restart: unless-stopped or always to your docker-compose.yml to have it come up on boot.

[–] [email protected] 1 points 8 months ago* (last edited 8 months ago) (1 children)

Yeah, podman's networking approach sent me back to docker as well. I have a bunch of services that don't even expose their ports to the local network, they just connect to each other, and only the reverse proxy is exposed. Switching to podman would require me to reconfigure all my port mappings to make sure there aren't any conflicts, and then update all the references. It's not a ton of work, but enough to keep me on docker for the time being.

edit: It looks like podman's networking stack has changed since I used it, so this is almost certainly wrong now

[–] [email protected] 1 points 8 months ago (1 children)

Is there a difference between networking approaches?

With rootful podman containers the only difference I noticed is that bridge networks aren't isolated by default.

Why would you need to reconfigure the port mappings?

[–] [email protected] 2 points 8 months ago

Tbh it's possible I messed something up, iirc the bridge network I was using wouldn't work, and it seemed like it would only work in host mode, hence the belief that I needed to remap things. This was over a year ago, and tbh I didn't try very hard to make it work.

[–] [email protected] 1 points 8 months ago

The only reasons I ever use podman is that is efficient with resources, and at work we can't use Docker due to licensing purposes.

[–] [email protected] 1 points 8 months ago (1 children)

Why do you believe podman is more secure than root-less Docker? Please educate me.

I run root-ful and root-less Docker daemons at the same time on the same machine because there are limitations to what you can do without as root privileges. So where possible, containers run in root-less Docker and the lucky few that require root privileges run in root-ful Docker.

[–] [email protected] 1 points 8 months ago