See how you can run a user space podman container inside another non-privileged container. For that, we create a CentOS 7 image with podman v3 installed. We spin up a Kubernetes non-privileged container from this image, and we show that we are able to run other podman containers successfully.

Tested on Kubernetes v1.22.9 with CentOS 7 Kubernetes agents and containerd container runtime v1.5.11

This is a tl;dr („too long; didn’t read“) style blog post that consists of headlines and code only. For questions, please add comments to the blog post.

Test Podman in a non-privileged Container online (free)

You can test podman in a Kubernetes container online on https://cloud.vocon-it.com/products. The (experimental) PyCharm service is based on a Podman container inside the Kubernetes container:

Instead of waiting PyCharm to spin up, you can open a terminal and test podman:

Running Podman online in a Kubernetes Container

As you can see from the PyCharm window, which is spinning up automatically, you can see that you can run X Window containers inside the Kubernetes container:

Running an Y Wndow Container (PyCharm) inside a Podman Container inside a Kubernetes Container

Do you want to do it yourself? No problem. Here is how:

Dockerfile

cat <<EOF > Dockerfile.centos7-with-podman-and-fuse
FROM centos:7

RUN \
  yum -y reinstall shadow-utils \
  && yum -y install podman fuse-overlayfs \
  && rm -rf /var/cache /var/log/dnf* /var/log/yum.*

RUN useradd podman \
  && echo podman:10000:5000 > /etc/subuid \
  && echo podman:10000:5000 > /etc/subgid

VOLUME /var/lib/containers
VOLUME /home/podman/.local/share/containers

ADD https://raw.githubusercontent.com/containers/libpod/master/contrib/podmanimage/stable/containers.conf /etc/containers/containers.conf
ADD https://raw.githubusercontent.com/containers/libpod/master/contrib/podmanimage/stable/podman-containers.conf /home/podman/.config/containers/containers.conf

RUN chown podman:podman -R /home/podman

# chmod containers.conf and adjust storage.conf to enable Fuse storage.
RUN chmod 644 /etc/containers/containers.conf \
  && sed -i -e 's|^#mount_program|mount_program|g' -e '/additionalimage.*/a "/var/lib/shared",' -e 's|^mountopt[[:space:]]*=.*$|mountopt = "nodev,fsync=0"|g' /etc/containers/storage.conf
RUN mkdir -p /var/lib/shared/overlay-images /var/lib/shared/overlay-layers /var/lib/shared/vfs-images /var/lib/shared/vfs-layers \
  && touch /var/lib/shared/overlay-images/images.lock \
  && touch /var/lib/shared/overlay-layers/layers.lock \
  && touch /var/lib/shared/vfs-images/images.lock \
  && touch /var/lib/shared/vfs-layers/layers.lock

ENV _CONTAINERS_USERNS_CONFIGURED=""

RUN podman version

# INFO: podman 1.6.4 did not work. podman run created an error Error: stat /sys/fs/cgroup/systemd/system.slice/containerd.service/kubepods-besteffort-podf422c927... command terminated with exit code 125

#
# Upgrade podman from 1.6.4 to 3.4.4
#

# Install podman v3 Deps
RUN yum install -y sudo \
  && yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm \
  && yum install -y \
       "@Development Tools" \
       curl \
       gcc \
       make \
       device-mapper-devel \
       git \
       btrfs-progs-devel \
       conmon \
       containernetworking-plugins \
       containers-common \
       git \
       glib2-devel \
       glibc-devel \
       glibc-static \
       golang-github-cpuguy83-md2man \
       gpgme-devel \
       iptables \
       libassuan-devel \
       libgpg-error-devel \
       libseccomp-devel \
       libselinux-devel \
       pkgconfig \
       systemd-devel \
       autoconf \
       python3 \
       python3-devel \
       python3-pip \
       yajl-devel \
       libcap-devel \
       jq \
       go

# Install conmon
RUN git clone https://github.com/containers/conmon \
  && cd conmon \
  && export GOCACHE="$(mktemp -d)" \
  && make \
  && sudo make podman \
  && cd .. \
  && conmon --version

# Install policies
RUN mkdir -p /etc/containers \
  && sudo curl -L -o /etc/containers/registries.conf https://src.fedoraproject.org/rpms/containers-common/raw/main/f/registries.conf \
  && sudo curl -L https://src.fedoraproject.org/rpms/containers-common/raw/main/f/default-policy.json | jq 'del(.transports.docker)' > /etc/containers/policy.json

# Install CentOS 7 friendly crun without systemd
RUN curl -s -L -o /usr/bin/crun https://github.com/alvistack/crun/releases/download/0.14.1/crun-0.14.1-linux-amd64 \
  && chmod +x /usr/bin/crun \
  && crun -V

# Install podman
RUN TAG="v3.4.4" \
  && rm -rf podman* \
  && curl -O -L https://github.com/containers/podman/archive/refs/tags/${TAG}.tar.gz \
  && tar xvf ${TAG}.tar.gz \
  && cd podman*/ \
  && make BUILDTAGS="selinux seccomp" \
  && make install PREFIX=/usr
EOF

Docker build & push

docker build . -f "Dockerfile.centos7-with-podman-and-fuse" -t vocon/podman:centos7
docker push vocon/podman:centos7

Prepare Kubernetes Agent

Enable User Namespaces temporarily (for testing)

cat /proc/sys/user/max_user_namespaces | egrep -q '^0$' \
  && echo "Enabling User Namespaces" \
  && echo 1000000 | sudo tee /proc/sys/user/max_user_namespaces

Enable User Namespaces permanently (surviving reboot)

cat <<EOF | sudo tee /etc/systemd/system/enable-user-namespaces.service
[Unit]
Description=Enable User Namespaces
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/sh -c '/usr/bin/echo 100000 > /proc/sys/user/max_user_namespaces'
TimeoutStartSec=0

[Install]
WantedBy=default.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable enable-user-namespaces.service
sudo systemctl start enable-user-namespaces.service
sudo systemctl status enable-user-namespaces.service

Install Fuse Device Plugin:

kubectl apply -f https://raw.githubusercontent.com/kuberenetes-learning-group/fuse-device-plugin/master/fuse-device-plugin-k8s-1.16.yml

# output: daemonset.apps/fuse-device-plugin-daemonset created

Please do not be confused by the „1.16“ in the link name. The link name officially works for all kubernetes versions >= v1.16.

Create a POD on Kubernetes

mkdir -p ${HOME}/.local/share/containers
chown -R 1000:1000 ${HOME}/.local/share/containers
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
 name: podman-centos7-no-priv
spec:
 containers:
   - name: podman-centos7-no-priv
     image: docker.io/vocon/podman:centos7
     args:
       - sleep
       - "1000000"
     securityContext:
       runAsUser: 1000
     resources:
       limits:
         github.com/fuse: 1
     volumeMounts:
       - mountPath: /home/podman/.local/share/containers
         name: podman-local
 volumes:
   - name: podman-local
     hostPath:
       path: ${HOME}/.local/share/containers
EOF

Run podman inside POD

Check user id

kubectl exec -it podman-centos7-no-priv -- sh -c 'id'

# output:
uid=1000(podman) gid=1000(podman) groups=1000(podman)

podman run

podman run --rm -it alpine echo hello podman
Output
? Please select an image:
    registry.fedoraproject.org/apline:latest
    registry.access.redhat.com/apline:latest
  ? docker.io/library/apline:latest #          <---------------- choose this one 
    quay.io/apline:latest
Resolved "alpine" as an alias (/headless/.cache/containers/short-name-aliases.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob 2408cc74d12b done
Copying config e66264b987 done
Writing manifest to image destination
Storing signatures
hello podman
Running non-privileged Podman in a container works!

Caveat

podman build does not (yet) work; we need to investigate, what is the exact reason…

58 comments

  2. My brother suggested I might like this website He was totally right This post actually made my day You cannt imagine just how much time I had spent for this information Thanks

    Antworten

  4. helloI really like your writing so a lot share we keep up a correspondence extra approximately your post on AOL I need an expert in this house to unravel my problem May be that is you Taking a look ahead to see you

    Antworten

  6. Normally I do not read article on blogs however I would like to say that this writeup very forced me to try and do so Your writing style has been amazed me Thanks quite great post

    Antworten

  7. I am not sure where youre getting your info but good topic I needs to spend some time learning much more or understanding more Thanks for magnificent info I was looking for this information for my mission

    Antworten

  10. Modern Talking ??? ???????? ??????, ?????????????? ? 1984 ????. ?? ???? ????? ?? ????? ????? ?????????????? ????????? ? ????????? ????????? ?????? ????????????? ????????. ?????? ????? ???????? „You’re My Heart, You’re My Soul“, „Brother Louie“, „Cheri, Cheri Lady“ ? „Geronimo’s Cadillac“. ?? ?????? ???????? ???????????? ???? ? ??????? ???-??????, ?????????? ?????????? ?????? ?????????????? ????????? ? ??????????????? ????????. Modern Talking ?????????? ???? ?????????? ? ? ???? ???, ????????? ????? ?? ???????? ????? ?????. ?????? 2024 ???? ??????? ?????? ? ??????? ????????? mp3.

    Antworten

  16. I think every concept you put up in your post is strong and will undoubtedly be implemented. Still, the posts are too brief for inexperienced readers. Would you kindly extend them a little bit from now on? I appreciate the post.

    Antworten

  19. I loved as much as youll receive carried out right here The sketch is tasteful your authored material stylish nonetheless you command get bought an nervousness over that you wish be delivering the following unwell unquestionably come more formerly again since exactly the same nearly a lot often inside case you shield this hike

    Antworten

  20. Thank you for the auspicious writeup It in fact was a amusement account it Look advanced to more added agreeable from you By the way how could we communicate

    Antworten

  21. I loved as much as youll receive carried out right here The sketch is attractive your authored material stylish nonetheless you command get bought an nervousness over that you wish be delivering the following unwell unquestionably come more formerly again as exactly the same nearly a lot often inside case you shield this hike

    Antworten

  22. Somebody essentially lend a hand to make significantly articles Id state That is the very first time I frequented your website page and up to now I surprised with the research you made to make this actual submit amazing Wonderful task

    Antworten

  23. What a fantastic resource! The articles are meticulously crafted, offering a perfect balance of depth and accessibility. I always walk away having gained new understanding. My sincere appreciation to the team behind this outstanding website.

    Antworten

  26. I just could not depart your web site prior to suggesting that I really loved the usual info an individual supply in your visitors Is gonna be back regularly to check up on new posts

    Antworten

  27. Thanks I have just been looking for information about this subject for a long time and yours is the best Ive discovered till now However what in regards to the bottom line Are you certain in regards to the supply

    Antworten

  28. Somebody essentially help to make significantly articles Id state This is the first time I frequented your web page and up to now I surprised with the research you made to make this actual post incredible Fantastic job

    Antworten

  29. I was recommended this website by my cousin I am not sure whether this post is written by him as nobody else know such detailed about my difficulty You are wonderful Thanks

    Antworten

  30. Normally I do not read article on blogs however I would like to say that this writeup very forced me to try and do so Your writing style has been amazed me Thanks quite great post

    Antworten

  31. I was recommended this website by my cousin I am not sure whether this post is written by him as nobody else know such detailed about my trouble You are amazing Thanks

    Antworten

  32. Somebody essentially help to make significantly articles Id state This is the first time I frequented your web page and up to now I surprised with the research you made to make this actual post incredible Fantastic job

    Antworten

  33. Excellent blog here Also your website loads up very fast What web host are you using Can I get your affiliate link to your host I wish my web site loaded up as quickly as yours lol

    Antworten

  34. I do trust all the ideas youve presented in your post They are really convincing and will definitely work Nonetheless the posts are too short for newbies May just you please lengthen them a bit from next time Thank you for the post

    Antworten

  35. I loved as much as you will receive carried out right here The sketch is tasteful your authored subject matter stylish nonetheless you command get got an edginess over that you wish be delivering the following unwell unquestionably come further formerly again as exactly the same nearly very often inside case you shield this hike

    Antworten

  37. certainly like your website but you need to take a look at the spelling on quite a few of your posts Many of them are rife with spelling problems and I find it very troublesome to inform the reality nevertheless I will definitely come back again

    Antworten

  38. you are in reality a just right webmaster The site loading velocity is incredible It seems that you are doing any unique trick In addition The contents are masterwork you have performed a wonderful task on this topic

    Antworten

  39. Hello Neat post Theres an issue together with your site in internet explorer would check this IE still is the marketplace chief and a large element of other folks will leave out your magnificent writing due to this problem

    Antworten

  40. I was suggested this web site by my cousin Im not sure whether this post is written by him as no one else know such detailed about my trouble You are incredible Thanks

    Antworten

  41. Fantastic beat I would like to apprentice while you amend your web site how could i subscribe for a blog site The account helped me a acceptable deal I had been a little bit acquainted of this your broadcast offered bright clear concept

    Antworten

  42. of course like your website but you have to check the spelling on several of your posts A number of them are rife with spelling issues and I in finding it very troublesome to inform the reality on the other hand I will certainly come back again

    Antworten

  44. Its like you read my mind You appear to know a lot about this like you wrote the book in it or something I think that you could do with some pics to drive the message home a little bit but instead of that this is fantastic blog An excellent read I will certainly be back

    Antworten

  45. Obrigado, recentemente estive procurando informações sobre este assunto há algum tempo e a sua é a maior que descobri até agora. Mas e em relação aos resultados financeiros Você tem certeza em relação ao fornecimento

    Antworten

  46. Your writing is like a breath of fresh air in the often stale world of online content. Your unique perspective and engaging style set you apart from the crowd. Thank you for sharing your talents with us.

    Antworten

  47. Wonderful beat I wish to apprentice while you amend your web site how could i subscribe for a blog web site The account aided me a acceptable deal I had been a little bit acquainted of this your broadcast provided bright clear idea

    Antworten

  48. I have been browsing online more than three hours today yet I never found any interesting article like yours It is pretty worth enough for me In my view if all website owners and bloggers made good content as you did the internet will be a lot more useful than ever before

    Antworten

  49. Eu li algumas coisas excelentes aqui Definitivamente vale a pena marcar como favorito para revisitar Eu me pergunto quanto esforço você fez para fazer esse tipo de site informativo excelente

    Antworten

  50. Hi i think that i saw you visited my web site thus i came to Return the favore Im attempting to find things to enhance my siteI suppose its ok to use a few of your ideas

    Antworten

  52. Obrigado, estou procurando informações sobre esse tópico há algum tempo e a sua é a melhor que descobri até agora. Mas e em relação aos resultados financeiros? Você tem certeza sobre o fornecimento

    Antworten

  54. Somebody essentially lend a hand to make significantly posts I might state That is the very first time I frequented your web page and up to now I surprised with the research you made to create this particular put up amazing Excellent job

    Antworten

  58. of course like your website but you have to check the spelling on several of your posts A number of them are rife with spelling issues and I in finding it very troublesome to inform the reality on the other hand I will certainly come back again

    Antworten

Comments

Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.