Managing Containers and Images on a Workstation

Definitions

Images are snapshots of a filesystem used to bootstrap a container.

Containers are isolated execution environments with their file system based upon an Image.

Monitoring

You can see which Containers are running on your system with:

podman ps -a

The -a means it will also show stopped containers.

You can see what Images are cached on your system with:

podman images

Images are usually cached from a container registry like ghcr.io. They can usually be deleted without any consequence except waiting for the download next time you launch a container that uses them.

Cleaning Up Space

The container and image cache is stored in /scratch/<fed_id>/podman. If you are short on space you can use the following.

To remove temporary images and stopped containers:

podman system prune

To remove individual images:

podman rmi IMAGE_ID # from third column of podman images

OR ...

podman rmi REPOSITORY:TAG # 1st , 2nd columns of podman images

To remove all images with wildcard on REPOSITORY:

podman rmi $(podman images --filter=reference='*PATTERN*' -q)

# PATTERN matches the REPOSITORY column

Defining a Container Image

The dev-c7 container image is built from the container context folder dev-c7/docker. The context contains a Dockerfile which details the steps to build the container image.

To experiment with making additions to the dev-c7 project, first clone the project and do a test build of the container:

git clone https://dls-controls.github.io/dev-c7/main/index.html
cd dev-c7
./build-dev-c7

build-dev-c7 will create a container image with the tag ghcr.io/dls-controls/dev-c7. This is the same tag that c7 uses so it will launch a container based on your newly built image.

Note

You will likely already have a dev-c7 container running and you must delete that before your new one will launch. Otherwise the launch script will just open a new bash session in the existing container. To do this see Deleting the container.

Now you are ready to make any changes you wish to the Dockerfile and add any additional files to the context (i.e. in the dev-c7/docker folder).

Below is the current Dockerfile, read on for an explanation of the commands used.

# Dockerfile for getting DLS RHEL7 dev environment working in a container
# hosted on RHEL8
#
# This is a stopgap so that we can use effort to promote the
# kubernetes model https://github.com/dls-controls instead of spending
# effort in rebuilding our toolchain for RHEL8

FROM centos:7

# environment
ENV LANG=en_US.UTF-8
ENV LC_ALL=en_US.UTF-8
ENV LC_CTYPE=en_US.UTF-8
ENV MODULEPATH=/etc/scl/modulefiles:/etc/scl/modulefiles:/etc/scl/modulefiles:/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/share/modulefiles:/dls_sw/apps/Modules/modulefiles:/dls_sw/etc/modulefiles:/home/hgv27681/privatemodules:/dls_sw/prod/tools/RHEL8-x86_64/defaults/modulefiles

# switch to the vault for centos packages!
RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo && \
    sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo && \
    sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo

# make QT not complain
ENV QT_X11_NO_MITSHM=1
ENV LIBGL_ALWAYS_INDIRECT=1

RUN yum update -y && \
    # dev tools and libraries
    yum groupinstall -y "Development Tools" && \
    yum install -y glibc.i686 redhat-lsb-core libusbx net-snmp-libs environment-modules git2u net-tools screen cmake lapack-devel readline-devel pcre-devel boost-devel libpcap-devel numactl-libs vim dbus-x11 bind-utils libssh2-devel xorg-x11-server-Xvfb \
    # edm dependencies
    epel-release giflib-devel libXmu-devel libpng-devel libXtst-devel zlib-devel xorg-x11-proto-devel motif-devel libX11-devel libXp-devel libXpm-devel libtirpc-devel \
    # areaDetector dependencies
    libxml2-devel libjpeg-turbo-devel libtiff-devel \
    # Odin dependencies
    hdf5-devel zeromq-devel librdkafka-devel \
    # QT4 dependencies
    pyqt4-devel PackageKit-gtk3-module libcanberra-gtk2 mesa-dri-drivers.x86_64 xcb-util xcb-util-devel libxcb.x86_64 libxkbcommon-x11 \
    # dls-remote-desktop-support
    xfreerdp zenity \
    # useful tools
    sudo meld tk dejavu-sans-mono-fonts gnome-terminal xterm xmessage evince eog firefox java openldap-clients && \
    # These packages are not found in the initial install so try again
    yum install -y zeromq-devel git2u meld && \
    #install libnet library for EPICS module builds
    yum install -y libnet libnet-devel && \
    # clean up caches
    yum clean all && \
    # nasty hack to avoid overlay filesystem problems with --userns=keep-id
    # hopefully can be removed when this PR is released and deployed at DLS
    # https://github.com/containers/fuse-overlayfs/pull/381
    rm -fr /var/lib/yum/yumdb/*

#install the required libusb developer library
RUN yum remove -y libusbx && \
    curl -SL http://rpmfind.net/linux/opensuse/distribution/leap/15.5/repo/oss/x86_64/libusb-1_0-0-1.0.24-150400.3.3.1.x86_64.rpm -o /tmp/libusb-1.rpm && \
    yum install -y /tmp/libusb-1.rpm && \
    rm /tmp/libusb-1.rpm && \
    curl -SL https://rpmfind.net/linux/opensuse/distribution/leap/15.5/repo/oss/x86_64/libusb-1_0-devel-1.0.24-150400.3.3.1.x86_64.rpm -o /tmp/libusb-devel-1.rpm && \
    yum install -y /tmp/libusb-devel-1.rpm && \
    rm /tmp/libusb-devel-1.rpm

# full sudo rights inside the container
COPY /sudoers /etc/sudoers

# Workaround to ensure all locales are available
RUN sed -i "/override_install_langs=en_US/d" /etc/yum.conf
RUN yum reinstall -y glibc-common

# set container time to BST
RUN ln -sf /usr/share/zoneinfo/Europe/London /etc/localtime

# latest git from the endpoint repo https://computingforgeeks.com/install-git-2-on-centos-7/
RUN yum remove -y git git-core && \
    # libcurl-devel for git-remote-https
    yum -y install libcurl-devel && \
    yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm && \
    yum -y install git git-core

# change the nobody account and group IDs to match RedHat
RUN sed -i 's/99:99/65534:65534/' /etc/passwd && \
    sed -i 's/:99:/:65534:/' /etc/group

The base image is selected with the FROM command this defines what the container file system looks like initially. Every command in the Dockerfile then adds a layer of change to the container image.

The majority of the Dockerfile for dev-c7 is using yum install to get packages installed into the OS. Yum is executed via the RUN command. Note that the RUN command takes a single shell command line input.

Note

It is common in Dockerfiles to use && \ to string multiple commands together in a single RUN. The reason to do this is to minimize the number of layers in the container image. In particular, if you want to clean up temporary files, it must be done in the same layer they were created or it will not reduce the size of the resulting image (because there is a layer with the temp files and then another layer that removes them).

The RUN command may execute any command that is available inside the container. For example the above uses a single RUN to get the source for a new version of git and compile it.

Another command in this Dockerfile is ENV which simply sets an environment variable for the processes the container launches.

The COPY command allows you to supply files from the context (dev-c7/docker folder in this case). Note that the path / is the root of the context. You can only copy files from inside the context, thus the context folder holds the entire definition of the image generated.