setup shadow tests in ubuntu container

setup-shadow-on-ubuntu.bash

#!/bin/bash
cat /etc/apt/sources.list
sed -i '/deb-src/d' /etc/apt/sources.list
sed -i '/^deb /p;s/ /-src /' /etc/apt/sources.list
export DEBIAN_PRIORITY=critical
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get -y dist-upgrade
sudo apt-get -y install ubuntu-dev-tools automake autopoint \
    xsltproc gettext eatmydata expect byacc libtool libbsd-dev \
    pkgconf tmux
su - ubuntu << EOF
git clone https://github.com/shadow-maint/shadow
cd shadow
autoreconf -v -f --install
./autogen.sh --without-selinux --disable-man --with-yescrypt
make -j4
EOF

We could maybe have several Dockerfiles, one for each package manager (maybe something like share/docker/shadow/deb.Dockerfile, share/docker/shadow/rpm.Dockerfile).

I'd prefer one per distribution, and I'm not sure about the location of the files.

Are Dockerfiles for RPM distros (Fedora, SuSE, RHEL, ...) any different at all? I expect the only difference is in the base image, which is already variable with an argument. If there's anything else different, maybe we can make it variable or maybe not. If not, sure, go for separate Dockerfiles.

About the location, I like to use FHS-like tree structures in my repos. That's why I chose ./share, which I use for the same meaning that /usr/share would have in a system. Dockerfiles are arch-independent, so that's a suitable place, I think.

I prefer the Dockerfile over github actions because I can run it locally. I'm not sure if those Dockerfiles can be reused in GH Actions or if we need to repeat everything in there.

GH Actions can use Dockerfiles directly: https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action. It increases the reusability by enabling both local and GH Action execution, so I think it's better to use Dockerfiles.

Nice.

The main point is that I should be able to test locally whatever that CI can test, to the point that CI is unnecessary. Which doesn't mean I'd remove it; it's good to have it there to simplify the workflow with one-time contributors, or to catch something if we forget to run it. But CI should be just a second safety, rather than the only one. I like being able to fully test locally.

I think the CI is more important than the local workflow, but that doesn't make any difference in this conversation as we both agree that we should enable both local and CI execution of those workflows.

:)

Without trying to convince you, I'll just say why I prefer local. It helps me compile, see the problems, and loop again until perfectly clean, very fast (much faster than going through the network, waiting to receive a notification, ...), and also I choose the interface (a terminal, much nicer than GH's or any other web GUI). After it succeeds locally, I know it's going to work in CI even before submitting (of course it may still fail, if I forgot something, or if there's some random behavior, but the chances are pretty low), so I reduce the noisy notifications that others will receive for each iteration of my tests and pushes.

--- Edit:

Oh, and another nice thing of having all tests local: you can always change the CI system; no lock-in, since it should work everywhere.

+1 for no CI system lock-in.

I'd love to see the complete set of commands you run to set up those. I've never worked with them. :)

Probably outside of the scope here mainly because I use different tools in different places :) For vm's on my main 'home' server, I use uvt-kvm, because I can do:

uvt-kvm create --memory 4096 --disk 30 rust release=jammy arch=amd64 --cpu 4 --run-script-once firstboot

For containers on my 'home' server I generally do

lxc-create -t download -n shadow -- -d ubuntu -r jammy - a amd64
lxc-start -n shadow
lxc-attach -n shadow -- << EOF
wget https://gisturl
./gist
EOF

In other environments I use lxd for both containers and vms. Those use cloud-init to specify a firstboot script.

Note that containers for shadow testsuite can be a problem as the testsuite uses a wide uid allocation.