From aa00ac94d0e103031b698945bb3f2ddd82741b2b Mon Sep 17 00:00:00 2001 From: Jinna Kiisuo Date: Mon, 5 Feb 2024 10:06:43 +0200 Subject: [PATCH] Fix debian versioning (#2472) * Make the ubuntu_docker_builder.sh script a bit easier to use - Entrypoint that installs dependencies on launch - Echo hint on how to build deb packages * Change debian packaging version string format to fix sort order The sort order is important so that newer packages are seen as an update and get installed, instead of apt preferring the older versions! With these changes, a package is generated as `kanidm_Ubuntu_22.04_1:1.1.0~rc.15-dev~202401311334+c8a9e2c_x86_64.deb` with the version string `1:1.1.0~rc.15-dev~202401311334+c8a9e2c` Deb package version string comparison is Complex: https://man7.org/linux/man-pages/man7/deb-version.7.html With the previous versioning scheme for dev packages, the git hash ended up getting prioritized over the date string, see for example: `dpkg --compare-versions 1.1.0-rc.15-dev-202401100453666448f lt 1.1.0-rc.15-dev-20240120072786916a3; echo $?` -> 1 (comparison failure) A simple schema change avoiding most dashes could rescue the hash trouble: `dpkg --compare-versions 1.1.0~rc.15-dev-202401100453+666448f lt 1.1.0-rc.15-dev-202401200727+86916a3; echo $?` -> 0 (comparison success) .. But, the second problem is seeing a stable release as newer: `dpkg --compare-versions 1.1.0~rc.15-dev~202401100453+666448f lt 1.1.0; echo $?` -> 1 (comparison failure) .. Which can be solved by forcing the entire dev portion to not be interpreted as a debian version by substituting tildes: `dpkg --compare-versions 1.1.0~rc.15-dev~202401100453+666448f lt 1.1.0; echo $?` -> 0 (comparison success) .. But, old schema versions still seem newer due to their debian version: `dpkg --compare-versions 1.1.0-rc.15-dev-202401100453666448f lt 1.1.0~rc.15-dev~202401200727+86916a3; echo $?` -> 1 (comparison failure) Thus, the only solution is to change the scheme and increment the epoch value once to force all lesser default epoch versions to be seen as older: `dpkg --compare-versions 1.1.0-rc.15-dev-202401100453666448f lt 1:1.1.0~rc.15-dev~202401200727+86916a3; echo $?` -> 0 (comparison success) `dpkg --compare-versions 1:1.1.0~rc.15-dev~202401200727+86916a3 lt 1:1.1.0; echo $?` -> 0 (comparison success) * Drop epoch field from deb filenames GitHub Actions enforces NTFS compatible artifact filenames, ergo the colon required for the epoch field is banned. The epoc is still in the version field itself, just not in the filename. --------- Co-authored-by: Jinna Kiisuo --- CONTRIBUTORS.md | 1 + book/src/packaging_debs.md | 9 ++++----- platform/debian/build_kanidm.sh | 17 ++++++++++++++--- platform/debian/interactive_entrypoint.sh | 13 +++++++++++++ platform/debian/kanidm-unixd/rules | 4 +++- platform/debian/kanidm/rules | 4 +++- platform/debian/kanidmd/rules | 4 +++- platform/debian/ubuntu_docker_builder.sh | 1 + 8 files changed, 42 insertions(+), 11 deletions(-) create mode 100755 platform/debian/interactive_entrypoint.sh diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 617227e34..486433821 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -33,6 +33,7 @@ - philipcristiano - Jianchen Zhao (bolu61) - Allan Zhang (allan2) +- Jinna Kiisuo (jinnatar) ## Acknowledgements diff --git a/book/src/packaging_debs.md b/book/src/packaging_debs.md index 5ba80e472..a035b231b 100644 --- a/book/src/packaging_debs.md +++ b/book/src/packaging_debs.md @@ -6,12 +6,11 @@ This happens in Docker currently, and here's some instructions for doing it for 1. Start in the root directory of the repository. 2. Run `./platform/debian/ubuntu_docker_builder.sh` This'll start a container, mounting the - repository in `~/kanidm/`. -3. Install the required dependencies by running `./scripts/install_ubuntu_dependencies.sh`. -4. Building packages uses make, get a list by running `make -f ./platform/debian/Makefile help` -5. So if you wanted to build the package for the Kanidm CLI, run + repository in `~/kanidm/` and installing dependencies via `./scripts/install_ubuntu_dependencies.sh`. +3. Building packages uses make, get a list by running `make -f ./platform/debian/Makefile help` +4. So if you wanted to build the package for the Kanidm CLI, run `make -f ./platform/debian/Makefile debs/kanidm`. -6. The package will be copied into the `target` directory of the repository on the docker host - not +5. The package will be copied into the `target` directory of the repository on the docker host - not just in the container. ## Adding a package diff --git a/platform/debian/build_kanidm.sh b/platform/debian/build_kanidm.sh index 111f42625..04460c630 100755 --- a/platform/debian/build_kanidm.sh +++ b/platform/debian/build_kanidm.sh @@ -67,7 +67,9 @@ if [ "$(which cargo | wc -l)" -eq 0 ]; then fi # this assumes the versions are in lock-step, which is fine at the moment. -KANIDM_VERSION="$(grep -ioE 'version.*' Cargo.toml | head -n1 | awk '{print $NF}' | tr -d '"')" +# Debian is picky abour dashes in version strings, so a bit of conversion +# is needed for the first one to prevent interference. +KANIDM_VERSION="$(grep -ioE 'version.*' Cargo.toml | head -n1 | awk '{print $NF}' | tr -d '"' | sed -e 's/-/~/')" # if we're in a github action, then it's easy to get the commit if [ -n "${GITHUB_SHA}" ]; then @@ -80,8 +82,17 @@ fi GIT_COMMIT="${GIT_HEAD:0:7}" DATESTR="$(date +%Y%m%d%H%M)" -PACKAGE_VERSION="${KANIDM_VERSION}-${DATESTR}${GIT_COMMIT}" -echo "Package Version: ${PACKAGE_VERSION}" + +# Due to previous version schemes we need to increment epoch above the default 0, +# to supercede old versions before the change. +EPOCH=1 + +# GitHub Actions forces NTFS compatibility which disallows colons in filenames +# ergo, we do not include the epoch in the filename. +FILENAME="${KANIDM_VERSION}~${DATESTR}+${GIT_COMMIT}" +PACKAGE_VERSION="${EPOCH}:${FILENAME}" + +echo "Deb package Version: ${PACKAGE_VERSION}" echo "Updating package dir" rm -rf "${BUILD_DIR:?}/*" diff --git a/platform/debian/interactive_entrypoint.sh b/platform/debian/interactive_entrypoint.sh new file mode 100755 index 000000000..434e5ee81 --- /dev/null +++ b/platform/debian/interactive_entrypoint.sh @@ -0,0 +1,13 @@ +#!/bin/bash + +# Install dependencies, for example make! +scripts/install_ubuntu_dependencies.sh + +# Make git happy +git config --global --add safe.directory /root/kanidm + +echo "To launch a deb build, try:" +echo "make -f ./platform/debian/Makefile debs/kanidm" + +# Launch shell +exec /bin/bash "$@" diff --git a/platform/debian/kanidm-unixd/rules b/platform/debian/kanidm-unixd/rules index f07858d5b..774b35778 100755 --- a/platform/debian/kanidm-unixd/rules +++ b/platform/debian/kanidm-unixd/rules @@ -15,7 +15,9 @@ PAMDIR=${PKGDIR}/usr/share/pam-configs/ DISTRIBUTOR_ID=$(shell lsb_release -is) DISTRIBUTOR_RELEASE=$(shell lsb_release -rs) DISTRIBUTOR=$(DISTRIBUTOR_ID)_$(DISTRIBUTOR_RELEASE) -DEB_VERSION_FULL=$(shell dpkg-parsechangelog --show-field Version) + +# GitHub enforces NTFS compatible filenames for artifacts so we need to drop the EPOCH field +DEB_VERSION_FULL=$(shell dpkg-parsechangelog --show-field Version | sed -E 's/^[0-9]+://') %: dh $@ --with systemd diff --git a/platform/debian/kanidm/rules b/platform/debian/kanidm/rules index ee0c3f6b7..e6f7c769b 100755 --- a/platform/debian/kanidm/rules +++ b/platform/debian/kanidm/rules @@ -14,7 +14,9 @@ SHARED_DIR=${PKGDIR}/usr/share/${PACKAGE} DISTRIBUTOR_ID=$(shell lsb_release -is) DISTRIBUTOR_RELEASE=$(shell lsb_release -rs) DISTRIBUTOR=$(DISTRIBUTOR_ID)_$(DISTRIBUTOR_RELEASE) -DEB_VERSION_FULL=$(shell dpkg-parsechangelog --show-field Version) + +#GitHub enforces NTFS compatible filenames for artifacts so we need to drop the EPOCH field +DEB_VERSION_FULL=$(shell dpkg-parsechangelog --show-field Version | sed -E 's/^[0-9]+://') %: dh $@ diff --git a/platform/debian/kanidmd/rules b/platform/debian/kanidmd/rules index 364e881ca..6fe8d3048 100755 --- a/platform/debian/kanidmd/rules +++ b/platform/debian/kanidmd/rules @@ -16,7 +16,9 @@ SHARED_DIR=${PKGDIR}/usr/share/${PACKAGE} DISTRIBUTOR_ID=$(shell lsb_release -is) DISTRIBUTOR_RELEASE=$(shell lsb_release -rs) DISTRIBUTOR=$(DISTRIBUTOR_ID)_$(DISTRIBUTOR_RELEASE) -DEB_VERSION_FULL=$(shell dpkg-parsechangelog --show-field Version) + +#GitHub enforces NTFS compatible filenames for artifacts so we need to drop the EPOCH field +DEB_VERSION_FULL=$(shell dpkg-parsechangelog --show-field Version | sed -E 's/^[0-9]+://') %: dh $@ --with systemd diff --git a/platform/debian/ubuntu_docker_builder.sh b/platform/debian/ubuntu_docker_builder.sh index 4cfcd92e4..ee13c94c5 100755 --- a/platform/debian/ubuntu_docker_builder.sh +++ b/platform/debian/ubuntu_docker_builder.sh @@ -14,4 +14,5 @@ docker run --rm -it \ -e "PACKAGING=1" \ -v "$(pwd):/root/kanidm/" \ --workdir "/root/kanidm/" \ + --entrypoint "/root/kanidm/platform/debian/interactive_entrypoint.sh" \ ubuntu:latest "$@"