7.1 KiB

Raw Permalink Blame History

helper functions

these changes focus on the installation of devuan with customisations suitable for docker images.

you can find these blend helper functions in ${BLENDPATH}/helpers-docker

additional or overridden helper functions to add or test functionality or workarounds specific to the blend to avoid changes to the sdk.

the associated configs are in the file config-docker in the blend directory.

Largely adapted from with comments lifted wholesale for this md.

build_docker_dist()

defines the sequence of calls that will result in a devuan install with docker image related optimisations.

docker-push()

tags and pushes to the docker registry specified in the config-docker

TODO: example config entries for gitlab_ci variables, dockerhub, or local registry registry_url=registry.example.com registry_user=username registry_token=password

Gitlab CI using CI/CD variables see: https://docs.gitlab.com/ee/user/packages/container_registry/

registry_url=$CI_REGISTRY # # # registry_user=$CI_REGISTRY_USER # # $CI_DEPLOY_USER # username registry_token=$CI_REGISTRY_PASSWORD # $CI_JOB_TOKEN # $CI_DEPLOY_PASSWORD # access_token registry_token=$CI_COMMIT_SHA

NB For CI/CD we use the .gitlab-ci.yml for the build and push, see: the url above $CI_REGISTRY_IMAGE would be resolved to the address of the registry tied to this project. CI_COMMIT_REF_SLUG is like $CI_COMMIT_REF_NAME but avoids the use of forward slashes by using only 0-9 a-z and replacing everything else with '-' or use $CI_COMMIT_SHA instead.

giving:

docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG -f ./Dockerfile
docker push     $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG

docker-import()

uses tar, with a separate tar-excludes file, to compress the debootstraped chroot and import it to docker.

docker-fixup()

removes redundent log and cache files, fixes some permissions.

docker-slimify()

using the slimIncludes and slimExcludes specified in the blends conf-docker it puts the associated paths into a dpkg configuration file so that they remain excluded for the image after build. It then removes the paths and restores the directory structure under /usr/share/man to avoid 'No such file or directory dpkg: error processing package' errors.

See: https://github.com/debuerreotype/debuerreotype/blob/master/scripts/debuerreotype-slimify

docker-purge_slimify_package()

remove the packages listed in config-docker slimify_packages list.

docker-init-policy()

to avoid errors like:

invoke-rc.d: policy-rc.d denied execution of restart.
invoke-rc.d: policy-rc.d denied execution of start.

Following: https://people.debian.org/~hmh/invokerc.d-policyrc.d-specification.txt

Most people using chroot jails just need an one-line
script which returns an exit status of 101 (101 - action forbidden by policy)
as the jailed /usr/sbin/policy-rc.d script.

The /usr/sbin/policy-rc.d file *must* be managed through the alternatives
system (/usr/sbin/update-alternatives) by any packages providing it.

So that's what we do: See: https://gitlab.com/paddy-hack/devuan/-/blob/master/scripts/tweak-init-policy

docker-apt-speedup() --> sysconf-docker

configure dpkg for 'force-unsafe-io'

For most Docker users, package installs happen during "docker build", which doesn't survive power loss and gets restarted clean afterwards anyhow, so this minor tweak gives us a nice speedup (much nicer on spinning disks, obviously).

docker-apt-autoremove-suggests() --> sysconf-docker

Since Docker users are looking for the smallest possible final images, the following emerges as a very common pattern:

RUN apt-get update \
 && apt-get install -y <packages> \
 && <do some compilation work> \
 && apt-get purge -y --auto-remove <packages>

By default, APT will actually keep packages installed via Recommends or Depends if another package Suggests them, even and including if the package that originally caused them to be installed is removed. Setting this to "false" ensures that APT is appropriately aggressive about removing the packages it added.

docker-apt-clean() --> sysconf-docker

adds hooks to dpkg to keep /var/cache/apt/archives empty

Since for most Docker users, package installs happen in "docker build" steps, they essentially become individual layers due to the way Docker handles layering, especially using CoW filesystems. What this means for us is that the caches that APT keeps end up just wasting space in those layers, making our layers unnecessarily large (especially since we'll normally never use these caches again and will instead just "docker build" again and make a brand new image).

Ideally, these would just be invoking "apt-get clean", but in our testing, that ended up being cyclic and we got stuck on APT's lock, so we get this fun creation that's essentially just "apt-get clean".

DPkg::Post-Invoke { $aptGetClean };
APT::Update::Post-Invoke { $aptGetClean };
Dir::Cache::pkgcache "";
Dir::Cache::srcpkgcache "";

Note that we do realize this isn't the ideal way to do this, and are always open to better suggestions https://github.com/debuerreotype/debuerreotype/issues.

docker-apt-gzip-indexes() --> sysconf-docker

Since Docker users using "RUN apt-get update && apt-get install -y ..." in their Dockerfiles don't go delete the lists files afterwards, we want them to be as small as possible on-disk, so we explicitly request that Apt keep them compressed on-disk too instead of decompressing them.

For comparison, an "apt-get update" layer without this on a pristine "debian:wheezy" base image was "29.88 MB", where with this it was only "8.273 MB".

docker-apt-no-languages() --> sysconf-docker

In Docker, we don't often need the "Translations" files, so we're just wasting time and space by downloading them, and this inhibits that. For users that do need them, it's a simple matter to delete the file created here and "apt-get update". :)

remove-apt-comments() --> sysconf-docker

needs apt_version to be set via apt-version()

older versions of apt do not support comments in the config files.

dpkg-version()

apt-version()

docker-hostname() --> sysconf-docker

sets the hostname

a container’s hostname defaults to be the container’s ID in Docker unless overriden at build. So this is less important here.

docker-resolv() --> sysconf-docker

replaces the /etc/resolv.conf using 1.1.1.1 and 1.0.0.1

docker-id()

Needs epoch to be set in config-docker

If inside a virtual machine or container environment the /etc/machine-id is usually taken from the container UUID

currently commented out.

unshare-debootstrap()

avoids bugs in packages that might leave things mounted ("/run/shm" is a common one that sometimes ends up dangling)

Not yet in the helpers but may be useful, I'm not sure we mount run in the chroot.

unshare-debootstrap() {
  # avoid bugs in packages that might leave things mounted
  # ("/run/shm" is a common one that sometimes ends up dangling)
  unshare --mount debootstrap "$@"
}

7.1 KiB Raw Permalink Blame History Unescape Escape