209 lines
7.1 KiB
Markdown
209 lines
7.1 KiB
Markdown
# helper functions
|
||
|
||
these changes focus on the installation of devuan with customisations suitable for docker images.
|
||
|
||
you can find these blend helper functions in `${BLENDPATH}/helpers-docker`
|
||
|
||
additional or overridden helper functions to add or test functionality or workarounds
|
||
specific to the blend to avoid changes to the sdk.
|
||
|
||
the associated configs are in the file config-docker in the blend directory.
|
||
|
||
Largely adapted from with comments lifted wholesale for this md.
|
||
|
||
- <https://github.com/debuerreotype/debuerreotype/blob/master/scripts/>
|
||
- <https://gitlab.com/paddy-hack/devuan>
|
||
|
||
## build_docker_dist()
|
||
|
||
defines the sequence of calls that will result in a devuan install with docker image related optimisations.
|
||
|
||
## docker-push()
|
||
|
||
tags and pushes to the docker registry specified in the config-docker
|
||
|
||
TODO: example config entries for gitlab_ci variables, dockerhub, or local registry
|
||
registry_url=registry.example.com
|
||
registry_user=username
|
||
registry_token=password
|
||
|
||
- Gitlab CI using CI/CD variables see: <https://docs.gitlab.com/ee/user/packages/container_registry/>
|
||
|
||
registry_url=$CI_REGISTRY # # #
|
||
registry_user=$CI_REGISTRY_USER # # $CI_DEPLOY_USER # username
|
||
registry_token=$CI_REGISTRY_PASSWORD # $CI_JOB_TOKEN # $CI_DEPLOY_PASSWORD # access_token
|
||
registry_token=$CI_COMMIT_SHA
|
||
|
||
NB For CI/CD we use the .gitlab-ci.yml for the build and push, see: the url above
|
||
$CI_REGISTRY_IMAGE would be resolved to the address of the registry tied to this project.
|
||
CI_COMMIT_REF_SLUG is like $CI_COMMIT_REF_NAME but avoids the use of forward slashes
|
||
by using only 0-9 a-z and replacing everything else with '-' or use $CI_COMMIT_SHA instead.
|
||
|
||
giving:
|
||
|
||
```bash
|
||
docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
|
||
docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG -f ./Dockerfile
|
||
docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
|
||
```
|
||
|
||
## docker-import()
|
||
|
||
uses tar, with a separate tar-excludes file, to compress the debootstraped
|
||
chroot and import it to docker.
|
||
|
||
## docker-fixup()
|
||
|
||
removes redundent log and cache files, fixes some permissions.
|
||
|
||
## docker-slimify()
|
||
|
||
using the slimIncludes and slimExcludes specified in the blends conf-docker it
|
||
puts the associated paths into a dpkg configuration file so that they remain
|
||
excluded for the image after build. It then removes the paths and restores the
|
||
directory structure under /usr/share/man to avoid 'No such file or directory
|
||
dpkg: error processing package' errors.
|
||
|
||
See: <https://github.com/debuerreotype/debuerreotype/blob/master/scripts/debuerreotype-slimify>
|
||
|
||
## docker-purge_slimify_package()
|
||
|
||
remove the packages listed in config-docker slimify_packages list.
|
||
|
||
## docker-init-policy()
|
||
|
||
to avoid errors like:
|
||
|
||
```bash
|
||
invoke-rc.d: policy-rc.d denied execution of restart.
|
||
invoke-rc.d: policy-rc.d denied execution of start.
|
||
```
|
||
|
||
Following: <https://people.debian.org/~hmh/invokerc.d-policyrc.d-specification.txt>
|
||
|
||
```text
|
||
Most people using chroot jails just need an one-line
|
||
script which returns an exit status of 101 (101 - action forbidden by policy)
|
||
as the jailed /usr/sbin/policy-rc.d script.
|
||
|
||
The /usr/sbin/policy-rc.d file *must* be managed through the alternatives
|
||
system (/usr/sbin/update-alternatives) by any packages providing it.
|
||
```
|
||
|
||
So that's what we do:
|
||
See: <https://gitlab.com/paddy-hack/devuan/-/blob/master/scripts/tweak-init-policy>
|
||
|
||
## docker-apt-speedup() --> sysconf-docker
|
||
|
||
configure dpkg for 'force-unsafe-io'
|
||
|
||
For most Docker users, package installs happen during "docker build", which
|
||
doesn't survive power loss and gets restarted clean afterwards anyhow, so
|
||
this minor tweak gives us a nice speedup (much nicer on spinning disks, obviously).
|
||
|
||
## docker-apt-autoremove-suggests() --> sysconf-docker
|
||
|
||
Since Docker users are looking for the smallest possible final images, the
|
||
following emerges as a very common pattern:
|
||
|
||
```bash
|
||
RUN apt-get update \
|
||
&& apt-get install -y <packages> \
|
||
&& <do some compilation work> \
|
||
&& apt-get purge -y --auto-remove <packages>
|
||
```
|
||
|
||
By default, APT will actually _keep_ packages installed via Recommends or
|
||
Depends if another package Suggests them, even and including if the package
|
||
that originally caused them to be installed is removed. Setting this to
|
||
"false" ensures that APT is appropriately aggressive about removing the
|
||
packages it added.
|
||
|
||
## docker-apt-clean() --> sysconf-docker
|
||
|
||
adds hooks to dpkg to keep /var/cache/apt/archives empty
|
||
|
||
Since for most Docker users, package installs happen in "docker build" steps,
|
||
they essentially become individual layers due to the way Docker handles
|
||
layering, especially using CoW filesystems. What this means for us is that
|
||
the caches that APT keeps end up just wasting space in those layers, making
|
||
our layers unnecessarily large (especially since we'll normally never use
|
||
these caches again and will instead just "docker build" again and make a brand
|
||
new image).
|
||
|
||
Ideally, these would just be invoking "apt-get clean", but in our testing,
|
||
that ended up being cyclic and we got stuck on APT's lock, so we get this fun
|
||
creation that's essentially just "apt-get clean".
|
||
|
||
```bash
|
||
DPkg::Post-Invoke { $aptGetClean };
|
||
APT::Update::Post-Invoke { $aptGetClean };
|
||
Dir::Cache::pkgcache "";
|
||
Dir::Cache::srcpkgcache "";
|
||
```
|
||
|
||
Note that we do realize this isn't the ideal way to do this, and are always
|
||
open to better suggestions <https://github.com/debuerreotype/debuerreotype/issues>.
|
||
|
||
## docker-apt-gzip-indexes() --> sysconf-docker
|
||
|
||
Since Docker users using "RUN apt-get update && apt-get install -y ..." in
|
||
their Dockerfiles don't go delete the lists files afterwards, we want them to
|
||
be as small as possible on-disk, so we explicitly request that Apt keep them
|
||
compressed on-disk too instead of decompressing them.
|
||
|
||
For comparison, an "apt-get update" layer without this on a pristine
|
||
"debian:wheezy" base image was "29.88 MB", where with this it was only "8.273 MB".
|
||
|
||
## docker-apt-no-languages() --> sysconf-docker
|
||
|
||
In Docker, we don't often need the "Translations" files, so we're just wasting
|
||
time and space by downloading them, and this inhibits that. For users that do
|
||
need them, it's a simple matter to delete the file created here and
|
||
"apt-get update". :)
|
||
|
||
## remove-apt-comments() --> sysconf-docker
|
||
|
||
needs apt_version to be set via apt-version()
|
||
|
||
older versions of apt do not support comments in the config files.
|
||
|
||
## dpkg-version()
|
||
|
||
## apt-version()
|
||
|
||
## docker-hostname() --> sysconf-docker
|
||
|
||
sets the hostname
|
||
|
||
a container’s hostname defaults to be the container’s ID in Docker
|
||
unless overriden at build. So this is less important here.
|
||
|
||
## docker-resolv() --> sysconf-docker
|
||
|
||
replaces the /etc/resolv.conf using 1.1.1.1 and 1.0.0.1
|
||
|
||
## docker-id()
|
||
|
||
Needs epoch to be set in config-docker
|
||
|
||
If inside a virtual machine or container environment the
|
||
/etc/machine-id is usually taken from the container UUID
|
||
|
||
currently commented out.
|
||
|
||
## unshare-debootstrap()
|
||
|
||
avoids bugs in packages that might leave things mounted
|
||
("/run/shm" is a common one that sometimes ends up dangling)
|
||
|
||
Not yet in the helpers but may be useful, I'm not sure we mount run in the chroot.
|
||
|
||
```bash
|
||
unshare-debootstrap() {
|
||
# avoid bugs in packages that might leave things mounted
|
||
# ("/run/shm" is a common one that sometimes ends up dangling)
|
||
unshare --mount debootstrap "$@"
|
||
}
|
||
```
|