devuan-sdk
/
os-build-system
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

review corrections as recommended by ARD and TH partners in DECODE

This commit is contained in:
Jaromil 2018-10-29 13:46:40 +01:00
parent 34fed726dd
commit 94ed3a9050
5 changed files with 232 additions and 218 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
doc/introduction.md
View File

@ -34,12 +34,12 @@ instructions.
Due to the experimental stage of development of other components in Due to the experimental stage of development of other components in
DECODE and according to the LEAN principles declared in the project, DECODE and according to the LEAN principles declared in the project,
this stable release doesn't only constitutes a final point of arrival this stable release doesn't only constitute a final point of arrival
for this development task. What DECODE OS can do today is facilitate for this development task. What DECODE OS can do today is facilitating
the deployement of lab-tested software applications (for example made the deployment of lab-tested software applications (for example made
in a Docker format, widely adopted by other partners in DECODE) and in a Docker format, widely adopted by other partners in DECODE) and
render these prototypes into a production ready format that can be render these prototypes into a production ready format that can be
deployed on the DECODE BOX open-hardware as well on virtual-machines. deployed on the open-hardware DECODE BOX as well on virtual-machines.
We consider this achievement highly beneficial for a project whose We consider this achievement highly beneficial for a project whose
development is still in-flux, as well for the free and open source development is still in-flux, as well for the free and open source
@ -47,8 +47,9 @@ community out there, since the access to the powerful features of the
SDK is now made very easy via an integrated continuous pipeline. SDK is now made very easy via an integrated continuous pipeline.
In light of these advantages, there is a clear intention within our In light of these advantages, there is a clear intention within our
organisation (DYNE) to keep maintaining DECODE OS also beyond the span organisation (mainly by DYNE) to keep maintaining DECODE OS also
of the project and this very task now concluded, since it greatly beyond the span of the project and this very task now concluded, since
helps the manning of prototypes into stable production environments. it greatly helps the manning of prototypes into stable production
environments.

174
doc/sdk-01-overview.md
View File

@ -21,20 +21,21 @@ of the base Devuan system.
libdevuansdk libdevuansdk
------------ ------------
libdevuansdk is the core of any part of the Devuan SDK. It holds the _libdevuansdk_ is the core of any part of the Devuan SDK. It holds the
common knowledge between all of the upper wrappers such as live-sdk, common knowledge between all of the upper wrappers such as _live-sdk_,
vm-sdk, and arm-sdk. Simply put, it is a shell script library to unify _vm-sdk_, and _arm-sdk_. Simply put, it is a shell script library to
the use and creation of various functions spread throughout the complete unify the use and creation of various functions spread throughout the
Devuan SDK. complete Devuan SDK.
The wrappers are designed to be used interactively from a terminal, as The wrappers are designed to be used interactively from a terminal, as
well as automated from shell scripts. libdevuansdk uses an additional well as automated from shell scripts. _libdevuansdk_ uses an
zsh library called [zuper](https://github.com/dyne/zuper) to ease the additional _zsh_ library called [zuper](https://github.com/dyne/zuper)
variable declaration and scoping, as well as error checking and to ease the variable declaration and scoping, as well as error
debugging. However, zuper is not included in libdevuansdk itself - one checking and debugging. However, _zuper_ is not included in
is required to include it in its respective wrapper. live-sdk, vm-sdk, _libdevuansdk_ itself - one is required to include it in its
and arm-sdk can be taken as example. libdevuansdk itself has some respective wrapper. _live-sdk_, _vm-sdk_, and _arm-sdk_ can be taken
software dependencies though: as example. libdevuansdk itself has some software dependencies that
should be installed prior to use:
``` ```
zsh zsh
@ -48,48 +49,49 @@ xz-utils
### Workflow ### Workflow
Working with libdevuansdk splits into categories of what you want to do. Working with _libdevuansdk_ splits into categories of what you want to
_zlibs_ are files separated into these categories: do. _zlibs_ are files separated into the following categories:
* ***bootstrap*** Contains the functions for the bootstrap process. * ***bootstrap*** Contains the functions for the bootstrap process.
Creating a minimal debootstrap base, and making it into a tarball for Creating a minimal debootstrap base system, and making it into a
later use so one does not have to wait for the lengthy bootstrap compressed file (tar.gz) for later use so one does not have to wait
process on each consequent build. for the lengthy bootstrap process on each consequent build.
* ***helpers*** Contains the helper functions for libdevuansdk that make * ***helpers*** Contains the helper functions for _libdevuansdk_ that
the workflow a bit easier to use and handle. make the workflow a bit easier to use and handle.
* ***imaging*** Contains the functions necessary for creating raw * ***imaging*** Contains the functions necessary for creating raw
dd-able images. dd-able images.
* ***rsync*** Contains rsync and copying functions. * ***rsync*** Contains rsync and file copying functions.
* ***sysconf*** Contains the default system configuration. * ***sysconf*** Contains the default system configuration.
### Usage ### Usage
As libdevuansdk is not very helpful when being used on its own, its As libdevuansdk is not very useful when invoked on its own, its usage
usage will be explained at later parts, for each specific wrapper. The will be explained at later parts, for each specific wrapper. The
Technical documentation of libdevuansdk will follow in its appropriate technical documentation of _libdevuansdk_ will follow in its
section. appropriate section.
The wrappers The wrappers
------------ ------------
As mentioned, libdevuansdk is the core library we wrap around. The As mentioned, _libdevuansdk_ is the core library we wrap around. The
currently existing wrappers are called _live-sdk_, _vm-sdk_, and currently existing wrappers are called _live-sdk_, _vm-sdk_, and
_arm-sdk_. These facilitate the builds of liveCDs, virtual machines, and _arm-sdk_. These facilitate the builds of liveCDs, virtual machines, and
images for embedded ARM devices, respectively. Each of them have their images for embedded ARM devices, respectively. Each of them have their
own section in this paper. own section in this paper.
Since all of these wrappers, along with libdevuansdk, hold a vanilla Since all of these wrappers, along with _libdevuansdk_, hold a
Devuan configuration, you might prefer not to change their code. Due to _vanilla_ Devuan configuration, it is best to keep their code
this, a concept called *blends* was introduced. Blends are a simple way untouched. To allow for custom configurations, we introduced a concept
to customize the base image before building it, allowing you to very called *blends*. Blends are a simple way to customize the base image
easily add packages, kernels, and virtually anything one might want to of the OS-to-be before building it, allowing to easily add packages,
do in the image. This exactly is the case with DECODE OS. kernels, and virtually anything one might want to do in the
image. This exactly is the case with DECODE OS.
arm-sdk arm-sdk
@ -103,25 +105,26 @@ according kernels and bootloaders.
### Directory structure ### Directory structure
arm-sdk's directory structure is separated into places where we hold our _arm-sdk_'s directory structure is separated into places where we hold
boards and their kernel configurations, device-specific directories with our boards and their kernel configurations, device-specific
firmware and/or configuration, and a lib directory (where we keep directories with firmware and/or configuration, and a _lib_ directory
libdevuansdk and the like). (where we keep _libdevuansdk_ and the like).
### Obtaining arm-sdk ### Obtaining arm-sdk
The SDK, like any other, should be obtained via git. The repositories The SDK, like any other part of Devuan's software toolchain, should be
are hosted on Devuan's Gitlab. To grab it, we simply issue a _git clone_ obtained via _git_. The repositories are hosted on Devuan's Gitlab. To
command, an since it contains git submodules - we append _--recursive_ grab it, we simply issue a _git clone_ command on a terminal, and
to it: since it contains linked git submodules - we append _--recursive_ to
it:
``` ```
$ git clone https://git.devuan.org/sdk/arm-sdk --recursive $ git clone https://git.devuan.org/sdk/arm-sdk --recursive
``` ```
Consult the README.md file found in this repository to see what are the Consult the _README.md_ file found in this repository to see what are
required dependencies to use arm-sdk. the required dependencies to use _arm-sdk_.
### Using arm-sdk ### Using arm-sdk
@ -130,13 +133,13 @@ Once the build system is obtained, it can now be used interactively. The
process is very simple, and to build an image one can actually use a process is very simple, and to build an image one can actually use a
single shell command. However, we shall first show how it works. single shell command. However, we shall first show how it works.
In arm-sdk, every board has its own script located in the _boards_ In _arm-sdk_, every board has its own script located in the _boards_
directory. In most cases, these scripts contain functions to build the directory. In most cases, these scripts contain functions to build the
Linux kernel, and a bootloader needed for the board to boot. This is the Linux kernel, and a bootloader needed for the board to boot. This is
only difference between all the boards, which requires every board to the only difference between all the boards, which requires every board
have their own script. We are able to reuse the rootfs that is to have their own script. We are able to reuse the _rootfs_ that was
bootstrapped before. For our example, let's take the _Nokia N900_ build bootstrapped before. For our example, let's take the _Nokia N900_
script. To build a vanilla image for it, we simply issue: build script. To build a _vanilla_ image for it, we simply issue:
``` ```
@ -152,10 +155,10 @@ The oneliner above is self-explanatory: We first start a new untainted
shell, source the sdk file to get an interactive SDK shell, then we shell, source the sdk file to get an interactive SDK shell, then we
initialize the operating system along with the board we are building, initialize the operating system along with the board we are building,
and finally we issue a helper command that calls all the necessary and finally we issue a helper command that calls all the necessary
functions to build our image. The _load_ command takes an optional third functions to build our image. The _load_ command takes an optional
argument which is the name of our blend (the way to customize our third argument which is the name of our _blend_ (the way to customize
vanilla image) which will be explained later. So in this case, our our _vanilla_ image) which will be explained later. So in this case,
oneliner would look like: our oneliner would look like:
``` ```
$ zsh -f -c 'source sdk && load devuan n900 decode && build_image_dist' $ zsh -f -c 'source sdk && load devuan n900 decode && build_image_dist'
@ -163,7 +166,7 @@ $ zsh -f -c 'source sdk && load devuan n900 decode && build_image_dist'
This would create an image with the _"decode"_ blend, which is available This would create an image with the _"decode"_ blend, which is available
by cloning the DECODE OS git repository. The *build_image_dist* command by cloning the DECODE OS git repository. The *build_image_dist* command
is a helper function located in libdevuansdk that wraps around the 8 is a helper function located in _libdevuansdk_ that wraps around the 8
functions needed to build our image. They are all explained in the functions needed to build our image. They are all explained in the
technical part of this paper. technical part of this paper.
@ -178,14 +181,14 @@ _arm-sdk_.
### Directory structure ### Directory structure
Unlike arm-sdk, in live-sdk we have no need for specific boards or Unlike _arm-sdk_, in _live-sdk_ we have no need for specific boards or
setups, so in this case we only host the interactive shell init, and setups, so in this case we only host the interactive shell init, and
libraries. libraries.
### Obtaining live-sdk ### Obtaining live-sdk
The SDK, like any other, should be obtained via git. The repositories The SDK, like any other, should be obtained via _git_. The repositories
are hosted on Devuan's Gitlab. To grab it, we simply issue a _git clone_ are hosted on Devuan's Gitlab. To grab it, we simply issue a _git clone_
command, an since it contains git submodules - we append _--recursive_ command, an since it contains git submodules - we append _--recursive_
to it: to it:
@ -194,18 +197,18 @@ to it:
$ git clone https://git.devuan.org/sdk/live-sdk --recursive $ git clone https://git.devuan.org/sdk/live-sdk --recursive
``` ```
Consult the README.md file found in this repository to see what are the Consult the _README.md_ file found in this repository to see what are
required dependencies to use live-sdk. the required dependencies to use _live-sdk_.
### Using live-sdk ### Using live-sdk
Much like _arm-sdk_, the _live-sdk_ is used the same way. With two Much like _arm-sdk_, the _live-sdk_ is used the same way. With two
specific differences. Since we don't have any need for specific boards, specific differences. Since we don't have any need for specific
with loading we don't specify a board, but rather the CPU architecture boards, when loading we don't specify a board, but rather the CPU
we are building for. Currently supported are *i386* and *amd64* which architecture we are building for. Currently supported are *i386* and
represent 32bit and 64bit respectively. To build a vanilla live ISO, we *amd64* which represent 32bit and 64bit respectively. To build a
issue: _vanilla_ live ISO, we issue:
``` ```
$ zsh -f -c 'source sdk && load devuan amd64 && build_iso_dist' $ zsh -f -c 'source sdk && load devuan amd64 && build_iso_dist'
@ -214,17 +217,18 @@ $ zsh -f -c 'source sdk && load devuan amd64 && build_iso_dist'
This will start the build process, and after a certain amount of time we This will start the build process, and after a certain amount of time we
will have our ISO ready and inside the _dist_ directory. will have our ISO ready and inside the _dist_ directory.
Just like in arm-sdk, we can use a _blend_ and customize our OS: Just like in _arm-sdk_, we can use a _blend_ and customize our OS:
``` ```
$ zsh -f -c 'source sdk && load devuan amd64 decode && build_iso_dist' $ zsh -f -c 'source sdk && load devuan amd64 decode && build_iso_dist'
``` ```
So this would create a live ISO of DECODE OS. Again as noted, this can So this would create a live ISO of DECODE OS. Again as noted, this can
be obtained by recursively cloning the decode-os git repository. be obtained by recursively cloning the corresponding (DECODE-OS) git
repository.
The *build_iso_dist* command is a helper function located in The *build_iso_dist* command is a helper function located in
libdevuansdk that wraps around the 9 functions needed to build our _libdevuansdk_ that wraps around the 9 functions needed to build our
image. They are all explained in the technical part of this manual. image. They are all explained in the technical part of this manual.
@ -232,31 +236,32 @@ vm-sdk
------ ------
The _vm-sdk_ is used to build VirtualBox/Vagrant boxes, and virtual The _vm-sdk_ is used to build VirtualBox/Vagrant boxes, and virtual
images for emulation, in QCOW2 format, which is a nifty byproduct of images for emulation, in QCOW2 format, which is the byproduct of
building a Vagrant box. Its structure is very similar to _live-sdk_ and building a Vagrant box. Its structure is very similar to _live-sdk_
is the smallest of the three wrappers currently found in the Devuan SDK. and is the smallest of the three wrappers currently found in the
Devuan SDK.
### Directory structure ### Directory structure
Like with live-sdk, in vm-sdk we have no need for specific boards or Like with _live-sdk_, in _vm-sdk_ we have no need for specific boards
setups, so in this case we only host the interactive shell init, and or setups, so in this case we only host the interactive shell init,
libraries. and libraries.
### Obtaining vm-sdk ### Obtaining vm-sdk
The SDK, like any other, should be obtained via git. The repositories The SDK, like any other, should be obtained via _git_. The
are hosted on Devuan's Gitlab. To grab it, we simply issue a _git clone_ repositories are hosted on Devuan's Gitlab. To grab it, we simply
command, an since it contains git submodules - we append _--recursive_ issue a _git clone_ command, an since it contains git submodules - we
to it: append _--recursive_ to it:
``` ```
$ git clone https://git.devuan.org/sdk/vm-sdk --recursive $ git clone https://git.devuan.org/sdk/vm-sdk --recursive
``` ```
Consult the README.md file found in this repository to see what are the Consult the _README.md_ file found in this repository to see what are
required dependencies to use vm-sdk. the required dependencies to use _vm-sdk_.
### Using vm-sdk ### Using vm-sdk
@ -264,24 +269,25 @@ required dependencies to use vm-sdk.
Once obtained, we can use it interactively. The process is very simple, Once obtained, we can use it interactively. The process is very simple,
and to build an image we use the oneliner we've already seen above. and to build an image we use the oneliner we've already seen above.
Also like with live-sdk, we don't need specific boards, however we also Also like with _live-sdk_, we don't build for specific boards, however
do not create any non-amd64 images, so we don't have to pass an we also do not create any non-amd64 images, so we don't have to pass
architecture to the load command either. To build a vanilla Vagrant Box, an architecture to the load command either. To build a _vanilla_
VirtualBox image, qcow2 image, and a cloud-based qcow2 image, we issue: Vagrant Box, VirtualBox image, QCOW2 image, and a cloud-based QCOW2
image, we issue:
``` ```
$ zsh -f -c 'source sdk && load devuan && build_vagrant_dist' $ zsh -f -c 'source sdk && load devuan && build_vagrant_dist'
``` ```
This line would create al the four types of the VM image. This line would create all the four types of the VM image.
As shown with the previous two, the _blend_ concept works as advertised As shown with the previous two wrappers, the _blend_ concept works as
here as well: advertised here as well:
``` ```
$ zsh -f -c 'source sdk && load deuvan decode && build_vagrant_dist' $ zsh -f -c 'source sdk && load deuvan decode && build_vagrant_dist'
``` ```
The *build_vagrant_dist* command is a helper function located in The *build_vagrant_dist* command is a helper function located in
libdevuansdk that wraps around the 11 functions needed to build our _libdevuansdk_ that wraps around the 11 functions needed to build our
image. They are all explained in the technical part of this manual. image. They are all explained in the technical part of this manual.

191
doc/sdk-02-blends.md
View File

@ -6,23 +6,24 @@ Introduction
------------ ------------
In the Devuan SDK, a _blend_ is the preferred way we use to make In the Devuan SDK, a _blend_ is the preferred way we use to make
customizations to the vanilla image. Using blends we can very easily customizations to the _vanilla_ image. Using blends we can very easily
create different flavors of our image, by easily including/excluding create different flavors of our image, by easily including/excluding
certain software packages, files, or anything we wish to do as a matter certain software packages, files, or anything we wish to do. Blends
of fact. Blends can become a very quick way of creating entire new can become a very quick way of creating entire new derivatives of the
derivatives of the vanilla distribution we are building. original _vanilla_ distribution we are building.
This time, we will take the DECODE OS as a blend example. In DECODE OS This time, we will take the DECODE OS as a _blend_ example. In DECODE
we provide a blend called _decode_ which is the blend we use to create OS we provide a blend called _decode_ which is the blend we use to
a production release of DECODE OS. The blend's files are contained create a production release of DECODE OS. The blend's files are
within their own directory in the decode-os git repository. contained within their own directory in the _decode-os_ git
repository.
Configuration Configuration
------------- -------------
Any SDK requires a single file to act as a blend. This file is also a Any SDK requires a single file to act as a _blend_. This file is also a
zsh script, and, at the very least, it must contain two functions _zsh_ script, and, at the very least, it must contain two functions
called: called:
``` ```
@ -31,13 +32,13 @@ blend_postinst()
``` ```
These functions are your pathway to expanding your blend into whatever These functions are your pathway to expanding your blend into whatever
you would like to do. The _preinst_ function is usually called right you would like it to do. The _preinst_ function is usually called
after bootstrapping the vanilla root filesystem, and the _postinst_ right after bootstrapping the _vanilla_ root filesystem, and the
function is called near the very end, just before packing or compressing _postinst_ function is called near the very end, just before packing
the image. These two strategic places should be enough to do changes or compressing the image. These two strategic places should be enough
within the image. If this is not enough, blends also allow you to simply to do changes within the image. If this is not enough, blends also
**override any variable or function** contained within libdevuansdk or allow you to simply **override any variable or function** contained
the sdk you are using. within _libdevuansdk_ or the sdk you are using.
Our _decode_ blend is such an example. It is a somewhat expanded blend, Our _decode_ blend is such an example. It is a somewhat expanded blend,
not contained within a single file, but rather a directory. This allows not contained within a single file, but rather a directory. This allows
@ -46,19 +47,19 @@ easier maintenance and makes the scripts clearer and cleaner.
### Adding and removing packages ### Adding and removing packages
When we want to add or remove specific packages to our build, we have to When we want to add or remove specific packages to our build, we have
override or append to libdevuansdk's arrays. The array for packages we to override or append to _libdevuansdk_'s arrays. The array for
want installed is called *extra_packages*, and the array for packages we packages we want installed is called *extra_packages*, and the array
want purged is called *purge_packages*. In the decode blend, these can for packages we want purged is called *purge_packages*. In the Decode
be found in the _config_ file located inside the decode-os blend blend, these can be found in the _config_ file located inside the
directory. Keep in mind that these arrays could already contain _decode-os_ blend directory. Keep in mind that these arrays could
specific packages, so you are advised to rather append to them, than already contain specific packages, so you are advised to rather append
overriding them. to them, than overriding them.
If the packages you want to install are not available in the repos, you If the packages you want to install are not available in the
still have a way of automatically installing them. All you need to do to repositories, you still have a way of automatically installing
take care of it is at some point in your blend - copy your .deb files to them. All you have to do is copy your corresponding .deb files to the
the following directory: following directory of the blend:
``` ```
$R/extra/custom-packages/ $R/extra/custom-packages/
@ -70,10 +71,11 @@ And when that is done, just call the function *install-custdebs*
Creating a blend Creating a blend
---------------- ----------------
Rather than explaining theory, you are best off viewing the blend files Rather than explaining the following in theory, you are best off
that are provided with _decode-os_. It is a fairly simple blend and viewing the blend files that are provided with _decode-os_. It is a
should give you enough insight on creating your own blend. Here are some fairly simple blend and should give you enough insight on how to
important guidelines for creating a blend: create your own blend. Here are some important guidelines for creating
a blend:
* The blend should always contain at least two functions * The blend should always contain at least two functions
@ -96,19 +98,20 @@ variables as well.
* Any arguments used after the blend name when loading from the SDK are * Any arguments used after the blend name when loading from the SDK are
free for you to use in the blend. free for you to use in the blend.
This means you can use anything **after $4** inside your blend if you This means you can use anything after the fourth argument (**$4** in
require passing arguments to it. _zsh_) inside your blend if you require passing arguments to it.
These are some of the more important guidelines. There is plenty more These are some of the more important guidelines. There is plenty more
tricks and quirks, but it's easy to find out once you read a blend or tricks and quirks, but it's easy to find out how to tweak the
two on your own... configuration files and the blend in general once you read through a
blend or two on your own.
### Enable the blend ### Enable the blend
To use your blend in the first place, you need to make the sdk know To use your blend in the first place, you need to make the SDK know
about it. To make this work, you need to append the path to your new about it. Thus you should append the path to your new blend inside
blend inside the **blend_map** of the _sdk_ file: the **blend_map** of the _sdk_ file:
``` ```
blend_map=( blend_map=(
@ -121,21 +124,21 @@ blend_map=(
As you can see, the map is a key-value storage. So you can have an alias As you can see, the map is a key-value storage. So you can have an alias
(name) for your blend, and just use that to point to the path of the (name) for your blend, and just use that to point to the path of the
blend. The blend file will be sourced by the sdk once it is told to do blend. The blend file will be sourced by the SDK once it is told to do
so. so.
### A configuration file ### A configuration file
For having a finer-grained control of what goes into our build, we can For having a finer-grained control of what goes into our build, we can
create a config file for our blend. From here we can easily control any create a config file for our blend. From here we can easily control
configurable aspect of our blend, such as packages that go in or out, any configurable aspect of it, such as packages that go in or out, the
the blend name, and much more. **Make sure you source this file from blend name, and much more. **Make sure you source this file from your
your blend.** blend.**
Adding and removing packages was abstractly mentioned earlier: it goes Adding and removing packages was abstractly mentioned earlier: it goes
into two separate arrays holding package names. To add packages, we into two separate arrays holding package names. To add packages, we
append to the **extra_packages** array, which would look like this: append to the `extra_packages` array, which would look like this:
``` ```
extra_packages+=( extra_packages+=(
@ -146,12 +149,13 @@ extra_packages+=(
) )
``` ```
This would install these four packages, along with the ones predefined This would install the four packages `my_new_package`, `foo`, `bar`,
in either libdevuansdk or the sdk you are using. You may also want to and `baz` along with the ones predefined in either _libdevuansdk_ or
see which those are in case you wish to exclude them, but they are sane the SDK you are using. You may also want to see which those are in
and useful utilities which should be included in your build if possible. case you wish to exclude them, but they are sane and useful utilities
Overriding all those packages, you would need to reset the whole array, which should be included in your build if possible. Overriding all
so you would simply issue this: those packages, you would need to reset the whole array, so you would
simply issue this:
``` ```
extra_packages=( extra_packages=(
@ -162,11 +166,11 @@ extra_packages=(
) )
``` ```
As you can see, we no longer have the _+=_, but rather only _=_, which As you can see, we no longer have the `+=`, but rather only `=`, which
means we are not appending to the array, but rather redefining it. means we are not appending to the array, but rather redefining it.
All of the above applies as well for removing packages, but in this case All of the above applies as well for removing packages, but in this case
the array is called **purge_packages**. the array is called `purge_packages`.
#### Custom packages #### Custom packages
@ -183,23 +187,23 @@ custom_deb_packages=(
) )
``` ```
To trigger installation of these packages, you will need to copy them to To trigger the installation of these packages, you will need to copy
`$R/extra/custom_packages`, and then call the **install_custdebs** them to `$R/extra/custom_packages`, and then call the
function somewhere from your blend. `install_custdebs` function somewhere from your blend.
### Custom files ### Custom files
Any files you want to add to the system to override what's there by Any files you want to add to the system to override what's there by
default you can add using a *rootfs overlay*. Create a directory inside default you can add using a *rootfs overlay*. Create a directory
your blend directory called *rootfs-overlay* and simply put files inside inside your blend directory called *rootfs-overlay* and simply put
it. The directory structure is absolute to the image we are building. files inside it. The directory structure is absolute to the image we
For example what's in "rootfs-overlay/etc/" would end up in the "/etc" are building. For example what's in "rootfs-overlay/etc/" would end
of our final image. See _hier(7)_ from the Linux manpages for more up in the "/etc" of our final image. See _hier(7)_ in the Linux
explanation on this directory hierarchy. manpages for more explanation on this directory hierarchy.
If you end up with any files here, to actually copy them, you will need If you end up with any files here, to actually copy them, you will need
to `cp -f` it, or `rsync` it if you prefer. to either run `cp -f` it, or `rsync` the directory if you prefer.
### The .blend file ### The .blend file
@ -215,10 +219,11 @@ BLENDPATH="${BLENDPATH:-$(dirname $0)}"
source $BLENDPATH/config source $BLENDPATH/config
``` ```
The minimum blend should contain two functions: **blend_preinst** and The minimum blend should contain two functions: `blend_preinst` and
**blend_postinst**. These functions are called at specific points in the `blend_postinst`. These functions are called at specific points in the
build, where they give the most power: just after bootstrapping the build, where they give the most power: just after bootstrapping the
vanilla system, and just before packaging the final build, respectively. _vanilla_ system, and just before packaging the final build,
respectively.
#### blend_preinst #### blend_preinst
@ -239,20 +244,20 @@ blend_preinst() {
} }
``` ```
So as you can see, the preinst function will add a new user with the As you can see, the pre-install function will add a new user with the
credentials `user:pass`, it will copy our custom debs where they can be credentials `user:pass`, it will copy our custom debs where they can
used, and finally it will trigger their installation. be used, and finally it will trigger their installation.
The `fn, req, ckreq` part on the top of the function is a safety check The `fn, req, ckreq` part on the top of the function is a safety check
for the function that is enabled by zuper. It allows us to check if for the function that is enabled by _zuper_. It allows us to check if
variables are defined when the function is called and fail if it is variables are defined when the function is called and fail if it is
wrong. You should utilize this as much as possible. The `zerr` calls are wrong. You should utilize this as much as possible. The `zerr` calls
used to exit if the function fails. are used to exit if the function fails.
#### blend_postinst #### blend_postinst
A postinst function can look like the following: A post-install function can look like the following:
``` ```
blend_postinst() { blend_postinst() {
@ -270,33 +275,33 @@ blend_postinst() {
This function would copy the `rootfs-overlay` to the `strapdir` (which This function would copy the `rootfs-overlay` to the `strapdir` (which
holds our image's filesystem) and it would call the `blend_finalize` holds our image's filesystem) and it would call the `blend_finalize`
function. By default this function doesn't exist, but it's an example so function. By default this function doesn't exist, we quote it as an
you can see you can call your own functions as well. You can define them example for you to see how it is possible to call your own functions
within the blend file. as well. You can define them within the blend file.
Using a blend Using a blend
------------- -------------
As explained in previous chapters, you can use your blends through the As previously explained, you can use your blends through the SDK's
interactive SDK shell. In decode-os the blend is placed in the root of interactive shell. In _decode-os_ the blend is placed in the root of
the git repository, and the sdk wrappers are located within. Therefore the git repository, and the sdk wrappers are located within. Therefore
an sdk would have to source it with such a path: an SDK would have to source it with such a path:
``` ```
$R/../decode.blend $R/../decode.blend
``` ```
If you take a look at vm-sdk's *sdk* file, you can see it in the If you take a look at _vm-sdk_'s `sdk` file, you will see the
*blend_map*. Using a new blend requires you to add it to this map in `blend_map` array. Using a new blend requires you to add it to this
the same manner. The map is key-value formatted, and on the left you map in the same manner. The map is key-value formatted, and on the
have an alias of your blend, and on the right you have a script you have left you have an alias of your blend, and on the right you have a
to write. It can either be the blend itself or any helper file you might script you have to write. It can either be the blend itself or any
need to initialize your blend. helper file you might need to initialize your blend.
After you've added it to the blend map, you simply initialize the sdk, After you've added it to the blend map, you simply initialize the SDK,
and use the same *load* command we learned earlier, while appending the and use the same *load* command we learned earlier, while appending
blend alias and any optional argument. the blend alias and any optional argument.
``` ```
$ zsh -f $ zsh -f
@ -304,14 +309,14 @@ $ source sdk
$ load devuan decode <these> <arguments> <we> <can> <use> <in> <the> <blend> $ load devuan decode <these> <arguments> <we> <can> <use> <in> <the> <blend>
``` ```
And we've initialized our *decode* blend. It's always good to add a With this, we've initialized our *decode* blend. It's always good to add a
*notice()* call to your blend to signal it's been loaded successfully. *notice()* call to your blend to signal it's been loaded successfully.
After this is done, we simply build the image the same way we learned Once this is done, we simply build the image the same way we have
before: learned before:
``` ```
$ build_vagrant_dist $ build_vagrant_dist
``` ```
Consult the vm-sdk chapter for this. Consult the _vm-sdk_ chapter for this.

51
doc/sdk-03-technical.md
View File

@ -9,10 +9,10 @@ glues together.
Configuration Configuration
------------- -------------
Much of the libdevuansdk configuration is done in `libdevuansdk/config`. Much of the _libdevuansdk_ configuration is done in
Here you can edit the defaults if you wish to do something your needs `libdevuansdk/config`. Here you can edit the defaults if you wish to
are expressing. However, overriding these through upper levels is do something your needs are expressing. However, overriding these
recommended. through upper levels is recommended.
### `config` file ### `config` file
@ -54,28 +54,29 @@ Helper functions
---------------- ----------------
You can find useful helper functions in `libdevuansdk/zlibs/helpers`. You can find useful helper functions in `libdevuansdk/zlibs/helpers`.
They are intended to help when it comes to writing wrappers, as well as They are intended to help when it comes to writing wrappers, as well
making the developers' jobs easier for developing libdevuansdk. Some of as making the developers' jobs easier for developing
these functions are required for libdevuansdk to work properly as well. _libdevuansdk_. Some of these functions are required for
_libdevuansdk_ to work properly as well.
### `build_image_dist()` ### `build_image_dist()`
This function is a kind of a wrapper function. It's used in arm-sdk to This function is a kind of a wrapper function. It's used in _arm-sdk_
build a complete dd-able image from start to end. To run, it requires to build a complete dd-able image from start to end. To run, it
`$arch`, `$size`, `$parted_type`, `$workdir`, `$strapdir`, and requires `$arch`, `$size`, `$parted_type`, `$workdir`, `$strapdir`,
`$image_name` to be declared. See the part of "Creating wrappers" for and `$image_name` to be declared. See the section dedicated to
insight on these variables. "Creating wrappers" for insight on these variables.
The workflow of this function is bootstrapping a complete rootfs, The workflow of this function is bootstrapping a complete _rootfs_,
creating a raw image, installing/compiling a kernel, rsyncing everything creating a raw image, installing/compiling a kernel, rsyncing
to the raw image, and finally compressing the raw image. everything to the raw image, and finally compressing the raw image.
This same workflow is applied in the next two functions in this file, This same workflow is applied in the next two functions in this file,
which are `build_iso_dist` and `build_vagrant_dist`. To get a better which are `build_iso_dist` and `build_vagrant_dist`. To get a better
understanding of libdevuansdk, it's recommended to go through one of understanding of _libdevuansdk_, it's recommended to go through one of
these functions and following it deeper to find and figure out the other these functions and following it deeper to find and figure out the
functions and how they work together. other functions and how they work together.
### `devprocsys()` ### `devprocsys()`
@ -96,18 +97,18 @@ access to hardware or the system's resources, i.e. cryptography.
### `dpkgdivert()` ### `dpkgdivert()`
This function, like `devprocsys` takes two arguments and will create or This function, like `devprocsys` takes two arguments and will create
remove a dpkg diversion in the place you tell it to and remove or remove a dpkg diversion in the place you tell it to and remove
`invoke-rc.d` so that apt does not autostart daemons when they are `invoke-rc.d` so that _apt_ does not autostart daemons when they are
installed. installed.
### `chroot-script()` ### `chroot-script()`
This very useful functions allows you to chroot into `$strapdir` and This very useful functions allows you to _chroot_ into `$strapdir` and
execute the script/binary that's passed as a parameter to this function. execute the script/binary that's passed as a parameter to this
It also takes an optional argument `-d` that will call dpkgdivert on and function. It also takes an optional argument `-d` that will call
off before and after execution. `dpkgdivert` on and off before and after execution.
The `chroot-script` is also an example on its own that shows how to use The `chroot-script` is also an example on its own that shows how to use
the `chroot-script` function. the `chroot-script` function.

19
doc/tordam-01-overview.md
View File

@ -30,10 +30,10 @@ Abstract
* Once the POST request is received, the node will validate the * Once the POST request is received, the node will validate the
request and return a secret encrypted with the requester's public request and return a secret encrypted with the requester's public
key. key.
* The requester will try to decrypt this secret, and return it plain * The requester will try to decrypt this secret, and return the
back to the node it's announcing to, along with a cryptographic secret in plain text back to the node it's announcing to, along
signature, so the node can confirm the requester is in actual with a cryptographic signature, so the node can confirm the
possession of the private key. requester is in actual possession of the private key.
* Tor DAM **does not validate** if a node is malicious or not. This is a * Tor DAM **does not validate** if a node is malicious or not. This is a
layer that has to be established on top. Tor DAM is just the entry layer that has to be established on top. Tor DAM is just the entry
point into the network. point into the network.
@ -66,11 +66,11 @@ correct example.
} }
``` ```
Sending this as a POST request to a node will make it ask for the public Sending this as a POST request to a node will make it ask for the
key of the given address from a HSDir in the Tor network. It will public key of the given address from a "hidden service directory"
retrieve the public key and try to validate the signature that was made. (HSDir) in the Tor network. It will retrieve the public key and try to
Validating this, we assume that the requester is in possession of the validate the signature that was made. Validating this, we assume that
private key. the requester is in possession of the private key.
Following up, the node shall generate a cryptographically secure random Following up, the node shall generate a cryptographically secure random
string and encrypt it using the before acquired public key. It will then string and encrypt it using the before acquired public key. It will then
@ -86,6 +86,7 @@ be encoded using base64 and sent back to the client:
The client will try to decode and decrypt this secret, and send it back The client will try to decode and decrypt this secret, and send it back
to the node to complete its part of the handshake. The POST request this to the node to complete its part of the handshake. The POST request this
time will contain the following data: time will contain the following data:
* `type` reflects the type of the node * `type` reflects the type of the node
* `address` holds the address of the Tor hidden service * `address` holds the address of the Tor hidden service
* `message` is the decrypted and base64 encoded secret that the server * `message` is the decrypted and base64 encoded secret that the server