Replicate all dependencies of a dataset first

Assuming we want to replicate the following pool: ``` NAME USED AVAIL REFER MOUNTPOINT ORIGIN testpool1 1.10M 38.2M 288K /Volumes/testpool1 - testpool1/A 326K 38.2M 293K /Volumes/testpool1/A testpool1/B@b testpool1/A/D 303K 38.2M 288K /Volumes/testpool1/A/D - testpool1/B 35.5K 38.2M 292K /Volumes/testpool1/B testpool1/C@a testpool1/C 306K 38.2M 290K /Volumes/testpool1/C - ``` Note the clone dependencies: `A -> B -> C`. Currently, syncoid notices that `A` and `B` are clones and defers syncing them. There are two problems: 1. Syncing `A/D` fails because we have deferred `A`. 2. The clone relation `A -> B` will not be recreated since the list of deferred datasets does not take into account clone relations between them. This PR solves both of these problems by collecting all dependencies of a dataset and syncing them before the dataset itself. --- One problematic case remains: if a dataset depends (transitively) on one of its own children, e.g.: ``` NAME USED AVAIL REFER MOUNTPOINT ORIGIN testpool1/E 58.5K 38.7M 298K /Volumes/testpool1/E testpool1/E/D@e testpool1/E/D 37.5K 38.7M 296K /Volumes/testpool1/E/D testpool1/A@d ``` Here, the first run of syncoid will fail to sync `E/D`. I've chosen to ignore this case for now because 1) it seems quite artificial and not like something that would occur in practice very often, and 2) a second run of syncoid will successfully sync `E/D` too (although the clone relation `E -> E/D` is lost).
Merge pull request #920 from phreaker0/dataset-cache
2024-06-09 14:13:15 +02:00 · 2024-04-26 16:28:48 -04:00 · 2024-04-24 00:09:40 +02:00 · 2024-04-23 23:38:47 +02:00 · 2024-04-22 12:14:40 -04:00 · 2024-04-22 12:13:43 -04:00
32 changed files with 2183 additions and 697 deletions
--- a/52
+++ b/52
@ -1,9 +1,49 @@
+2.2.0  [overall]   documentation updates, small fixes (@azmodude, @deviantintegral, @jimsalterjrs, @alexhaydock, @cbreak-black, @kd8bny, @JavaScriptDude, @veeableful, @rsheasby, @Topslakr, @mavhc, @adam-stamand, @joelishness, @jsoref, @dodexahedron, @phreaker0)
+       [syncoid]   implemented flag for preserving properties without the zfs -p flag (@phreaker0)
+       [syncoid]   implemented target snapshot deletion (@mat813)
+       [syncoid]   support bookmarks which are taken in the same second (@delxg, @phreaker0)
+       [syncoid]   exit with an error if the specified src dataset doesn't exist (@phreaker0)
+       [syncoid]   rollback is now done implicitly instead of explicit (@jimsalterjrs, @phreaker0)
+       [syncoid]   append a rand int to the socket name to prevent collisions with parallel invocations (@Gryd3)
+       [syncoid]   implemented support for ssh_config(5) files (@endreszabo)
+       [syncoid]   snapshot hold/unhold support (@rbike)
+       [sanoid]    handle duplicate key definitions gracefully (@phreaker0)
+       [syncoid]   implemented removal of conflicting snapshots with force-delete option (@phreaker0)
+       [sanoid]    implemented pre pruning script hook (@phreaker0)
+       [syncoid]   implemented direct connection support (bypass ssh) for the actual data transfer (@phreaker0)
+
+2.1.0  [overall]   documentation updates, small fixes (@HavardLine, @croadfeldt, @jimsalterjrs, @jim-perkins, @kr4z33, @phreaker0)
+       [syncoid]   do not require user to be specified for syncoid (@aerusso)
+       [syncoid]   implemented option for keeping sync snaps (@phreaker0)
+       [syncoid]   use sudo if necessary for checking pool capabilities regarding resumable send (@phreaker0)
+       [syncoid]   catch another case were the resume state isn't available anymore (@phreaker0)
+       [syncoid]   check for an invalid argument combination (@phreaker0)
+       [syncoid]   fix iszfsbusy check for similar dataset names (@phreaker0)
+       [syncoid]   append timezone offset to the syncoid snapshot name to fix DST collisions (@phreaker0)
+       [packaging] post install script for debian package to remove old unused snapshot cache file (@phreaker0)
+       [syncoid]   implemented fallback for listing snapshots on solaris (@phreaker0)
+       [sanoid]    remove invalid locks (@phreaker0)
+       [packaging] removed debian dependency for systemd (@phreaker0)
+       [sanoid]    move sanoid cache and lock files to subdirectories (@lopsided98)
+       [sanoid]    remove 's in monitoring messages (@dlangille)
+       [findoid]   reworked argument parsing and error out if file path is not provided (@phreaker0)
+       [findoid]   also show current file version if available (@phreaker0)
+       [findoid]   handle FileNotFound errors properly (@phreaker0)
+       [findoid]   don't use hardcoded paths (@phreaker0)
+       [findoid]   improve dataset detection by only including mounted datasets (@phreaker0)
+       [sanoid]    pass more information to pre/post/prune scripts and execute them only once per dataset (@tiedotguy, @phreaker0)
+       [syncoid]   implemented option for preserving recordsizes on initial replications (@phreaker0)
+       [syncoid]   fixed send size estimation for latest FreeBSD versions (@phreaker0)
+       [syncoid]   add ability to configure pv (@gdevenyi)
+       [sanoid]    don't use hardcoded paths (@phreaker0)
+       [syncoid]   gracefully handle error when source dataset disappeared (@mschout)
+
 2.0.3  [sanoid]    reverted DST handling and improved it as quickfix (@phreaker0)

 2.0.2  [overall]   documentation updates, new dependencies, small fixes, more warnings (@benyanke, @matveevandrey, @RulerOf, @klemens-u, @johnramsden, @danielewood, @g-a-c, @hartzell, @fryfrog, @phreaker0)
       [sanoid]    changed and simplified DST handling (@shodanshok)
       [syncoid]   reset partially resume state automatically (@phreaker0)
-       [syncoid]   handle some zfs erros automatically by parsing the stderr outputs (@phreaker0)
+       [syncoid]   handle some zfs errors automatically by parsing the stderr outputs (@phreaker0)
       [syncoid]   fixed ordering of snapshots with the same creation timestamp (@phreaker0)
       [syncoid]   don't use hardcoded paths (@phreaker0)
       [syncoid]   fix for special setup with listsnapshots=on (@phreaker0)
@ -20,7 +60,7 @@
       [syncoid]   option to change mbuffer size (@TerraTech)
       [tests]     fixes for FreeBSD (@phreaker0)
       [sanoid]    support for zfs recursion (@jMichaelA, @phreaker0)
-       [syncoid]   fixed bookmark handling for volumens (@ppcontrib)
+       [syncoid]   fixed bookmark handling for volumes (@ppcontrib)
       [sanoid]    allow time units for monitoring warn/crit values (@phreaker0)

 2.0.1  [sanoid]    fixed broken monthly warn/critical monitoring values in default template (@jimsalterjrs)
@ -58,7 +98,7 @@
       [sanoid]    implemented monitor-capacity flag for checking zpool capacity limits (@phreaker0)
       [syncoid]   Added support for ZStandard compression.(@danielewood)
       [syncoid]   implemented support for excluding datasets from replication with regular expressions (@phreaker0)
-       [syncoid]   correctly parse zfs column output, fixes resumeable send with datasets containing spaces (@phreaker0)
+       [syncoid]   correctly parse zfs column output, fixes resumable send with datasets containing spaces (@phreaker0)
       [syncoid]   added option for using extra identification in the snapshot name for replication to multiple targets (@phreaker0)
       [syncoid]   added option for skipping the parent dataset in recursive replication (@phreaker0)
       [syncoid]   typos (@UnlawfulMonad, @jsavikko, @phreaker0)
@ -92,12 +132,12 @@
       replicating to target/parent/child2.  This could still use some cleanup TBH; syncoid SHOULD exit 3
       if any of these errors happen (to assist detection of errors in scripting) but now would exit 0.

-1.4.12 Sanoid now strips trailing whitespace in template definitions in sanoid.conf, per Github #61
+1.4.12 Sanoid now strips trailing whitespace in template definitions in sanoid.conf, per GitHub #61

 1.4.11 enhanced Syncoid to use zfs `guid` property rather than `creation` property to ensure snapshots on source
       and target actually match. This immediately prevents conflicts due to timezone differences on source and target,
       and also paves the way in the future for Syncoid to find matching snapshots even after `zfs rename` on source
-       or target. Thank you Github user @mailinglists35 for the idea!
+       or target. Thank you GitHub user @mailinglists35 for the idea!

 1.4.10 added --compress=pigz-fast and --compress=pigz-slow. On a Xeon E3-1231v3, pigz-fast is equivalent compression
       to --compress=gzip but with compressed throughput of 75.2 MiB/s instead of 18.1 MiB/s. pigz-slow is around 5%
@ -215,4 +255,4 @@

 1.0.1 ported slightly modified iszfsbusy sub from syncoid to sanoid (to keep from thinning snapshots during replications)

-1.0.0 initial commit to Github
+1.0.0 initial commit to GitHub
--- a/CODE_OF_CONDUCT.md
+++ b/CODE_OF_CONDUCT.md
@ -0,0 +1,128 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -0,0 +1 @@
+Any and all contributions made to this project must be compatible with the project's own GPLv3 license.
--- a/FREEBSD.readme
+++ b/FREEBSD.readme
@ -1,4 +1,10 @@
-FreeBSD users will need to change the Perl shebangs at the top of the executables from #!/usr/bin/perl 
+Syncoid assumes a bourne style shell on remote hosts. Using (t)csh (the default for root under FreeBSD)
+will cause syncoid to fail cryptically due to 2>&1 output redirects.
+
+To use syncoid successfully with FreeBSD targets, you must use the chsh command to change the root shell:
+    root@bsd:~# chsh -s /bin/sh
+
+FreeBSD users will also need to change the Perl shebangs at the top of the executables from #!/usr/bin/perl 
 to #!/usr/local/bin/perl in most cases.

 Sorry folks, but if I set this with #!/usr/bin/env perl as suggested, then nothing works properly
@ -11,14 +17,3 @@ If you don't want to have to change the shebangs, your other option is to drop a
 root@bsd:~# ln -s /usr/local/bin/perl /usr/bin/perl

 After putting this symlink in place, ANY perl script shebanged for Linux will work on your system too.
-
-Syncoid assumes a bourne style shell on remote hosts. Using (t)csh (the default for root under FreeBSD)
-has some known issues:
-
-* If mbuffer is present, syncoid will fail with an "Ambiguous output redirect." error. So if you:
-  root@bsd:~# ln -s /usr/local/bin/mbuffer /usr/bin/mbuffer
-  make sure the remote user is using an sh compatible shell.
-
-To change to a compatible shell, use the chsh command:
-
-root@bsd:~# chsh -s /bin/sh
--- a/INSTALL.md
+++ b/INSTALL.md
@ -8,6 +8,7 @@
 	- [Debian/Ubuntu](#debianubuntu)
 	- [CentOS](#centos)
 	- [FreeBSD](#freebsd)
+	- [Alpine Linux / busybox](#alpine-Linux-busybox-based-distributions)
 	- [Other OSes](#other-oses)
 - [Configuration](#configuration)
 	- [Sanoid](#sanoid)
@ -21,39 +22,52 @@ Install prerequisite software:

 ```bash

-apt install debhelper libcapture-tiny-perl libconfig-inifiles-perl pv lzop mbuffer
+apt install debhelper libcapture-tiny-perl libconfig-inifiles-perl pv lzop mbuffer build-essential git

 ```

 Clone this repo, build the debian package and install it (alternatively you can skip the package and do it manually like described below for CentOS):

 ```bash
-# Download the repo as root to avoid changing permissions later
-sudo git clone https://github.com/jimsalterjrs/sanoid.git
+git clone https://github.com/jimsalterjrs/sanoid.git
 cd sanoid
 # checkout latest stable release or stay on master for bleeding edge stuff (but expect bugs!)
 git checkout $(git tag | grep "^v" | tail -n 1)
 ln -s packages/debian .
 dpkg-buildpackage -uc -us
-apt install ../sanoid_*_all.deb
+sudo apt install ../sanoid_*_all.deb
 ```

 Enable sanoid timer:
 ```bash
 # enable and start the sanoid timer
-sudo systemctl enable sanoid.timer
-sudo systemctl start sanoid.timer
+sudo systemctl enable --now sanoid.timer
 ```

-## CentOS
+## CentOS/RHEL

 Install prerequisite software:

 ```bash
-# Install and enable epel if we don't already have it, and git too
+# Install and enable EPEL if we don't already have it, and git too:
+# (Note that on RHEL we cannot enable EPEL with the epel-release
+# package, so you should follow the instructions on the main EPEL site.)
 sudo yum install -y epel-release git
+# On CentOS, we also need to enable the PowerTools repo:
+sudo yum config-manager --set-enabled powertools
+# For Centos 8 you need to enable the PowerTools repo to make all the needed Perl modules available (Recommended)
+sudo dnf config-manager --set-enabled powertools
+# On RHEL, instead of PowerTools, we need to enable the CodeReady Builder repo:
+sudo subscription-manager repos --enable=codeready-builder-for-rhel-8-x86_64-rpms
 # Install the packages that Sanoid depends on:
-sudo yum install -y perl-Config-IniFiles perl-Data-Dumper perl-Capture-Tiny lzop mbuffer mhash pv
+sudo yum install -y perl-Config-IniFiles perl-Data-Dumper perl-Capture-Tiny perl-Getopt-Long lzop mbuffer mhash pv
+# The repositories above should contain all the relevant Perl modules, but if you
+# still cannot find them then you can install them from CPAN manually:
+sudo dnf install perl-CPAN perl-CPAN
+cpan # answer the questions and paste the following lines:
+# install Capture::Tiny
+# install Config::IniFiles
+# install Getopt::Long
 ```

 Clone this repo, then put the executables and config files into the appropriate directories:
@ -136,8 +150,7 @@ sudo systemctl daemon-reload
 # Enable sanoid-prune.service to allow it to be triggered by sanoid.service
 sudo systemctl enable sanoid-prune.service
 # Enable and start the Sanoid timer
-sudo systemctl enable sanoid.timer
-sudo systemctl start sanoid.timer
+sudo systemctl enable --now sanoid.timer
 ```

 Now, proceed to configure [**Sanoid**](#configuration)
@ -147,7 +160,7 @@ Now, proceed to configure [**Sanoid**](#configuration)
 Install prerequisite software:

 ```bash
-pkg install p5-Config-Inifiles p5-Capture-Tiny pv mbuffer lzop
+pkg install p5-Config-Inifiles p5-Capture-Tiny pv mbuffer lzop sanoid
 ```

 **Additional notes:**
@ -156,7 +169,59 @@ pkg install p5-Config-Inifiles p5-Capture-Tiny pv mbuffer lzop

 *   Simplest path workaround is symlinks, eg `ln -s /usr/local/bin/lzop /usr/bin/lzop` or similar, as appropriate to create links in **/usr/bin** to wherever the utilities actually are on your system.

-*   See note about mbuffer and other things in FREEBSD.readme
+*   See note about tcsh unpleasantness and other things in FREEBSD.readme
+
+## Alpine Linux / busybox based distributions
+
+The busybox implementation of ps is lacking needed arguments so a proper ps program needs to be installed.
+For Alpine Linux this can be done with:
+
+`apk --no-cache add procps`
+
+## MacOS
+
+Install prerequisite software:
+
+```
+perl -MCPAN -e install Config::IniFiles
+```
+
+The crontab can be used as on a normal unix. To use launchd instead, this example config file can be use can be used. Modify it for your needs. In particular, adjust the sanoid path.
+It will start sanoid once per hour, at minute 51. Missed invocations due to standby will be merged into a single invocation at the next wakeup.
+
+```bash
+cat << "EOF" | sudo tee /Library/LaunchDaemons/net.openoid.Sanoid.plist
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>Label</key>
+	<string>net.openoid.Sanoid</string>
+	<key>ProgramArguments</key>
+	<array>
+		<string>/usr/local/sanoid/sanoid</string>
+		<string>--cron</string>
+	</array>
+	<key>EnvironmentVariables</key>
+	<dict>
+		<key>TZ</key>
+		<string>UTC</string>
+		<key>PATH</key>
+		<string>/usr/local/zfs/bin:$PATH:/usr/local/bin</string>
+	</dict>
+	<key>StartCalendarInterval</key>
+	<array>
+		<dict>
+			<key>Minute</key>
+			<integer>51</integer>
+		</dict>
+	</array>
+</dict>
+</plist>
+EOF
+
+sudo launchctl load /Library/LaunchDaemons/net.openoid.Sanoid.plist
+```

 ## Other OSes

@ -171,6 +236,17 @@ pkg install p5-Config-Inifiles p5-Capture-Tiny pv mbuffer lzop
 3.  Create the config directory `/etc/sanoid` and put `sanoid.defaults.conf` in there, and create `sanoid.conf` in it too
 4.  Create a cron job or a systemd timer that runs `sanoid --cron` once per minute

+## cron
+
+If you use cron there is the need to ensure that only one instance of sanoid is run at any time (or else there will be funny error messages about missing snapshots, ...). It's also good practice to separate the snapshot taking and pruning so the later won't block the former in case of long running pruning operations. Following is the recommend setup for a standard install:
+
+```
+*/15 * * * * root flock -n /var/run/sanoid/cron-take.lock -c "TZ=UTC sanoid --take-snapshots"
+*/15 * * * * root flock -n /var/run/sanoid/cron-prune.lock -c "sanoid --prune-snapshots"
+```
+
+Adapt the timer interval to the lowest configured snapshot interval.
+
 # Configuration

 **Sanoid** won't do anything useful unless you tell it how to handle your ZFS datasets in `/etc/sanoid/sanoid.conf`.
@ -182,3 +258,12 @@ pkg install p5-Config-Inifiles p5-Capture-Tiny pv mbuffer lzop
 Take a look at the files `sanoid.defaults.conf` and `sanoid.conf` for all possible configuration options.

 Also have a look at the README.md for a simpler suggestion for `sanoid.conf`.
+
+## Syncoid
+If you are pushing or pulling from a remote host, create a user with privileges to `ssh` as well as `sudo`. To ensure that `zfs send/receive` can execute, adjust the privileges of the user to execute `sudo` **without** a password for only the `zfs` binary (run `which zfs` to find the path of the `zfs` binary). Modify `/etc/sudoers` by running `# visudo`. Add the following line for your user.
+
+```
+...
+<user> ALL=NOPASSWD: <path of zfs binary>
+...
+```
--- a/README.md
+++ b/README.md
@ -1,15 +1,26 @@
-<p align="center"><img src="http://www.openoid.net/wp-content/themes/openoid/images/sanoid_logo.png" alt="sanoid logo" title="sanoid logo"></p>
+<table align="center">
+	<tr>
+		<td border="1" width="750">
+			<p align="center">
+				<img src="http://www.openoid.net/wp-content/themes/openoid/images/sanoid_logo.png" alt="sanoid logo" title="sanoid logo">
+			</p>
+			<img src="https://openoid.net/gplv3-127x51.png" width=127 height=51 align="right">
+			<p align="left">Sanoid is provided to you completely free and libre, now and in perpetuity, via the GPL v3.0 license. If you find the project useful, please consider either a recurring or one-time donation at <a href="https://www.patreon.com/PracticalZFS" target="_blank">Patreon</a> or <a href="https://www.paypal.com/donate/?hosted_button_id=5BLPNV86D4S9N" target="_blank">PayPal</a>—your contributions will support both this project and the Practical ZFS <a href="https://discourse.practicalzfs.com/" target="_blank">forum</a>.
+			</p>
+		</td>
+	</tr>
+</table>

-<img src="http://openoid.net/gplv3-127x51.png" width=127 height=51 align="right">Sanoid is a policy-driven snapshot management tool for ZFS filesystems.  When combined with the Linux KVM hypervisor, you can use it to make your systems <a href="http://openoid.net/transcend" target="_blank">functionally immortal</a>.  
+Sanoid is a policy-driven snapshot management tool for ZFS filesystems.  When combined with the Linux KVM hypervisor, you can use it to make your systems <a href="https://openoid.net/transcend" target="_blank">functionally immortal</a> via automated snapshot management and over-the-air replication.

 <p align="center"><a href="https://youtu.be/ZgowLNBsu00" target="_blank"><img src="http://www.openoid.net/sanoid_video_launcher.png" alt="sanoid rollback demo" title="sanoid rollback demo"></a><br clear="all"><sup>(Real time demo: rolling back a full-scale cryptomalware infection in seconds!)</sup></p>

-More prosaically, you can use Sanoid to create, automatically thin, and monitor snapshots and pool health from a single eminently human-readable TOML config file at /etc/sanoid/sanoid.conf.  (Sanoid also requires a "defaults" file located at /etc/sanoid/sanoid.defaults.conf, which is not user-editable.)  A typical Sanoid system would have a single cron job:
+More prosaically, you can use Sanoid to create, automatically thin, and monitor snapshots and pool health from a single eminently human-readable TOML config file at /etc/sanoid/sanoid.conf.  (Sanoid also requires a "defaults" file located at /etc/sanoid/sanoid.defaults.conf, which is not user-editable.)  A typical Sanoid system would have a single cron job but see INSTALL.md for more details:
 ```
 * * * * * TZ=UTC /usr/local/bin/sanoid --cron
 ```

-`Note`: Using UTC as timezone is recommend to prevent problems with daylight saving times
+`Note`: Using UTC as timezone is recommended to prevent problems with daylight saving times

 And its /etc/sanoid/sanoid.conf might look something like this:

@ -39,6 +50,10 @@ And its /etc/sanoid/sanoid.conf might look something like this:

 Which would be enough to tell sanoid to take and keep 36 hourly snapshots, 30 dailies, 3 monthlies, and no yearlies for all datasets under data/images (but not data/images itself, since process_children_only is set).  Except in the case of data/images/win7, which follows the same template (since it's a child of data/images) but only keeps 4 hourlies for whatever reason.

+For more full details on sanoid.conf settings see [Wiki page](https://github.com/jimsalterjrs/sanoid/wiki/Sanoid#options).
+
+**Note**: Be aware that if you don't specify some interval options the defaults will be used (from /etc/sanoid/sanoid.defaults.conf)
+
 ##### Sanoid Command Line Options

 + --cron
@ -91,7 +106,7 @@ Which would be enough to tell sanoid to take and keep 36 hourly snapshots, 30 da

 + --quiet

-	Supress non-error output.
+	Suppress non-error output.

 + --verbose

@ -99,7 +114,7 @@ Which would be enough to tell sanoid to take and keep 36 hourly snapshots, 30 da

 + --debug

-	This prints out quite alot of additional information during a sanoid run, and is normally not needed.
+	This prints out quite a lot of additional information during a sanoid run, and is normally not needed.

 + --readonly

@ -109,6 +124,100 @@ Which would be enough to tell sanoid to take and keep 36 hourly snapshots, 30 da

 	Show help message.

+### Sanoid script hooks
+
+There are three script types which can optionally be executed at various stages in the lifecycle of a snapshot:
+
+#### `pre_snapshot_script`
+
+Will be executed before the snapshot(s) of a single dataset are taken. The following environment variables are passed:
+
+| Env vars           | Description                                                                                                                                          |
+| -----------------  | -----------                                                                                                                                          |
+| `SANOID_SCRIPT`    | The type of script being executed, one of `pre`, `post`, or `prune`.  Allows for one script to be used for multiple tasks                            |
+| `SANOID_TARGET`    | **DEPRECATED** The dataset about to be snapshot (only the first dataset will be provided)                                                            |
+| `SANOID_TARGETS`   | Comma separated list of all datasets to be snapshotted (currently only a single dataset, multiple datasets will be possible later with atomic groups) |
+| `SANOID_SNAPNAME`  | **DEPRECATED** The name of the snapshot that will be taken (only the first name will be provided, does not include the dataset name)                 |
+| `SANOID_SNAPNAMES` | Comma separated list of all snapshot names that will be taken (does not include the dataset name)                                                    |
+| `SANOID_TYPES`     | Comma separated list of all snapshot types to be taken (yearly, monthly, weekly, daily, hourly, frequently)                                          |
+
+If the script returns a non-zero exit code, the snapshot(s) will not be taken unless `no_inconsistent_snapshot` is false.
+
+#### `post_snapshot_script`
+
+Will be executed when:
+
+- The pre-snapshot script succeeded or
+- The pre-snapshot script failed and `force_post_snapshot_script` is true.
+
+| Env vars             | Description                                          |
+| -------------------- | -----------                                          |
+| `SANOID_SCRIPT`      | as above                                             |
+| `SANOID_TARGET`      | **DEPRECATED** as above                              |
+| `SANOID_TARGETS`     | as above                                             |
+| `SANOID_SNAPNAME`    | **DEPRECATED** as above                              |
+| `SANOID_SNAPNAMES`   | as above                                             |
+| `SANOID_TYPES`       | as above                                             |
+| `SANOID_PRE_FAILURE` | This will indicate if the pre-snapshot script failed |
+
+#### `pruning_script`
+
+Will be executed after a snapshot is successfully deleted. The following environment variables will be passed:
+
+| Env vars          | Description |
+| ----------------- | ----------- |
+| `SANOID_SCRIPT`   | as above    |
+| `SANOID_TARGET`   | as above    |
+| `SANOID_SNAPNAME` | as above    |
+
+
+#### example
+
+**sanoid.conf**:
+```
+...
+[sanoid-test-0]
+	use_template = production
+	recursive = yes
+	pre_snapshot_script = /tmp/debug.sh
+	post_snapshot_script = /tmp/debug.sh
+	pruning_script = /tmp/debug.sh
+...
+```
+
+**verbose sanoid output**:
+```
+...
+executing pre_snapshot_script '/tmp/debug.sh' on dataset 'sanoid-test-0'
+taking snapshot sanoid-test-0@autosnap_2020-02-12_14:49:33_yearly
+taking snapshot sanoid-test-0@autosnap_2020-02-12_14:49:33_monthly
+taking snapshot sanoid-test-0@autosnap_2020-02-12_14:49:33_daily
+taking snapshot sanoid-test-0@autosnap_2020-02-12_14:49:33_hourly
+executing post_snapshot_script '/tmp/debug.sh' on dataset 'sanoid-test-0'
+...
+```
+
+**pre script env variables**:
+```
+SANOID_SCRIPT=pre
+SANOID_TARGET=sanoid-test-0/b/bb
+SANOID_TARGETS=sanoid-test-0/b/bb
+SANOID_SNAPNAME=autosnap_2020-02-12_14:49:32_yearly
+SANOID_SNAPNAMES=autosnap_2020-02-12_14:49:32_yearly,autosnap_2020-02-12_14:49:32_monthly,autosnap_2020-02-12_14:49:32_daily,autosnap_2020-02-12_14:49:32_hourly
+SANOID_TYPES=yearly,monthly,daily,hourly
+```
+
+**post script env variables**:
+```
+SANOID_SCRIPT=post
+SANOID_TARGET=sanoid-test-0/b/bb
+SANOID_TARGETS=sanoid-test-0/b/bb
+SANOID_SNAPNAME=autosnap_2020-02-12_14:49:32_yearly
+SANOID_SNAPNAMES=autosnap_2020-02-12_14:49:32_yearly,autosnap_2020-02-12_14:49:32_monthly,autosnap_2020-02-12_14:49:32_daily,autosnap_2020-02-12_14:49:32_hourly
+SANOID_TYPES=yearly,monthly,daily,hourly
+SANOID_PRE_FAILURE=0
+```
+
 ----------

 # Syncoid
@ -134,7 +243,7 @@ syncoid root@remotehost:data/images/vm backup/images/vm
 Which would pull-replicate the filesystem from the remote host to the local system over an SSH tunnel.

 Syncoid supports recursive replication (replication of a dataset and all its child datasets) and uses mbuffer buffering, lzop compression, and pv progress bars if the utilities are available on the systems used.
-If ZFS supports resumeable send/receive streams on both the source and target those will be enabled as default.
+If ZFS supports resumable send/receive streams on both the source and target those will be enabled as default.

 As of 1.4.18, syncoid also automatically supports and enables resume of interrupted replication when both source and target support this feature.

@ -176,7 +285,7 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup

 + --identifier=

-	Adds the given identifier to the snapshot name after "syncoid_" prefix and before the hostname. This enables the use case of reliable replication to multiple targets from the same host. The following chars are allowed: a-z, A-Z, 0-9, _, -, : and . .
+	Adds the given identifier to the snapshot and hold name after "syncoid_" prefix and before the hostname. This enables the use case of reliable replication to multiple targets from the same host. The following chars are allowed: a-z, A-Z, 0-9, _, -, : and . .

 + -r --recursive

@ -188,15 +297,15 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup

 + --compress <compression type>

-	Currently accepted options: gzip, pigz-fast, pigz-slow, zstd-fast, zstd-slow, lz4, xz, lzo (default) & none. If the selected compression method is unavailable on the source and destination, no compression will be used.
+	Compression method to use for network transfer. Currently accepted options: gzip, pigz-fast, pigz-slow, zstd-fast, zstd-slow, lz4, xz, lzo (default) & none. If the selected compression method is unavailable on the source and destination, no compression will be used.

 + --source-bwlimit <limit t|g|m|k>

-	This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limits are desired. 
+	This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limits are desired.

-+ --target-bw-limit <limit t|g|m|k>
+ --target-bwlimit <limit t|g|m|k>

-	This is the bandwidth limit in bytes (kbytes, mbytesm etc) per second imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limits are desired.
+	This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limits are desired.

 + --no-command-checks

@ -210,9 +319,29 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup

 	This argument tells syncoid to restrict itself to existing snapshots, instead of creating a semi-ephemeral syncoid snapshot at execution time. Especially useful in multi-target (A->B, A->C) replication schemes, where you might otherwise accumulate a large number of foreign syncoid snapshots.

+ --keep-sync-snap
+
+    This argument tells syncoid to skip pruning old snapshots created and used by syncoid for replication if '--no-sync-snap' isn't specified.
+
 + --create-bookmark

-	This argument tells syncoid to create a zfs bookmark for the newest snapshot after it got replicated successfully. The bookmark name will be equal to the snapshot name. Only works in combination with the --no-sync-snap option. This can be very useful for irregular replication where the last matching snapshot on the source was already deleted but the bookmark remains so a replication is still possible. 
+	This argument tells syncoid to create a zfs bookmark for the newest snapshot after it got replicated successfully. The bookmark name will be equal to the snapshot name. Only works in combination with the --no-sync-snap option. This can be very useful for irregular replication where the last matching snapshot on the source was already deleted but the bookmark remains so a replication is still possible.
+
+ --use-hold
+	This argument tells syncoid to add a hold to the newest snapshot on the source and target after replication succeeds and to remove the hold after the next successful replication. Setting a hold prevents the snapshots from being destroyed. The hold name includes the identifier if set. This allows for separate holds in case of replication to multiple targets.
+
+ --preserve-recordsize
+
+	This argument tells syncoid to set the recordsize on the target before writing any data to it matching the one set on the replication src. This only applies to initial sends.
+
+ --preserve-properties
+
+	This argument tells syncoid to get all locally set dataset properties from the source and apply all supported ones on the target before writing any data. It's similar to the '-p' flag for zfs send but also works for encrypted datasets in non raw sends. This only applies to initial sends.
+
+ --delete-target-snapshots
+
+	With this argument snapshots which are missing on the source will be destroyed on the target. Use this if you only want to handle snapshots on the source.
+	Note that snapshot deletion is only done after a successful synchronization. If no new snapshots are found, no synchronization is done and no deletion either.

 + --no-clone-rollback

@ -224,19 +353,33 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup

 + --exclude=REGEX

-	The given regular expression will be matched against all datasets which would be synced by this run and excludes them. This argument can be specified multiple times.
+	__DEPRECATION NOTICE:__ `--exclude` has been deprecated and will be removed in a future release. Please use `--exclude-datasets` instead.
+
+	The given regular expression will be matched against all datasets which would be synced by this run and excludes them. This argument can be specified multiple times. The provided regex pattern is matched against the dataset name only; this option does not affect which snapshots are synchronized. If both `--exclude` and `--exclude-datasets` are provided, then `--exclude` is ignored.
+
+ --exclude-datasets=REGEX
+
+	The given regular expression will be matched against all datasets which would be synced by this run and excludes them. This argument can be specified multiple times. The provided regex pattern is matched against the dataset name only; this option does not affect which snapshots are synchronized.
+
+ --exclude-snaps=REGEX
+
+	Exclude specific snapshots that match the given regular expression. The provided regex pattern is matched against the snapshot name only. Can be specified multiple times. If a snapshot matches both the exclude-snaps and include-snaps patterns, then it will be excluded.
+
+ --include-snaps=REGEX
+
+	Only include snapshots that match the given regular expression. The provided regex pattern is matched against the snapshot name only. Can be specified multiple times. If a snapshot matches both the exclude-snaps and include-snaps patterns, then it will be excluded.

 + --no-resume

-	This argument tells syncoid to not use resumeable zfs send/receive streams.
+	This argument tells syncoid to not use resumable zfs send/receive streams.

 + --force-delete

-	Remove target datasets recursively (WARNING: this will also affect child datasets with matching snapshots/bookmarks), if there are no matching snapshots/bookmarks.
+	Remove target datasets recursively (WARNING: this will also affect child datasets with matching snapshots/bookmarks), if there are no matching snapshots/bookmarks. Also removes conflicting snapshots if the replication would fail because of a snapshot which has the same name between source and target but different contents.

 + --no-clone-handling

-	This argument tells syncoid to not recreate clones on the targe on initial sync and doing a normal replication instead.
+	This argument tells syncoid to not recreate clones on the target on initial sync, and do a normal replication instead.

 + --dumpsnaps

@ -262,13 +405,18 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup

 	Use specified identity file as per ssh -i.

+ --insecure-direct-connection=IP:PORT[,IP:PORT,[TIMEOUT,[mbuffer]]]
+
+	WARNING: This is an insecure option as the data is not encrypted while being sent over the network. Only use if you trust the complete network path.
+	Use a direct tcp connection (with socat and busybox nc/mbuffer) for the actual zfs send/recv stream. All control commands are still executed via the ssh connection. The first address pair is used for connecting to the target host from the source host and the second pair is for listening on the target host. If the later isn't provided the same as the former is used. This can be used for saturating high throughput connection like >= 10GBe network which isn't easy with the overhead off ssh. It can also be useful for encrypted datasets to lower the cpu usage needed for replication but be aware that metadata is NOT ENCRYPTED in this case. The default timeout is 60 seconds and can be overridden by providing it as third argument. By default busybox nc is used for the listeing tcp socket, if mbuffer is preferred specify its name as fourth argument but be aware that mbuffer listens on all interfaces and uses an optionally provided ip address for access restriction (This option can't be used for relaying between two remote hosts)
+
 + --quiet

-	Supress non-error output.
+	Suppress non-error output.

 + --debug

-	This prints out quite alot of additional information during a sanoid run, and is normally not needed.
+	This prints out quite a lot of additional information during a syncoid run, and is normally not needed.

 + --help

--- a/SECURITY.md
+++ b/SECURITY.md
@ -0,0 +1,13 @@
+# Security Policy
+
+## Supported Versions
+
+The Sanoid project directly supports both the code in the main branch, and the last two releases found here on GitHub.
+
+Community support is available for all versions, with the understanding that in some cases "upgrade to a newer version" may be the support offered. 
+If you've installed Sanoid from your distribution's repositories, we're happy to offer community support with the same caveat!
+
+## Reporting a Vulnerability
+
+If you believe you've found a serious security vulnerability in Sanoid, please create an Issue here on GitHub. If you prefer a private contact channel to disclose
+particularly sensitive or private details, you may request one in the GitHub Issue you create.
--- a/2
+++ b/2
@ -1 +1 @@
-2.0.3
+2.2.0
--- a/5
+++ b/5
@ -4,7 +4,7 @@
 # from http://www.gnu.org/licenses/gpl-3.0.html on 2014-11-17.  A copy should also be available in this
 # project's Git repository at https://github.com/jimsalterjrs/sanoid/blob/master/LICENSE.

-$::VERSION = '2.0.2';
+$::VERSION = '2.2.0';

 use strict;
 use warnings;
@ -25,6 +25,9 @@ if ($args{'path'} eq '') {
 	}
 }

+# resolve given path to a canonical one
+$args{'path'} = Cwd::realpath($args{'path'});
+
 my $dataset = getdataset($args{'path'});

 my %versions = getversions($args{'path'}, $dataset);
--- a/packages/debian/changelog
+++ b/packages/debian/changelog
@ -1,3 +1,51 @@
+sanoid (2.2.0) unstable; urgency=medium
+
+  [overall]   documentation updates, small fixes (@azmodude, @deviantintegral, @jimsalterjrs, @alexhaydock, @cbreak-black, @kd8bny, @JavaScriptDude, @veeableful, @rsheasby, @Topslakr, @mavhc, @adam-stamand, @joelishness, @jsoref, @dodexahedron, @phreaker0)
+  [syncoid]   implemented flag for preserving properties without the zfs -p flag (@phreaker0)
+  [syncoid]   implemented target snapshot deletion (@mat813)
+  [syncoid]   support bookmarks which are taken in the same second (@delxg, @phreaker0)
+  [syncoid]   exit with an error if the specified src dataset doesn't exist (@phreaker0)
+  [syncoid]   rollback is now done implicitly instead of explicit (@jimsalterjrs, @phreaker0)
+  [syncoid]   append a rand int to the socket name to prevent collisions with parallel invocations (@Gryd3)
+  [syncoid]   implemented support for ssh_config(5) files (@endreszabo)
+  [syncoid]   snapshot hold/unhold support (@rbike)
+  [sanoid]    handle duplicate key definitions gracefully (@phreaker0)
+  [syncoid]   implemented removal of conflicting snapshots with force-delete option (@phreaker0)
+  [sanoid]    implemented pre pruning script hook (@phreaker0)
+  [syncoid]   implemented direct connection support (bypass ssh) for the actual data transfer (@phreaker0)
+
+ -- Jim Salter <github@jrs-s.net>  Tue, 18 Jul 2023 10:04:00 +0200
+
+sanoid (2.1.0) unstable; urgency=medium
+
+  [overall]   documentation updates, small fixes (@HavardLine, @croadfeldt, @jimsalterjrs, @jim-perkins, @kr4z33, @phreaker0)
+  [syncoid]   do not require user to be specified for syncoid (@aerusso)
+  [syncoid]   implemented option for keeping sync snaps (@phreaker0)
+  [syncoid]   use sudo if necessary for checking pool capabilities regarding resumable send (@phreaker0)
+  [syncoid]   catch another case were the resume state isn't available anymore (@phreaker0)
+  [syncoid]   check for an invalid argument combination (@phreaker0)
+  [syncoid]   fix iszfsbusy check for similar dataset names (@phreaker0)
+  [syncoid]   append timezone offset to the syncoid snapshot name to fix DST collisions (@phreaker0)
+  [packaging] post install script for debian package to remove old unused snapshot cache file (@phreaker0)
+  [syncoid]   implemented fallback for listing snapshots on solaris (@phreaker0)
+  [sanoid]    remove invalid locks (@phreaker0)
+  [packaging] removed debian dependency for systemd (@phreaker0)
+  [sanoid]    move sanoid cache and lock files to subdirectories (@lopsided98)
+  [sanoid]    remove 's in monitoring messages (@dlangille)
+  [findoid]   reworked argument parsing and error out if file path is not provided (@phreaker0)
+  [findoid]   also show current file version if available (@phreaker0)
+  [findoid]   handle FileNotFound errors properly (@phreaker0)
+  [findoid]   don't use hardcoded paths (@phreaker0)
+  [findoid]   improve dataset detection by only including mounted datasets (@phreaker0)
+  [sanoid]    pass more information to pre/post/prune scripts and execute them only once per dataset (@tiedotguy, @phreaker0)
+  [syncoid]   implemented option for preserving recordsizes on initial replications (@phreaker0)
+  [syncoid]   fixed send size estimation for latest FreeBSD versions (@phreaker0)
+  [syncoid]   add ability to configure pv (@gdevenyi)
+  [sanoid]    don't use hardcoded paths (@phreaker0)
+  [syncoid]   gracefully handle error when source dataset disappeared (@mschout)
+
+ -- Jim Salter <github@jrs-s.net>  Tue, 24 Nov 2020 11:47:00 +0100
+
 sanoid (2.0.3) unstable; urgency=medium

  [sanoid]    reverted DST handling and improved it as quickfix (@phreaker0)
@ -9,7 +57,7 @@ sanoid (2.0.2) unstable; urgency=medium
  [overall]   documentation updates, new dependencies, small fixes, more warnings (@benyanke, @matveevandrey, @RulerOf, @klemens-u, @johnramsden, @danielewood, @g-a-c, @hartzell, @fryfrog, @phreaker0)
  [syncoid]   changed and simplified DST handling (@shodanshok)
  [syncoid]   reset partially resume state automatically (@phreaker0)
-  [syncoid]   handle some zfs erros automatically by parsing the stderr outputs (@phreaker0)
+  [syncoid]   handle some zfs errors automatically by parsing the stderr outputs (@phreaker0)
  [syncoid]   fixed ordering of snapshots with the same creation timestamp (@phreaker0)
  [syncoid]   don't use hardcoded paths (@phreaker0)
  [syncoid]   fix for special setup with listsnapshots=on (@phreaker0)
@ -72,7 +120,7 @@ sanoid (2.0.0) unstable; urgency=medium
  [sanoid]    implemented monitor-capacity flag for checking zpool capacity limits (@phreaker0)
  [syncoid]   Added support for ZStandard compression.(@danielewood)
  [syncoid]   implemented support for excluding datasets from replication with regular expressions (@phreaker0)
-  [syncoid]   correctly parse zfs column output, fixes resumeable send with datasets containing spaces (@phreaker0)
+  [syncoid]   correctly parse zfs column output, fixes resumable send with datasets containing spaces (@phreaker0)
  [syncoid]   added option for using extra identification in the snapshot name for replication to multiple targets (@phreaker0)
  [syncoid]   added option for skipping the parent dataset in recursive replication (@phreaker0)
  [syncoid]   typos (@UnlawfulMonad, @jsavikko, @phreaker0)
--- a/packages/debian/control
+++ b/packages/debian/control
@ -12,7 +12,7 @@ Package: sanoid
 Architecture: all
 Depends: libcapture-tiny-perl,
         libconfig-inifiles-perl,
-         zfsutils-linux | zfs,
+         zfsutils-linux | zfs | openzfs-zfsutils,
         ${misc:Depends},
         ${perl:Depends}
 Recommends: gzip,
--- a/packages/rhel/sanoid-1.4.18.tar.gz
+++ b/packages/rhel/sanoid-1.4.18.tar.gz
--- a/packages/rhel/sanoid.spec
+++ b/packages/rhel/sanoid.spec
@ -1,4 +1,4 @@
-%global version 2.0.3
+%global version 2.2.0
 %global git_tag v%{version}

 # Enable with systemctl "enable sanoid.timer"
@ -111,11 +111,15 @@ echo "* * * * * root %{_sbindir}/sanoid --cron" > %{buildroot}%{_docdir}/%{name}
 %endif

 %changelog
+* Tue Jul 18 2023 Christoph Klaffl <christoph@phreaker.eu> - 2.2.0
+- Bump to 2.2.0
+* Tue Nov 24 2020 Christoph Klaffl <christoph@phreaker.eu> - 2.1.0
+- Bump to 2.1.0
 * Wed Oct 02 2019 Christoph Klaffl <christoph@phreaker.eu> - 2.0.3
 - Bump to 2.0.3
 * Wed Sep 25 2019 Christoph Klaffl <christoph@phreaker.eu> - 2.0.2
 - Bump to 2.0.2
-* Wed Dec 04 2018 Christoph Klaffl <christoph@phreaker.eu> - 2.0.0
+* Tue Dec 04 2018 Christoph Klaffl <christoph@phreaker.eu> - 2.0.0
 - Bump to 2.0.0
 * Sat Apr 28 2018 Dominic Robinson <github@dcrdev.com> - 1.4.18-1
 - Bump to 1.4.18
--- a/packages/rhel/sources
+++ b/packages/rhel/sources
@ -1 +0,0 @@
-cf0ec23c310d2f9416ebabe48f5edb73  sanoid-1.4.18.tar.gz
--- a/368
+++ b/368
@ -4,7 +4,7 @@
 # from http://www.gnu.org/licenses/gpl-3.0.html on 2014-11-17.  A copy should also be available in this
 # project's Git repository at https://github.com/jimsalterjrs/sanoid/blob/master/LICENSE.

-$::VERSION = '2.0.3';
+$::VERSION = '2.2.0';
 my $MINIMUM_DEFAULTS_VERSION = 2;

 use strict;
@ -30,39 +30,86 @@ GetOptions(\%args, "verbose", "debug", "cron", "readonly", "quiet",
          ) or pod2usage(2);

 # If only config directory (or nothing) has been specified, default to --cron --verbose
-if (keys %args < 2) {
+if (keys %args < 4) {
 	$args{'cron'} = 1;
 	$args{'verbose'} = 1;
 }

-my $pscmd = '/bin/ps';
+# for compatibility reasons, older versions used hardcoded command paths
+$ENV{'PATH'} = $ENV{'PATH'} . ":/bin:/sbin";

-my $zfs = '/sbin/zfs';
-my $zpool = '/sbin/zpool';
+my $pscmd = 'ps';
+
+my $zfs = 'zfs';
+my $zpool = 'zpool';

 my $conf_file = "$args{'configdir'}/sanoid.conf";
 my $default_conf_file = "$args{'configdir'}/sanoid.defaults.conf";

-# parse config file
-my %config = init($conf_file,$default_conf_file);
-
 my $cache_dir = $args{'cache-dir'};
 my $run_dir = $args{'run-dir'};

 make_path($cache_dir);
 make_path($run_dir);

-# if we call getsnaps(%config,1) it will forcibly update the cache, TTL or no TTL
-my $forcecacheupdate = 0;
+my $cacheTTL = 1200; # 20 minutes
+
+# Allow a much older snapshot cache file than default if _only_ "--monitor-*" action commands are given
+# (ignore "--verbose", "--configdir" etc)
+if (
+	(
+		$args{'monitor-snapshots'}
+		|| $args{'monitor-health'}
+		|| $args{'monitor-capacity'}
+	) && ! (
+		$args{'cron'}
+		|| $args{'force-update'}
+		|| $args{'take-snapshots'}
+		|| $args{'prune-snapshots'}
+		|| $args{'force-prune'}
+	)
+) {
+	# The command combination above must not assert true for any command that takes or prunes snapshots
+	$cacheTTL = 18000; # 5 hours
+	if ($args{'debug'}) { print "DEBUG: command combo means that the cache file (provided it exists) will be allowed to be older than default.\n"; }
+}
+
+# snapshot cache
 my $cache = "$cache_dir/snapshots.txt";
-my $cacheTTL = 900; # 15 minutes
-my %snaps = getsnaps( \%config, $cacheTTL, $forcecacheupdate );
+
+# configured dataset cache
+my $cachedatasetspath = "$cache_dir/datasets.txt";
+my @cachedatasets;
+
+# parse config file
+my %config = init($conf_file,$default_conf_file);
+
 my %pruned;
 my %capacitycache;

-my %snapsbytype = getsnapsbytype( \%config, \%snaps );
+my %snaps;
+my %snapsbytype;
+my %snapsbypath;

-my %snapsbypath = getsnapsbypath( \%config, \%snaps );
+# get snapshot list only if needed
+if ($args{'monitor-snapshots'}
+	|| $args{'monitor-health'}
+	|| $args{'cron'}
+	|| $args{'take-snapshots'}
+	|| $args{'prune-snapshots'}
+	|| $args{'force-update'}
+	|| $args{'debug'}
+) {
+	my $forcecacheupdate = 0;
+	if ($args{'force-update'}) 	{
+		$forcecacheupdate = 1;
+	}
+
+	%snaps = getsnaps( \%config, $cacheTTL, $forcecacheupdate);
+
+	%snapsbytype = getsnapsbytype( \%config, \%snaps );
+	%snapsbypath = getsnapsbypath( \%config, \%snaps );
+}

 # let's make it a little easier to be consistent passing these hashes in the same order to each sub
 my @params = ( \%config, \%snaps, \%snapsbytype, \%snapsbypath );
@ -71,7 +118,6 @@ if ($args{'debug'})		{ $args{'verbose'}=1; blabber (@params); }
 if ($args{'monitor-snapshots'}) { monitor_snapshots(@params); }
 if ($args{'monitor-health'}) 	{ monitor_health(@params); }
 if ($args{'monitor-capacity'})  { monitor_capacity(@params); }
-if ($args{'force-update'}) 	{ my $snaps = getsnaps( \%config, $cacheTTL, 1 ); }

 if ($args{'cron'}) {
 	if ($args{'quiet'}) { $args{'verbose'} = 0; }
@ -127,7 +173,7 @@ sub monitor_snapshots {

 	my ($config, $snaps, $snapsbytype, $snapsbypath) = @_;
 	my %datestamp = get_date();
-	my $errorlevel = 0;
+	my $errlevel = 0;
 	my $msg;
 	my @msgs;
 	my @paths;
@ -166,7 +212,7 @@ sub monitor_snapshots {
 			my $dispcrit = displaytime($crit);
 			if ( $elapsed > $crit || $elapsed == -1) {
 				if ($crit > 0) {
-					if (! $config{$section}{'monitor_dont_crit'}) { $errorlevel = 2; }
+					if (! $config{$section}{'monitor_dont_crit'}) { $errlevel = 2; }
 					if ($elapsed == -1) {
 						push @msgs, "CRIT: $path has no $type snapshots at all!";
 					} else {
@ -175,7 +221,7 @@ sub monitor_snapshots {
 				}
 			 } elsif ($elapsed > $warn) {
 				if ($warn > 0) {
-					if (! $config{$section}{'monitor_dont_warn'} && ($errorlevel < 2) ) { $errorlevel = 1; }
+					if (! $config{$section}{'monitor_dont_warn'} && ($errlevel < 2) ) { $errlevel = 1; }
 					push @msgs, "WARN: $path newest $type snapshot is $dispelapsed old (should be < $dispwarn)";
 				}
 			} else {
@ -193,7 +239,7 @@ sub monitor_snapshots {
 	if ($msg eq '') { $msg = "OK: all monitored datasets \($paths\) have fresh snapshots"; }

 	print "$msg\n";
-	exit $errorlevel;
+	exit $errlevel;
 }


@ -262,7 +308,6 @@ sub prune_snapshots {
 	my ($config, $snaps, $snapsbytype, $snapsbypath) = @_;

 	my %datestamp = get_date();
-	my $forcecacheupdate = 0;

 	foreach my $section (keys %config) {
 		if ($section =~ /^template/) { next; }
@ -316,6 +361,25 @@ sub prune_snapshots {
 					if (checklock('sanoid_pruning')) {
 						writelock('sanoid_pruning');
 						foreach my $snap( @prunesnaps ){
+							my $dataset = (split '@', $snap)[0];
+							my $snapname = (split '@', $snap)[1];
+
+							if (! $args{'readonly'} && $config{$dataset}{'pre_pruning_script'}) {
+								$ENV{'SANOID_TARGET'} = $dataset;
+								$ENV{'SANOID_SNAPNAME'} = $snapname;
+								if ($args{'verbose'}) { print "executing pre_pruning_script '".$config{$dataset}{'pre_pruning_script'}."' on dataset '$dataset'\n"; }
+								my $ret = runscript('pre_pruning_script', $dataset);
+
+								delete $ENV{'SANOID_TARGET'};
+								delete $ENV{'SANOID_SNAPNAME'};
+
+								if ($ret != 0) {
+									# warning was already thrown by runscript function
+									# skip pruning if pre snapshot script returns non zero exit code
+									next;
+								}
+							}
+
 							if ($args{'verbose'}) { print "INFO: pruning $snap ... \n"; }
 							if (!$args{'force-prune'} && iszfsbusy($path)) {
 								if ($args{'verbose'}) { print "INFO: deferring pruning of $snap - $path is currently in zfs send or receive.\n"; }
@ -323,16 +387,16 @@ sub prune_snapshots {
 								if (! $args{'readonly'}) {
 									if (system($zfs, "destroy", $snap) == 0) {
 										$pruned{$snap} = 1;
-										my $dataset = (split '@', $snap)[0];
-										my $snapname = (split '@', $snap)[1];
 										if ($config{$dataset}{'pruning_script'}) {
 											$ENV{'SANOID_TARGET'} = $dataset;
 											$ENV{'SANOID_SNAPNAME'} = $snapname;
+											$ENV{'SANOID_SCRIPT'} = 'prune';
 											if ($args{'verbose'}) { print "executing pruning_script '".$config{$dataset}{'pruning_script'}."' on dataset '$dataset'\n"; }
 											my $ret = runscript('pruning_script',$dataset);

 											delete $ENV{'SANOID_TARGET'};
 											delete $ENV{'SANOID_SNAPNAME'};
+											delete $ENV{'SANOID_SCRIPT'};
 										}
 									} else {
 										warn "could not remove $snap : $?";
@ -367,7 +431,7 @@ sub take_snapshots {
 	my %datestamp = get_date();
 	my $forcecacheupdate = 0;

-	my @newsnaps;
+	my %newsnapsgroup;

 	# get utc timestamp of the current day for DST check
 	my $daystartUtc = timelocal(0, 0, 0, $datestamp{'mday'}, ($datestamp{'mon'}-1), $datestamp{'year'});
@ -389,9 +453,9 @@ sub take_snapshots {
 		if ($config{$section}{'process_children_only'}) { next; }

 		my $path = $config{$section}{'path'};
-                my @types = ('yearly','monthly','weekly','daily','hourly','frequently');
+		my @types = ('yearly','monthly','weekly','daily','hourly','frequently');

-                foreach my $type (@types) {
+		foreach my $type (@types) {
 			if ($config{$section}{$type} > 0) {

 				my $newestage; # in seconds
@ -511,55 +575,58 @@ sub take_snapshots {
 				my $maxage = time()-$lastpreferred;

 				if ( $newestage > $maxage ) {
-					# update to most current possible datestamp
-					%datestamp = get_date();
 					# print "we should have had a $type snapshot of $path $maxage seconds ago; most recent is $newestage seconds old.\n";
-
-					my $flags = "";
-					# use zfs (atomic) recursion if specified in config
-					if ($config{$section}{'zfs_recursion'}) {
-						$flags .= "r";
-					}
-					if ($handleDst) {
-						$flags .= "d";
+					if (!exists $newsnapsgroup{$path}) {
+						$newsnapsgroup{$path} = {
+							'recursive' => $config{$section}{'zfs_recursion'},
+							'handleDst' => $handleDst,
+							'datasets' => [$path], # for later atomic grouping, currently only a one element array
+							'types' => []
+						};
 					}

-					if ($flags ne "") {
-						push(@newsnaps, "$path\@autosnap_$datestamp{'sortable'}_$type\@$flags");
-					} else {
-						push(@newsnaps, "$path\@autosnap_$datestamp{'sortable'}_$type");
-					}
+					push(@{$newsnapsgroup{$path}{'types'}}, $type);
 				}
 			}
 		}
 	}

-	if ( (scalar(@newsnaps)) > 0) {
-		foreach my $snap ( @newsnaps ) {
-			my $extraMessage = "";
-			my @split = split '@', $snap, -1;
-			my $recursiveFlag = 0;
-			my $dstHandling = 0;
-			if (scalar(@split) == 3) {
-				my $flags = $split[2];
-				if (index($flags, "r") != -1) {
-					$recursiveFlag = 1;
-					$extraMessage = " (zfs recursive)";
-					chop $snap;
-				}
-				if (index($flags, "d") != -1) {
-					$dstHandling = 1;
-					chop $snap;
-				}
-				chop $snap;
+	if (%newsnapsgroup) {
+		while ((my $path, my $snapData) = each(%newsnapsgroup)) {
+			my $recursiveFlag = $snapData->{recursive};
+			my $dstHandling = $snapData->{handleDst};
+
+			my @datasets = @{$snapData->{datasets}};
+			my $dataset = $datasets[0];
+			my @types = @{$snapData->{types}};
+
+			# same timestamp for all snapshots types (daily, hourly, ...)
+			my %datestamp = get_date();
+			my @snapshots;
+
+			foreach my $type (@types) {
+				my $snapname = "autosnap_$datestamp{'sortable'}_$type";
+				push(@snapshots, $snapname);
 			}
-			my $dataset = $split[0];
-			my $snapname = $split[1];
+
+			my $datasetString = join(",", @datasets);
+			my $typeString = join(",", @types);
+			my $snapshotString = join(",", @snapshots);
+
+			my $extraMessage = "";
+			if ($recursiveFlag) {
+				$extraMessage = " (zfs recursive)";
+			}
+
 			my $presnapshotfailure = 0;
 			my $ret = 0;
 			if ($config{$dataset}{'pre_snapshot_script'}) {
 				$ENV{'SANOID_TARGET'} = $dataset;
-				$ENV{'SANOID_SNAPNAME'} = $snapname;
+				$ENV{'SANOID_TARGETS'} = $datasetString;
+				$ENV{'SANOID_SNAPNAME'} = $snapshots[0];
+				$ENV{'SANOID_SNAPNAMES'} = $snapshotString;
+				$ENV{'SANOID_TYPES'} = $typeString;
+				$ENV{'SANOID_SCRIPT'} = 'pre';
 				if ($args{'verbose'}) { print "executing pre_snapshot_script '".$config{$dataset}{'pre_snapshot_script'}."' on dataset '$dataset'\n"; }

 				if (!$args{'readonly'}) {
@ -567,7 +634,11 @@ sub take_snapshots {
 				}

 				delete $ENV{'SANOID_TARGET'};
+				delete $ENV{'SANOID_TARGETS'};
 				delete $ENV{'SANOID_SNAPNAME'};
+				delete $ENV{'SANOID_SNAPNAMES'};
+				delete $ENV{'SANOID_TYPES'};
+				delete $ENV{'SANOID_SCRIPT'};

 				if ($ret != 0) {
 					# warning was already thrown by runscript function
@ -575,47 +646,58 @@ sub take_snapshots {
 					$presnapshotfailure = 1;
 				}
 			}
-			if ($args{'verbose'}) { print "taking snapshot $snap$extraMessage\n"; }
-			if (!$args{'readonly'}) {
-				my $stderr;
-				my $exit;
-				($stderr, $exit) = tee_stderr {
-					if ($recursiveFlag) {
-						system($zfs, "snapshot", "-r", "$snap");
-					} else {
-						system($zfs, "snapshot", "$snap");
-					}
-				};

-				$exit == 0 or do {
-					if ($dstHandling) {
-						if ($stderr =~ /already exists/) {
-							$exit = 0;
-							$snap =~ s/_([a-z]+)$/dst_$1/g;
-							if ($args{'verbose'}) { print "taking dst snapshot $snap$extraMessage\n"; }
-							if ($recursiveFlag) {
-								system($zfs, "snapshot", "-r", "$snap") == 0
-									or warn "CRITICAL ERROR: $zfs snapshot -r $snap failed, $?";
-							} else {
-								system($zfs, "snapshot", "$snap") == 0
-									or warn "CRITICAL ERROR: $zfs snapshot $snap failed, $?";
+			foreach my $snap (@snapshots) {
+				$snap = "$dataset\@$snap";
+				if ($args{'verbose'}) { print "taking snapshot $snap$extraMessage\n"; }
+
+				if (!$args{'readonly'}) {
+					my $stderr;
+					my $exit;
+					($stderr, $exit) = tee_stderr {
+						if ($recursiveFlag) {
+							system($zfs, "snapshot", "-r", "$snap");
+						} else {
+							system($zfs, "snapshot", "$snap");
+						}
+					};
+
+					$exit == 0 or do {
+						if ($dstHandling) {
+							if ($stderr =~ /already exists/) {
+								$exit = 0;
+								$snap =~ s/_([a-z]+)$/dst_$1/g;
+								if ($args{'verbose'}) { print "taking dst snapshot $snap$extraMessage\n"; }
+								if ($recursiveFlag) {
+									system($zfs, "snapshot", "-r", "$snap") == 0
+										or warn "CRITICAL ERROR: $zfs snapshot -r $snap failed, $?";
+								} else {
+									system($zfs, "snapshot", "$snap") == 0
+										or warn "CRITICAL ERROR: $zfs snapshot $snap failed, $?";
+								}
 							}
 						}
-					}
-				};
+					};

-				$exit == 0 or do {
-					if ($recursiveFlag) {
-						warn "CRITICAL ERROR: $zfs snapshot -r $snap failed, $?";
-					} else {
-						warn "CRITICAL ERROR: $zfs snapshot $snap failed, $?";
-					}
-				};
+					$exit == 0 or do {
+						if ($recursiveFlag) {
+							warn "CRITICAL ERROR: $zfs snapshot -r $snap failed, $?";
+						} else {
+							warn "CRITICAL ERROR: $zfs snapshot $snap failed, $?";
+						}
+					};
+				}
 			}
+
 			if ($config{$dataset}{'post_snapshot_script'}) {
 				if (!$presnapshotfailure or $config{$dataset}{'force_post_snapshot_script'}) {
 					$ENV{'SANOID_TARGET'} = $dataset;
-					$ENV{'SANOID_SNAPNAME'} = $snapname;
+					$ENV{'SANOID_TARGETS'} = $datasetString;
+					$ENV{'SANOID_SNAPNAME'} = $snapshots[0];
+					$ENV{'SANOID_SNAPNAMES'} = $snapshotString;
+					$ENV{'SANOID_TYPES'} = $typeString;
+					$ENV{'SANOID_SCRIPT'} = 'post';
+					$ENV{'SANOID_PRE_FAILURE'} = $presnapshotfailure;
 					if ($args{'verbose'}) { print "executing post_snapshot_script '".$config{$dataset}{'post_snapshot_script'}."' on dataset '$dataset'\n"; }

 					if (!$args{'readonly'}) {
@ -623,7 +705,12 @@ sub take_snapshots {
 					}

 					delete $ENV{'SANOID_TARGET'};
+					delete $ENV{'SANOID_TARGETS'};
 					delete $ENV{'SANOID_SNAPNAME'};
+					delete $ENV{'SANOID_SNAPNAMES'};
+					delete $ENV{'SANOID_TYPES'};
+					delete $ENV{'SANOID_SCRIPT'};
+					delete $ENV{'SANOID_PRE_FAILURE'};
 				}
 			}
 		}
@ -771,7 +858,7 @@ sub getsnaps {
 		if (checklock('sanoid_cacheupdate')) {
 			writelock('sanoid_cacheupdate');
 			if ($args{'verbose'}) {
-				if ($args{'force-update'}) {
+				if ($forcecacheupdate) {
 					print "INFO: cache forcibly expired - updating from zfs list.\n";
 				} else {
 					print "INFO: cache expired - updating from zfs list.\n";
@ -781,9 +868,10 @@ sub getsnaps {
 			@rawsnaps = <FH>;
 			close FH;

-			open FH, "> $cache" or die 'Could not write to $cache!\n';
+			open FH, "> $cache.tmp" or die 'Could not write to $cache.tmp!\n';
 			print FH @rawsnaps;
 			close FH;
+			rename("$cache.tmp", "$cache") or die 'Could not rename to $cache!\n';
 			removelock('sanoid_cacheupdate');
 		} else {
 			if ($args{'verbose'}) { print "INFO: deferring cache update - valid cache update lock held by another sanoid process.\n"; }
@ -846,6 +934,20 @@ sub init {
 		die "FATAL: you're using sanoid.defaults.conf v$defaults_version, this version of sanoid requires a minimum sanoid.defaults.conf v$MINIMUM_DEFAULTS_VERSION";
 	}

+	my @updatedatasets;
+
+	# load dataset cache if valid
+	if (!$args{'force-update'} && -f $cachedatasetspath) {
+		my ($dev, $ino, $mode, $nlink, $uid, $gid, $rdev, $size, $atime, $mtime, $ctime, $blksize, $blocks) = stat($cachedatasetspath);
+
+		if ((time() - $mtime) <= $cacheTTL) {
+			if ($args{'debug'}) { print "DEBUG: dataset cache not expired (" . (time() - $mtime) . " seconds old with TTL of $cacheTTL): pulling dataset list from cache.\n"; }
+			open FH, "< $cachedatasetspath";
+			@cachedatasets = <FH>;
+			close FH;
+		}
+	}
+
 	foreach my $section (keys %ini) {

 		# first up - die with honor if unknown parameters are set in any modules or templates by the user.
@ -853,6 +955,15 @@ sub init {
 			if (! defined ($defaults{'template_default'}{$key})) {
 				die "FATAL ERROR: I don't understand the setting $key you've set in \[$section\] in $conf_file.\n";
 			}
+
+			# in case of duplicate lines we will end up with an array of all values
+			my $value = $ini{$section}{$key};
+			if (ref($value) eq 'ARRAY') {
+				warn "duplicate key '$key' in section '$section', using the value from the first occurence and ignoring the others.\n";
+				$ini{$section}{$key} = $value->[0];
+			}
+			# trim
+			$ini{$section}{$key} =~ s/^\s+|\s+$//g;
 		}

 		if ($section =~ /^template_/) { next; } # don't process templates directly
@ -861,7 +972,7 @@ sub init {
 		# for sections directly when they've already been defined recursively, without starting them over from scratch.
 		if (! defined ($config{$section}{'initialized'})) {
 			if ($args{'debug'}) { print "DEBUG: initializing \$config\{$section\} with default values from $default_conf_file.\n"; }
-			# set default values from %defaults, which can then be overriden by template
+			# set default values from %defaults, which can then be overridden by template
 			# and/or local settings within the module.
 			foreach my $key (keys %{$defaults{'template_default'}}) {
 				if (! ($key =~ /template|recursive|children_only/)) {
@ -926,6 +1037,10 @@ sub init {
 			$config{$section}{'path'} = $section;
 		}

+		if (! @cachedatasets) {
+			push (@updatedatasets, "$config{$section}{'path'}\n");
+		}
+
 		# how 'bout some recursion? =)
 		if ($config{$section}{'zfs_recursion'} && $config{$section}{'zfs_recursion'} == 1 && $config{$section}{'autosnap'} == 1) {
 			warn "ignored autosnap configuration for '$section' because it's part of a zfs recursion.\n";
@ -943,6 +1058,10 @@ sub init {

 			@datasets = getchilddatasets($config{$section}{'path'});
 			DATASETS: foreach my $dataset(@datasets) {
+				if (! @cachedatasets) {
+					push (@updatedatasets, $dataset);
+				}
+
 				chomp $dataset;

 				if ($zfsRecursive) {
@ -974,9 +1093,27 @@ sub init {
 				$config{$dataset}{'initialized'} = 1;
 			}
 		}
+	}

-
-
+	# update dataset cache if it was unused
+	if (! @cachedatasets) {
+		if (checklock('sanoid_cachedatasetupdate')) {
+			writelock('sanoid_cachedatasetupdate');
+			if ($args{'verbose'}) {
+				if ($args{'force-update'}) {
+					print "INFO: dataset cache forcibly expired - updating from zfs list.\n";
+				} else {
+					print "INFO: dataset cache expired - updating from zfs list.\n";
+				}
+			}
+			open FH, "> $cachedatasetspath.tmp" or die 'Could not write to $cachedatasetspath.tmp!\n';
+			print FH @updatedatasets;
+			close FH;
+			rename("$cachedatasetspath.tmp", "$cachedatasetspath") or die 'Could not rename to $cachedatasetspath!\n';
+			removelock('sanoid_cachedatasetupdate');
+		} else {
+			if ($args{'verbose'}) { print "INFO: deferring dataset cache update - valid cache update lock held by another sanoid process.\n"; }
+		}
 	}

 	return %config;
@ -1109,7 +1246,7 @@ sub check_zpool() {
 		}
 	}

-	# Tony: Debuging
+	# Tony: Debugging
 	# print "Size: $size \t Used: $used \t Avai: $avail \t Cap: $cap \t Health: $health\n";

 	close(STAT);
@ -1211,7 +1348,7 @@ sub check_zpool() {
 			## no display for verbose level 1
 			next if ($verbose==1);
 			## don't display working devices for verbose level 2
-			if ($verbose==2 && ($state eq "OK" || $sta eq "ONLINE" || $sta eq "AVAIL" || $sta eq "INUSE")) {
+			if ($verbose==2 && ($state eq "OK" || $sta eq "ONLINE" || $sta eq "AVAIL")) {
 				# check for io/checksum errors

 				my @vdeverr = ();
@ -1526,6 +1663,30 @@ sub getchilddatasets {
 	my $fs = shift;
 	my $mysudocmd = '';

+	# use dataset cache if available
+	if (@cachedatasets) {
+		my $foundparent = 0;
+		my @cachechildren = ();
+		foreach my $dataset (@cachedatasets) {
+			chomp $dataset;
+			my $ret = rindex $dataset, "${fs}/", 0;
+			if ($ret == 0) {
+				push (@cachechildren, $dataset);
+			} else {
+				if ($dataset eq $fs) {
+					$foundparent = 1;
+				}
+			}
+		}
+
+		# sanity check
+		if ($foundparent) {
+			return @cachechildren;
+		}
+
+		# fallback if cache misses items for whatever reason
+	}
+
 	my $getchildrencmd = "$mysudocmd $zfs list -o name -t filesystem,volume -Hr $fs |";
 	 if ($args{'debug'}) { print "DEBUG: getting list of child datasets on $fs using $getchildrencmd...\n"; }
 	open FH, $getchildrencmd;
@ -1572,16 +1733,17 @@ sub removecachedsnapshots {
 	my @rawsnaps = <FH>;
 	close FH;

-	open FH, "> $cache" or die 'Could not write to $cache!\n';
+	open FH, "> $cache.tmp" or die 'Could not write to $cache.tmp!\n';
 	foreach my $snapline ( @rawsnaps ) {
 		my @columns = split("\t", $snapline);
 		my $snap = $columns[0];
 		print FH $snapline unless ( exists($pruned{$snap}) );
 	}
 	close FH;
+	rename("$cache.tmp", "$cache") or die 'Could not rename to $cache!\n';

 	removelock('sanoid_cacheupdate');
-	%snaps = getsnaps(\%config,$cacheTTL,$forcecacheupdate);
+	%snaps = getsnaps(\%config,$cacheTTL,0);

 	# clear hash
 	undef %pruned;
--- a/sanoid.conf
+++ b/sanoid.conf
@ -3,30 +3,39 @@
 # It should go in /etc/sanoid.       #
 ######################################

-# name your backup modules with the path to their ZFS dataset - no leading slash.
-[zpoolname/datasetname]
-	# pick one or more templates - they're defined (and editable) below. Comma separated, processed in order.
-	# in this example, template_demo's daily value overrides template_production's daily value.
-	use_template = production,demo
+## name your backup modules with the path to their ZFS dataset - no leading slash.
+#[zpoolname/datasetname]
+#	# pick one or more templates - they're defined (and editable) below. Comma separated, processed in order.
+#	# in this example, template_demo's daily value overrides template_production's daily value.
+#	use_template = production,demo
+#
+#	# if you want to, you can override settings in the template directly inside module definitions like this.
+#	# in this example, we override the template to only keep 12 hourly and 1 monthly snapshot for this dataset.
+#	hourly = 12
+#	monthly = 1
+#
+## you can also handle datasets recursively.
+#[zpoolname/parent]
+#	use_template = production
+#	recursive = yes
+#	# if you want sanoid to manage the child datasets but leave this one alone, set process_children_only.
+#	process_children_only = yes
+#
+## you can selectively override settings for child datasets which already fall under a recursive definition.
+#[zpoolname/parent/child]
+#	# child datasets already initialized won't be wiped out, so if you use a new template, it will
+#	# only override the values already set by the parent template, not replace it completely.
+#	use_template = demo

-	# if you want to, you can override settings in the template directly inside module definitions like this.
-	# in this example, we override the template to only keep 12 hourly and 1 monthly snapshot for this dataset.
-	hourly = 12
-	monthly = 1

-# you can also handle datasets recursively.
-[zpoolname/parent]
+# you can also handle datasets recursively in an atomic way without the possibility to override settings for child datasets.
+[zpoolname/parent2]
 	use_template = production
-	recursive = yes
-	# if you want sanoid to manage the child datasets but leave this one alone, set process_children_only.
-	process_children_only = yes
-
-# you can selectively override settings for child datasets which already fall under a recursive definition.
-[zpoolname/parent/child]
-	# child datasets already initialized won't be wiped out, so if you use a new template, it will
-	# only override the values already set by the parent template, not replace it completely.
-	use_template = demo
-
+	# there are two options for recursive: zfs or yes
+	# * zfs - taken a zfs snapshot with the '-r' flag; zfs will recursively take a snapshot of the whole
+	#         dataset tree which is consistent.
+	# * yes - the snapshots will be taken one-at-time through the sanoid code; not necessarily consistent.
+	recursive = zfs



@ -91,12 +100,14 @@
 	daily_crit = 4d

 [template_scripts]
-	### dataset and snapshot name will be supplied as environment variables
-	### for all pre/post/prune scripts ($SANOID_TARGET, $SANOID_SNAPNAME)
+	### information about the snapshot will be supplied as environment variables,
+	### see the README.md file for details about what is passed when.
 	### run script before snapshot
 	pre_snapshot_script = /path/to/script.sh
 	### run script after snapshot
 	post_snapshot_script = /path/to/script.sh
+	### run script before pruning snapshot
+	pre_pruning_script = /path/to/script.sh
 	### run script after pruning snapshot
 	pruning_script = /path/to/script.sh
 	### don't take an inconsistent snapshot (skip if pre script fails)
--- a/sanoid.defaults.conf
+++ b/sanoid.defaults.conf
@ -19,8 +19,10 @@ use_template =
 process_children_only =
 skip_children =

+# See "Sanoid script hooks" in README.md for information about scripts.
 pre_snapshot_script =
 post_snapshot_script =
+pre_pruning_script =
 pruning_script =
 script_timeout = 5
 no_inconsistent_snapshot =
@ -108,5 +110,6 @@ yearly_warn = 0
 yearly_crit = 0

 # default limits for capacity checks (if set to 0, limit will not be checked)
+# for overriding these values one needs to specify them in a root pool section! ([tank]\n ...)
 capacity_warn = 80
 capacity_crit = 95
--- a/1487
+++ b/1487
--- a/tests/common/lib.sh
+++ b/tests/common/lib.sh
@ -34,7 +34,7 @@ function checkEnvironment {
        echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
        echo "you should be running this test in a"
        echo "dedicated vm, as it will mess with your system!"
-        echo "Are you sure you wan't to continue? (y)"
+        echo "Are you sure you want to continue? (y)"
        echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
        set -x

--- a/tests/run-tests.sh
+++ b/tests/run-tests.sh
@ -2,7 +2,7 @@

 # run's all the available tests

-for test in */; do
+for test in $(find . -mindepth 1 -maxdepth 1 -type d -printf "%P\n" | sort -g); do
    if [ ! -x "${test}/run.sh" ]; then
        continue
    fi
@ -17,8 +17,11 @@ for test in */; do
    cd "${test}"
    echo -n y | bash run.sh > "${LOGFILE}" 2>&1

-    if [ $? -eq 0 ]; then
+    ret=$?
+    if [ $ret -eq 0 ]; then
        echo "[PASS]"
+    elif [ $ret -eq 130 ]; then
+        echo "[SKIPPED]"
    else
        echo "[FAILED] (see ${LOGFILE})"
    fi
--- a/tests/syncoid/001_bookmark_replication_intermediate/run.sh
+++ b/tests/syncoid/001_bookmark_replication_intermediate/run.sh
--- a/tests/syncoid/002_bookmark_replication_no_intermediate/run.sh
+++ b/tests/syncoid/002_bookmark_replication_no_intermediate/run.sh
--- a/tests/syncoid/003_force_delete/run.sh
+++ b/tests/syncoid/003_force_delete/run.sh
--- a/tests/syncoid/004_bookmark_replication_edge_case/run.sh
+++ b/tests/syncoid/004_bookmark_replication_edge_case/run.sh
--- a/tests/syncoid/005_reset_resume_state/run.sh
+++ b/tests/syncoid/005_reset_resume_state/run.sh
@ -28,6 +28,8 @@ zfs create -o mountpoint="${MOUNT_TARGET}" "${POOL_NAME}"/src

 dd if=/dev/urandom of="${MOUNT_TARGET}"/big_file bs=1M count=200

+sleep 1
+
 ../../../syncoid --debug --compress=none --source-bwlimit=2m "${POOL_NAME}"/src "${POOL_NAME}"/dst &
 syncoid_pid=$!
 sleep 5
@ -45,6 +47,9 @@ wait
 sleep 1

 ../../../syncoid --debug --compress=none --no-resume "${POOL_NAME}"/src "${POOL_NAME}"/dst | grep "reset partial receive state of syncoid"
+
+sleep 1
+
 ../../../syncoid --debug --compress=none "${POOL_NAME}"/src "${POOL_NAME}"/dst

 exit $?
--- a/tests/syncoid/006_reset_resume_state2/run.sh
+++ b/tests/syncoid/006_reset_resume_state2/run.sh
@ -28,6 +28,8 @@ zfs create -o mountpoint="${MOUNT_TARGET}" "${POOL_NAME}"/src

 dd if=/dev/urandom of="${MOUNT_TARGET}"/big_file bs=1M count=200

+sleep 1
+
 zfs snapshot "${POOL_NAME}"/src@big
 ../../../syncoid --debug --no-sync-snap --compress=none --source-bwlimit=2m "${POOL_NAME}"/src "${POOL_NAME}"/dst &
 syncoid_pid=$!
@ -47,6 +49,9 @@ sleep 1

 zfs destroy "${POOL_NAME}"/src@big
 ../../../syncoid --debug --compress=none "${POOL_NAME}"/src "${POOL_NAME}"/dst # | grep "reset partial receive state of syncoid"
+
+sleep 1
+
 ../../../syncoid --debug --compress=none "${POOL_NAME}"/src "${POOL_NAME}"/dst

 exit $?
--- a/tests/syncoid/007_preserve_recordsize/run.sh
+++ b/tests/syncoid/007_preserve_recordsize/run.sh
@ -0,0 +1,48 @@
+#!/bin/bash
+
+# test preserving the recordsize from the src filesystem to the target one
+
+set -x
+set -e
+
+. ../../common/lib.sh
+
+POOL_IMAGE="/tmp/syncoid-test-7.zpool"
+MOUNT_TARGET="/tmp/syncoid-test-7.mount"
+POOL_SIZE="1000M"
+POOL_NAME="syncoid-test-7"
+
+truncate -s "${POOL_SIZE}" "${POOL_IMAGE}"
+
+zpool create -m none -f "${POOL_NAME}" "${POOL_IMAGE}"
+
+function cleanUp {
+  zpool export "${POOL_NAME}"
+}
+
+# export pool in any case
+trap cleanUp EXIT
+
+zfs create "${POOL_NAME}"/src
+zfs create -V 100M -o volblocksize=4k "${POOL_NAME}"/src/zvol4
+zfs create -V 100M -o volblocksize=16k "${POOL_NAME}"/src/zvol16
+zfs create -V 100M -o volblocksize=64k "${POOL_NAME}"/src/zvol64
+zfs create -o recordsize=16k "${POOL_NAME}"/src/16
+zfs create -o recordsize=32k "${POOL_NAME}"/src/32
+zfs create -o recordsize=128k "${POOL_NAME}"/src/128
+../../../syncoid --preserve-recordsize --recursive --debug --compress=none "${POOL_NAME}"/src "${POOL_NAME}"/dst
+
+zfs get -t filesystem -r recordsize "${POOL_NAME}"/dst
+zfs get -t volume -r volblocksize "${POOL_NAME}"/dst
+
+if [ "$(zfs get -H -o value -t filesystem recordsize "${POOL_NAME}"/dst/16)" != "16K" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem recordsize "${POOL_NAME}"/dst/32)" != "32K" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem recordsize "${POOL_NAME}"/dst/128)" != "128K" ]; then
+	exit 1
+fi
--- a/tests/syncoid/008_force_delete_snapshot/run.sh
+++ b/tests/syncoid/008_force_delete_snapshot/run.sh
@ -0,0 +1,48 @@
+#!/bin/bash
+
+# test replication with deletion of conflicting snapshot on target
+
+set -x
+set -e
+
+. ../../common/lib.sh
+
+POOL_IMAGE="/tmp/syncoid-test-8.zpool"
+POOL_SIZE="200M"
+POOL_NAME="syncoid-test-8"
+TARGET_CHECKSUM="ee439200c9fa54fc33ce301ef64d4240a6c5587766bfeb651c5cf358e11ec89d  -"
+
+truncate -s "${POOL_SIZE}" "${POOL_IMAGE}"
+
+zpool create -m none -f "${POOL_NAME}" "${POOL_IMAGE}"
+
+function cleanUp {
+  zpool export "${POOL_NAME}"
+}
+
+# export pool in any case
+trap cleanUp EXIT
+
+zfs create "${POOL_NAME}"/src
+zfs snapshot "${POOL_NAME}"/src@duplicate
+
+# initial replication
+../../../syncoid -r --debug --compress=none "${POOL_NAME}"/src "${POOL_NAME}"/dst
+# recreate snapshot with the same name on src
+zfs destroy "${POOL_NAME}"/src@duplicate
+zfs snapshot "${POOL_NAME}"/src@duplicate
+sleep 1
+../../../syncoid -r --force-delete --debug --compress=none "${POOL_NAME}"/src "${POOL_NAME}"/dst || exit 1
+
+# verify
+output1=$(zfs list -t snapshot -r -H -o guid,name "${POOL_NAME}"/src | sed 's/@syncoid_.*$'/@syncoid_/)
+checksum1=$(echo "${output1}" | shasum -a 256)
+
+output2=$(zfs list -t snapshot -r -H -o guid,name "${POOL_NAME}"/dst | sed 's/@syncoid_.*$'/@syncoid_/ | sed 's/dst/src/')
+checksum2=$(echo "${output2}" | shasum -a 256)
+
+if [ "${checksum1}" != "${checksum2}" ]; then
+	exit 1
+fi
+
+exit 0
--- a/tests/syncoid/009_preserve_properties/run.sh
+++ b/tests/syncoid/009_preserve_properties/run.sh
@ -0,0 +1,71 @@
+#!/bin/bash
+
+# test preserving locally set properties from the src dataset to the target one
+
+set -x
+set -e
+
+. ../../common/lib.sh
+
+POOL_IMAGE="/tmp/syncoid-test-9.zpool"
+MOUNT_TARGET="/tmp/syncoid-test-9.mount"
+POOL_SIZE="1000M"
+POOL_NAME="syncoid-test-9"
+
+truncate -s "${POOL_SIZE}" "${POOL_IMAGE}"
+
+zpool create -m none -f "${POOL_NAME}" "${POOL_IMAGE}"
+
+function cleanUp {
+  zpool export "${POOL_NAME}"
+}
+
+# export pool in any case
+trap cleanUp EXIT
+
+zfs create -o recordsize=16k -o xattr=on -o mountpoint=none -o primarycache=none "${POOL_NAME}"/src
+zfs create -V 100M -o volblocksize=8k "${POOL_NAME}"/src/zvol8
+zfs create -V 100M -o volblocksize=16k -o primarycache=all "${POOL_NAME}"/src/zvol16
+zfs create -V 100M -o volblocksize=64k "${POOL_NAME}"/src/zvol64
+zfs create -o recordsize=16k -o primarycache=none "${POOL_NAME}"/src/16
+zfs create -o recordsize=32k -o acltype=posixacl "${POOL_NAME}"/src/32
+zfs set 'net.openoid:var-name'='with whitespace and !"§$%&/()= symbols' "${POOL_NAME}"/src/32
+
+../../../syncoid --preserve-properties --recursive --debug --compress=none "${POOL_NAME}"/src "${POOL_NAME}"/dst
+
+
+if [ "$(zfs get -H -o value -t filesystem recordsize "${POOL_NAME}"/dst)" != "16K" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem mountpoint "${POOL_NAME}"/dst)" != "none" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem xattr "${POOL_NAME}"/dst)" != "on" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem primarycache "${POOL_NAME}"/dst)" != "none" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem recordsize "${POOL_NAME}"/dst/16)" != "16K" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem primarycache "${POOL_NAME}"/dst/16)" != "none" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem recordsize "${POOL_NAME}"/dst/32)" != "32K" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem acltype "${POOL_NAME}"/dst/32)" != "posix" ]; then
+	exit 1
+fi
+
+if [ "$(zfs get -H -o value -t filesystem 'net.openoid:var-name' "${POOL_NAME}"/dst/32)" != "with whitespace and !\"§$%&/()= symbols" ]; then
+	exit 1
+fi
--- a/tests/syncoid/010_filter_snaps/run.sh
+++ b/tests/syncoid/010_filter_snaps/run.sh
@ -0,0 +1,142 @@
+#!/bin/bash
+
+# test filtering snapshot names using --include-snaps and --exclude-snaps
+
+set -x
+set -e
+
+. ../../common/lib.sh
+
+POOL_IMAGE="/tmp/syncoid-test-10.zpool"
+MOUNT_TARGET="/tmp/syncoid-test-10.mount"
+POOL_SIZE="100M"
+POOL_NAME="syncoid-test-10"
+
+truncate -s "${POOL_SIZE}" "${POOL_IMAGE}"
+
+zpool create -m none -f "${POOL_NAME}" "${POOL_IMAGE}"
+
+#####
+# Create source snapshots and destroy the destination snaps and dataset.
+#####
+function setup_snaps {
+  # create intermediate snapshots
+  # sleep is needed so creation time can be used for proper sorting
+  sleep 1
+  zfs snapshot "${POOL_NAME}"/src@monthly1
+  sleep 1
+  zfs snapshot "${POOL_NAME}"/src@daily1
+  sleep 1
+  zfs snapshot "${POOL_NAME}"/src@daily2
+  sleep 1
+  zfs snapshot "${POOL_NAME}"/src@hourly1
+  sleep 1
+  zfs snapshot "${POOL_NAME}"/src@hourly2
+  sleep 1
+  zfs snapshot "${POOL_NAME}"/src@daily3
+  sleep 1
+  zfs snapshot "${POOL_NAME}"/src@hourly3
+  sleep 1
+  zfs snapshot "${POOL_NAME}"/src@hourly4
+}
+
+#####
+# Remove the destination snapshots and dataset so that each test starts with a
+# blank slate.
+#####
+function clean_snaps {
+  zfs destroy "${POOL_NAME}"/dst@%
+  zfs destroy "${POOL_NAME}"/dst
+}
+
+#####
+# Verify that the correct set of snapshots is present on the destination.
+#####
+function verify_checksum {
+  zfs list -r -t snap "${POOL_NAME}"
+
+  checksum=$(zfs list -t snap -r -H -o name "${POOL_NAME}" | sed 's/@syncoid_.*/@syncoid_/' | shasum -a 256)
+
+  echo "Expected checksum: $1"
+  echo "Actual checksum: $checksum"
+  return $( [[ "$checksum" == "$1" ]] )
+}
+
+function cleanUp {
+  zpool export "${POOL_NAME}"
+}
+
+# export pool in any case
+trap cleanUp EXIT
+
+zfs create "${POOL_NAME}"/src
+setup_snaps
+
+#####
+# TEST 1
+#
+# --exclude-snaps is provided and --no-stream is omitted.  Hourly snaps should
+# be missing from the destination, and all other intermediate snaps should be
+# present.
+#####
+
+../../../syncoid --debug --compress=none --no-sync-snap --exclude-snaps='hourly' "${POOL_NAME}"/src "${POOL_NAME}"/dst
+verify_checksum '494b6860415607f1d670e4106a10e1316924ba6cd31b4ddacffe0ad6d30a6339  -'
+clean_snaps
+
+#####
+# TEST 2
+#
+# --exclude-snaps and --no-stream are provided.  Only the daily3 snap should be
+# present on the destination.
+#####
+
+../../../syncoid --debug --compress=none --no-sync-snap --exclude-snaps='hourly' --no-stream "${POOL_NAME}"/src "${POOL_NAME}"/dst
+verify_checksum '0a5072f42180d231cfdd678682972fbbb689140b7f3e996b3c348b7e78d67ea2  -'
+clean_snaps
+
+#####
+# TEST 3
+#
+# --include-snaps is provided and --no-stream is omitted.  Hourly snaps should
+# be present on the destination, and all other snaps should be missing
+#####
+
+../../../syncoid --debug --compress=none --no-sync-snap --include-snaps='hourly' "${POOL_NAME}"/src "${POOL_NAME}"/dst
+verify_checksum 'd32862be4c71c6cde846322a7d006fd5e8edbd3520d3c7b73953492946debb7f  -'
+clean_snaps
+
+#####
+# TEST 4
+#
+# --include-snaps and --no-stream are provided.  Only the hourly4 snap should
+# be present on the destination.
+#####
+
+../../../syncoid --debug --compress=none --no-sync-snap --include-snaps='hourly' --no-stream "${POOL_NAME}"/src "${POOL_NAME}"/dst
+verify_checksum '81ef1a8298006a7ed856430bb7e05e8b85bbff530ca9dd7831f1da782f8aa4c7  -'
+clean_snaps
+
+#####
+# TEST 5
+#
+# --include-snaps='hourly' and --exclude-snaps='3' are both provided. The
+# hourly snaps should be present on the destination except for hourly3; daily
+# and monthly snaps should be missing.
+#####
+
+../../../syncoid --debug --compress=none --no-sync-snap --include-snaps='hourly' --exclude-snaps='3' "${POOL_NAME}"/src "${POOL_NAME}"/dst
+verify_checksum '5a9dd92b7d4b8760a1fcad03be843da4f43b915c64caffc1700c0d59a1581239  -'
+clean_snaps
+
+#####
+# TEST 6
+#
+# --exclude-snaps='syncoid' and --no-stream are provided, and --no-sync-snap is
+# omitted. The sync snap should be created on the source but not sent to the
+# destination; only hourly4 should be sent.
+#####
+
+../../../syncoid --debug --compress=none --no-stream --exclude-snaps='syncoid' "${POOL_NAME}"/src "${POOL_NAME}"/dst
+verify_checksum '9394fdac44ec72764a4673202552599684c83530a2a724dae5b411aaea082b02  -'
+clean_snaps
--- a/tests/syncoid/011_sync_out-of-order_snapshots/run.sh
+++ b/tests/syncoid/011_sync_out-of-order_snapshots/run.sh
@ -0,0 +1,50 @@
+#!/bin/bash
+
+# test verifying snapshots with out-of-order snapshot creation datetimes
+
+set -x
+set -e
+
+. ../../common/lib.sh
+
+if [ -z "$ALLOW_INVASIVE_TESTS" ]; then
+  exit 130
+fi
+
+POOL_IMAGE="/tmp/syncoid-test-11.zpool"
+POOL_SIZE="64M"
+POOL_NAME="syncoid-test-11"
+
+truncate -s "${POOL_SIZE}" "${POOL_IMAGE}"
+
+zpool create -m none -f "${POOL_NAME}" "${POOL_IMAGE}"
+
+function cleanUp {
+  zpool export "${POOL_NAME}"
+  rm -f "${POOL_IMAGE}"
+}
+
+# export pool and remove the image in any case
+trap cleanUp EXIT
+
+zfs create "${POOL_NAME}"/before
+zfs snapshot "${POOL_NAME}"/before@this-snapshot-should-make-it-into-the-after-dataset
+
+disableTimeSync
+setdate 1155533696
+zfs snapshot "${POOL_NAME}"/before@oldest-snapshot
+
+zfs snapshot "${POOL_NAME}"/before@another-snapshot-does-not-matter
+../../../syncoid --sendoptions="Lec" "${POOL_NAME}"/before "${POOL_NAME}"/after
+
+# verify
+saveSnapshotList "${POOL_NAME}" "snapshot-list.txt"
+
+grep "${POOL_NAME}/before@this-snapshot-should-make-it-into-the-after-dataset" "snapshot-list.txt" || exit $?
+grep "${POOL_NAME}/after@this-snapshot-should-make-it-into-the-after-dataset" "snapshot-list.txt" || exit $?
+grep "${POOL_NAME}/before@oldest-snapshot" "snapshot-list.txt" || exit $?
+grep "${POOL_NAME}/after@oldest-snapshot" "snapshot-list.txt" || exit $?
+grep "${POOL_NAME}/before@another-snapshot-does-not-matter" "snapshot-list.txt" || exit $?
+grep "${POOL_NAME}/after@another-snapshot-does-not-matter" "snapshot-list.txt" || exit $?
+
+exit 0
--- a/tests/syncoid/run-tests.sh
+++ b/tests/syncoid/run-tests.sh
@ -2,7 +2,7 @@

 # run's all the available tests

-for test in */; do
+for test in $(find . -mindepth 1 -maxdepth 1 -type d -printf "%P\n" | sort -g); do
    if [ ! -x "${test}/run.sh" ]; then
        continue
    fi
@ -17,8 +17,11 @@ for test in */; do
    cd "${test}"
    echo | bash run.sh > "${LOGFILE}" 2>&1

-    if [ $? -eq 0 ]; then
+    ret=$?
+    if [ $ret -eq 0 ]; then
        echo "[PASS]"
+    elif [ $ret -eq 130 ]; then
+        echo "[SKIPPED]"
    else
        echo "[FAILED] (see ${LOGFILE})"
    fi
				`@ -0,0 +1 @@`
				`Any and all contributions made to this project must be compatible with the project's own GPLv3 license.`
 @ -1 +1 @@
 .0.3
 .2.0
				`@ -1 +0,0 @@`
				`cf0ec23c310d2f9416ebabe48f5edb73 sanoid-1.4.18.tar.gz`