mirror of https://github.com/jimsalterjrs/sanoid
Compare commits
6 Commits
17337361d5
...
8791076a37
| Author | SHA1 | Date |
|---|---|---|
crpb
|
8791076a37 | |
Jim Salter
|
6beef5fee6 | |
cb
|
ef178f8cd8 | |
|
cb
|
70f8bff190 | |
|
cb
|
047b0dcdda | |
|
cb
|
4ab023b63d |
|
|
@ -0,0 +1 @@
|
|||
/debian
|
||||
|
|
@ -26,9 +26,10 @@ apt install debhelper libcapture-tiny-perl libconfig-inifiles-perl pv lzop mbuff
|
|||
|
||||
```
|
||||
|
||||
Clone this repo, build the debian package and install it (alternatively you can skip the package and do it manually like described below for CentOS):
|
||||
Clone this repo under /tmp (to make sure the apt user has access to the unpacked clone), build the debian package and install it (alternatively you can skip the package and do it manually like described below for CentOS):
|
||||
|
||||
```bash
|
||||
cd /tmp
|
||||
git clone https://github.com/jimsalterjrs/sanoid.git
|
||||
cd sanoid
|
||||
# checkout latest stable release or stay on master for bleeding edge stuff (but expect bugs!)
|
||||
|
|
@ -73,6 +74,7 @@ cpan # answer the questions and paste the following lines:
|
|||
Clone this repo, then put the executables and config files into the appropriate directories:
|
||||
|
||||
```bash
|
||||
cd /tmp
|
||||
# Download the repo as root to avoid changing permissions later
|
||||
sudo git clone https://github.com/jimsalterjrs/sanoid.git
|
||||
cd sanoid
|
||||
|
|
@ -225,9 +227,9 @@ sudo launchctl load /Library/LaunchDaemons/net.openoid.Sanoid.plist
|
|||
|
||||
## Other OSes
|
||||
|
||||
**Sanoid** depends on the Perl module Config::IniFiles and will not operate without it. Config::IniFiles may be installed from CPAN, though the project strongly recommends using your distribution's repositories instead.
|
||||
**Sanoid** depends on the Perl modules Config::IniFiles and Capture::Tiny and will not operate without them. These modules may be installed from CPAN, though the project strongly recommends using your distribution's repositories instead.
|
||||
|
||||
**Syncoid** depends on ssh, pv, gzip, lzop, and mbuffer. It can run with reduced functionality in the absence of any or all of the above. SSH is only required for remote synchronization. On newer FreeBSD and Ubuntu Xenial chacha20-poly1305@openssh.com, on other distributions arcfour crypto is the default for SSH transport since v1.4.6. Syncoid runs will fail if one of them is not available on either end of the transport.
|
||||
**Syncoid** depends on ssh, pv, gzip, lzop, and mbuffer as well as sharing sanoid's dependency on Capture::Tiny. Capture::Tiny is mandatory, but syncoid can function with reduced functionality without any or all of the command-line dependencies. SSH is only required for remote synchronization. On newer FreeBSD and Ubuntu Xenial chacha20-poly1305@openssh.com, on other distributions arcfour crypto is the default for SSH transport since v1.4.6. Syncoid runs will fail if one of them is not available on either end of the transport.
|
||||
|
||||
### General outline for installation
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,9 @@
|
|||
sanoid (2.2.1) unstable; urgency=medium
|
||||
|
||||
[syncoid] add simple systemd.units
|
||||
|
||||
-- Christopher Bock <christopher@bocki.com> Sat, 10 Feb 2023 23:15:00 +0200
|
||||
|
||||
sanoid (2.2.0) unstable; urgency=medium
|
||||
|
||||
[overall] documentation updates, small fixes (@azmodude, @deviantintegral, @jimsalterjrs, @alexhaydock, @cbreak-black, @kd8bny, @JavaScriptDude, @veeableful, @rsheasby, @Topslakr, @mavhc, @adam-stamand, @joelishness, @jsoref, @dodexahedron, @phreaker0)
|
||||
|
|
|
|||
|
|
@ -20,4 +20,11 @@ Recommends: gzip,
|
|||
mbuffer,
|
||||
openssh-client | ssh-client,
|
||||
pv
|
||||
Description: Policy-driven snapshot management and replication tools
|
||||
Description: Policy-driven ZFS snapshot management and replication tool
|
||||
Sanoid is a policy-driven snapshot management and replication tool for ZFS
|
||||
filesystems.
|
||||
.
|
||||
More prosaically, you can use Sanoid to create, automatically thin, and
|
||||
monitor snapshots and pool health from a single eminently human-readable
|
||||
TOML config file. Sanoid also includes a replication tool, syncoid, which
|
||||
facilitates the asynchronous incremental replication of ZFS filesystems.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,24 @@
|
|||
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.3.
|
||||
.TH FINDOID "8" "February 2024" "findoid version 2.2.0" "System Administration Utilities"
|
||||
.SH NAME
|
||||
findoid \- manual page for findoid version 2.2.0
|
||||
.SH DESCRIPTION
|
||||
.SS "Usage:"
|
||||
.IP
|
||||
findoid [options] FILE
|
||||
.TP
|
||||
FILE
|
||||
local path to file for version listing
|
||||
.IP
|
||||
Options:
|
||||
.TP
|
||||
\fB\-\-path\fR=\fI\,FILE\/\fR
|
||||
alternative to specify file path to list versions for
|
||||
.TP
|
||||
\fB\-\-help\fR
|
||||
Prints this helptext
|
||||
.TP
|
||||
\fB\-\-version\fR
|
||||
Prints the version number
|
||||
.PP
|
||||
(Getopt::Long::GetOptions version 2.52; Perl version 5.36.0)
|
||||
|
|
@ -7,30 +7,8 @@
|
|||
%:
|
||||
dh $@
|
||||
|
||||
|
||||
DESTDIR = $(CURDIR)/debian/sanoid
|
||||
override_dh_auto_install:
|
||||
install -d $(DESTDIR)/etc/sanoid
|
||||
install -m 664 sanoid.defaults.conf $(DESTDIR)/etc/sanoid
|
||||
|
||||
install -d $(DESTDIR)/lib/systemd/system
|
||||
install -m 664 debian/sanoid-prune.service debian/sanoid.timer \
|
||||
$(DESTDIR)/lib/systemd/system
|
||||
|
||||
install -d $(DESTDIR)/usr/sbin
|
||||
install -m 775 \
|
||||
findoid sanoid sleepymutex syncoid \
|
||||
$(DESTDIR)/usr/sbin
|
||||
|
||||
install -d $(DESTDIR)/usr/share/doc/sanoid
|
||||
install -m 664 sanoid.conf \
|
||||
$(DESTDIR)/usr/share/doc/sanoid/sanoid.conf.example
|
||||
|
||||
override_dh_installinit:
|
||||
dh_installinit --noscripts
|
||||
|
||||
override_dh_systemd_enable:
|
||||
dh_systemd_enable sanoid.timer
|
||||
dh_systemd_enable sanoid-prune.service
|
||||
|
||||
override_dh_systemd_start:
|
||||
dh_systemd_start sanoid.timer
|
||||
|
|
|
|||
|
|
@ -0,0 +1,66 @@
|
|||
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.3.
|
||||
.TH SANOID "8" "February 2024" "sanoid version 2.2.0" "System Administration Utilities"
|
||||
.SH NAME
|
||||
sanoid \- manual page for sanoid version 2.2.0
|
||||
.SH DESCRIPTION
|
||||
.SS "Usage:"
|
||||
.IP
|
||||
sanoid [options]
|
||||
.IP
|
||||
Assumes \fB\-\-cron\fR \fB\-\-verbose\fR if no other arguments (other than configdir)
|
||||
are specified
|
||||
.IP
|
||||
Options:
|
||||
.TP
|
||||
\fB\-\-configdir\fR=\fI\,DIR\/\fR
|
||||
Specify a directory to find config file sanoid.conf
|
||||
.TP
|
||||
\fB\-\-cache\-dir\fR=\fI\,DIR\/\fR
|
||||
Specify a directory to store the zfs snapshot cache
|
||||
.TP
|
||||
\fB\-\-run\-dir\fR=\fI\,DIR\/\fR
|
||||
Specify a directory for temporary files such as lock files
|
||||
.TP
|
||||
\fB\-\-cron\fR
|
||||
Creates snapshots and purges expired snapshots
|
||||
.TP
|
||||
\fB\-\-verbose\fR
|
||||
Prints out additional information during a sanoid run
|
||||
.TP
|
||||
\fB\-\-readonly\fR
|
||||
Simulates creation/deletion of snapshots
|
||||
.TP
|
||||
\fB\-\-quiet\fR
|
||||
Suppresses non\-error output
|
||||
.TP
|
||||
\fB\-\-force\-update\fR
|
||||
Clears out sanoid's zfs snapshot cache
|
||||
.TP
|
||||
\fB\-\-monitor\-health\fR
|
||||
Reports on zpool "health", in a Nagios compatible format
|
||||
.TP
|
||||
\fB\-\-monitor\-capacity\fR
|
||||
Reports on zpool capacity, in a Nagios compatible format
|
||||
.TP
|
||||
\fB\-\-monitor\-snapshots\fR
|
||||
Reports on snapshot "health", in a Nagios compatible format
|
||||
.TP
|
||||
\fB\-\-take\-snapshots\fR
|
||||
Creates snapshots as specified in sanoid.conf
|
||||
.TP
|
||||
\fB\-\-prune\-snapshots\fR
|
||||
Purges expired snapshots as specified in sanoid.conf
|
||||
.TP
|
||||
\fB\-\-force\-prune\fR
|
||||
Purges expired snapshots even if a send/recv is in progress
|
||||
.TP
|
||||
\fB\-\-help\fR
|
||||
Prints this helptext
|
||||
.TP
|
||||
\fB\-\-version\fR
|
||||
Prints the version number
|
||||
.TP
|
||||
\fB\-\-debug\fR
|
||||
Prints out a lot of additional information during a sanoid run
|
||||
.PP
|
||||
(Getopt::Long::GetOptions version 2.52; Perl version 5.36.0)
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
sanoid.conf
|
||||
debian/syncoid.conf
|
||||
debian/syncoid.timer
|
||||
debian/syncoid.service
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
syncoid /usr/sbin/
|
||||
sanoid /usr/sbin/
|
||||
findoid /usr/sbin/
|
||||
sleepymutex /usr/sbin/
|
||||
sanoid.defaults.conf /usr/share/sanoid/
|
||||
debian/sanoid-prune.service /lib/systemd/system
|
||||
CHANGELIST /usr/share/doc/sanoid/changelog
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
debian/findoid.8
|
||||
debian/sanoid.8
|
||||
debian/syncoid.8
|
||||
|
|
@ -0,0 +1,133 @@
|
|||
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.3.
|
||||
.TH SYNCOID "8" "February 2024" "syncoid version 2.2.0" "System Administration Utilities"
|
||||
.SH NAME
|
||||
syncoid \- manual page for syncoid version 2.2.0
|
||||
.SH DESCRIPTION
|
||||
.SS "Usage:"
|
||||
.IP
|
||||
syncoid [options]... SOURCE TARGET
|
||||
or syncoid [options]... SOURCE [[USER]@]HOST:TARGET
|
||||
or syncoid [options]... [[USER]@]HOST:SOURCE TARGET
|
||||
or syncoid [options]... [[USER]@]HOST:SOURCE [[USER]@]HOST:TARGET
|
||||
.TP
|
||||
SOURCE
|
||||
Source ZFS dataset. Can be either local or remote
|
||||
.TP
|
||||
TARGET
|
||||
Target ZFS dataset. Can be either local or remote
|
||||
.IP
|
||||
Options:
|
||||
.TP
|
||||
\fB\-\-compress\fR=\fI\,FORMAT\/\fR
|
||||
Compresses data during transfer. Currently accepted options are gzip, pigz\-fast, pigz\-slow, zstd\-fast, zstd\-slow, lz4, xz, lzo (default) & none
|
||||
.TP
|
||||
\fB\-\-identifier\fR=\fI\,EXTRA\/\fR
|
||||
Extra identifier which is included in the snapshot name. Can be used for replicating to multiple targets.
|
||||
.TP
|
||||
\fB\-\-recursive\fR|r
|
||||
Also transfers child datasets
|
||||
.TP
|
||||
\fB\-\-skip\-parent\fR
|
||||
Skips syncing of the parent dataset. Does nothing without '\-\-recursive' option.
|
||||
.TP
|
||||
\fB\-\-source\-bwlimit=\fR<limit k|m|g|t>
|
||||
Bandwidth limit in bytes/kbytes/etc per second on the source transfer
|
||||
.TP
|
||||
\fB\-\-target\-bwlimit=\fR<limit k|m|g|t>
|
||||
Bandwidth limit in bytes/kbytes/etc per second on the target transfer
|
||||
.TP
|
||||
\fB\-\-mbuffer\-size\fR=\fI\,VALUE\/\fR
|
||||
Specify the mbuffer size (default: 16M), please refer to mbuffer(1) manual page.
|
||||
.TP
|
||||
\fB\-\-pv\-options\fR=\fI\,OPTIONS\/\fR
|
||||
Configure how pv displays the progress bar, default '\-p \fB\-t\fR \fB\-e\fR \fB\-r\fR \fB\-b\fR'
|
||||
.TP
|
||||
\fB\-\-no\-stream\fR
|
||||
Replicates using newest snapshot instead of intermediates
|
||||
.TP
|
||||
\fB\-\-no\-sync\-snap\fR
|
||||
Does not create new snapshot, only transfers existing
|
||||
.TP
|
||||
\fB\-\-keep\-sync\-snap\fR
|
||||
Don't destroy created sync snapshots
|
||||
.TP
|
||||
\fB\-\-create\-bookmark\fR
|
||||
Creates a zfs bookmark for the newest snapshot on the source after replication succeeds (only works with \fB\-\-no\-sync\-snap\fR)
|
||||
.TP
|
||||
\fB\-\-use\-hold\fR
|
||||
Adds a hold to the newest snapshot on the source and target after replication succeeds and removes the hold after the next successful replication. The hold name includes the identifier if set. This allows for separate holds in case of multiple targets
|
||||
.HP
|
||||
\fB\-\-preserve\-recordsize\fR Preserves the recordsize on initial sends to the target
|
||||
.HP
|
||||
\fB\-\-preserve\-properties\fR Preserves locally set dataset properties similar to the zfs send \fB\-p\fR flag but this one will also work for encrypted datasets in non raw sends
|
||||
.TP
|
||||
\fB\-\-no\-rollback\fR
|
||||
Does not rollback snapshots on target (it probably requires a readonly target)
|
||||
.HP
|
||||
\fB\-\-delete\-target\-snapshots\fR With this argument snapshots which are missing on the source will be destroyed on the target. Use this if you only want to handle snapshots on the source.
|
||||
.TP
|
||||
\fB\-\-exclude\fR=\fI\,REGEX\/\fR
|
||||
DEPRECATED. Equivalent to \fB\-\-exclude\-datasets\fR, but will be removed in a future release. Ignored if \fB\-\-exclude\-datasets\fR is also provided.
|
||||
.HP
|
||||
\fB\-\-exclude\-datasets\fR=\fI\,REGEX\/\fR Exclude specific datasets which match the given regular expression. Can be specified multiple times
|
||||
.HP
|
||||
\fB\-\-exclude\-snaps\fR=\fI\,REGEX\/\fR Exclude specific snapshots that match the given regular expression. Can be specified multiple times. If a snapshot matches both the exclude\-snaps and include\-snaps patterns, then it will be excluded.
|
||||
.HP
|
||||
\fB\-\-include\-snaps\fR=\fI\,REGEX\/\fR Only include snapshots that match the given regular expression. Can be specified multiple times. If a snapshot matches both the exclude\-snaps and include\-snaps patterns, then it will be excluded.
|
||||
.HP
|
||||
\fB\-\-sendoptions\fR=\fI\,OPTIONS\/\fR Use advanced options for zfs send (the arguments are filtered as needed), e.g. syncoid \fB\-\-sendoptions=\fR"Lc e" sets zfs send \fB\-L\fR \fB\-c\fR \fB\-e\fR ...
|
||||
.HP
|
||||
\fB\-\-recvoptions\fR=\fI\,OPTIONS\/\fR Use advanced options for zfs receive (the arguments are filtered as needed), e.g. syncoid \fB\-\-recvoptions=\fR"ux recordsize o compression=lz4" sets zfs receive \fB\-u\fR \fB\-x\fR recordsize \fB\-o\fR compression=lz4 ...
|
||||
.TP
|
||||
\fB\-\-sshconfig\fR=\fI\,FILE\/\fR
|
||||
Specifies an ssh_config(5) file to be used
|
||||
.TP
|
||||
\fB\-\-sshkey\fR=\fI\,FILE\/\fR
|
||||
Specifies a ssh key to use to connect
|
||||
.TP
|
||||
\fB\-\-sshport\fR=\fI\,PORT\/\fR
|
||||
Connects to remote on a particular port
|
||||
.TP
|
||||
\fB\-\-sshcipher\fR|c=CIPHER
|
||||
Passes CIPHER to ssh to use a particular cipher set
|
||||
.TP
|
||||
\fB\-\-sshoption\fR|o=OPTION
|
||||
Passes OPTION to ssh for remote usage. Can be specified multiple times
|
||||
.TP
|
||||
\fB\-\-insecure\-direct\-connection\fR=\fI\,IP\/\fR:PORT[,IP:PORT]
|
||||
WARNING: DATA IS NOT ENCRYPTED. First address pair is for connecting to the target and the second for listening at the target
|
||||
.TP
|
||||
\fB\-\-help\fR
|
||||
Prints this helptext
|
||||
.TP
|
||||
\fB\-\-version\fR
|
||||
Prints the version number
|
||||
.TP
|
||||
\fB\-\-debug\fR
|
||||
Prints out a lot of additional information during a syncoid run
|
||||
.TP
|
||||
\fB\-\-monitor\-version\fR
|
||||
Currently does nothing
|
||||
.TP
|
||||
\fB\-\-quiet\fR
|
||||
Suppresses non\-error output
|
||||
.TP
|
||||
\fB\-\-dumpsnaps\fR
|
||||
Dumps a list of snapshots during the run
|
||||
.TP
|
||||
\fB\-\-no\-command\-checks\fR
|
||||
Do not check command existence before attempting transfer. Not recommended
|
||||
.TP
|
||||
\fB\-\-no\-resume\fR
|
||||
Don't use the ZFS resume feature if available
|
||||
.TP
|
||||
\fB\-\-no\-clone\-handling\fR
|
||||
Don't try to recreate clones on target
|
||||
.TP
|
||||
\fB\-\-no\-privilege\-elevation\fR
|
||||
Bypass the root check, for use with ZFS permission delegation
|
||||
.TP
|
||||
\fB\-\-force\-delete\fR
|
||||
Remove target datasets recursively, if there are no matching snapshots/bookmarks (also overwrites conflicting named snapshots)
|
||||
.PP
|
||||
(Getopt::Long::GetOptions version 2.52; Perl version 5.36.0)
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
# syncoid config example for simple one target/source synchronization
|
||||
#
|
||||
# syntax: VARIABLE='foo' # don't forget the ''
|
||||
# keep in mind that the syncoid.service will call
|
||||
# ExecStart=/usr/sbin/syncoid $DEBUG $OPTS $EXCLUDES $INCLUDES $SSHOPTS $SOURCE $TARGET
|
||||
# there are so many variables so we don't have to write an gigantic line
|
||||
# and as i couldn't use e.g. --exclude-datasets="" it can't be used otherwise
|
||||
#
|
||||
# TODO: write syncoid-generator and probably a template syncoid@.service which
|
||||
# is driven by configuration files in /etc/sanoid/syncoid/$JOB.conf?
|
||||
# hint: systemd.path(5) PathChanged= for automatic config change detection
|
||||
# this will get rid of those disgusting variables
|
||||
#
|
||||
# all variables should be set, so don't comment them out!
|
||||
# DEBUG='--debug'
|
||||
DEBUG=''
|
||||
# OPTS="--compress=zstd-fast --delete-target-snapshots --force-delete --no-privilege-elevation --recursive"
|
||||
OPTS=''
|
||||
# SOURCE='rpool/home'
|
||||
# SOURCE='user@host:pool/data/set'
|
||||
SOURCE=''
|
||||
# TARGET='pool1/data/set'
|
||||
# TARGET='user@host:pool'
|
||||
TARGET=''
|
||||
# SSHOPTS='--sshkey=/path/to/.ssh/id_rsa --sshport=22'
|
||||
SSHOPTS=''
|
||||
# EXCLUDES="--exclude-datasets='.*nosnap.*'"
|
||||
EXCLUDES=''
|
||||
# INCLUDES="--include-datasets=
|
||||
INCLUDES=''
|
||||
# mbuffer behaves icky because env HOME isn't available
|
||||
MBUFFERRC='/etc/mbuffer.rc'
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
[Unit]
|
||||
Description=Transfer ZFS Snapshots
|
||||
Documentation=man:syncoid(8)
|
||||
Requires=local-fs.target
|
||||
Wants=sanoid.service
|
||||
ConditionFileNotEmpty=/etc/sanoid/syncoid.conf
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/etc/sanoid/syncoid.conf
|
||||
Type=oneshot
|
||||
ExecStart=/usr/sbin/syncoid $DEBUG $OPTS $EXCLUDES $INCLUDES $SSHOPTS $SOURCE $TARGET
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
[Unit]
|
||||
Description=Transfer ZFS Snapshots
|
||||
Documentation=man:syncoid(8)
|
||||
Documentation=man:systemd.time(7)
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
|
||||
[Timer]
|
||||
RandomizedDelaySec=60
|
||||
OnBootSec=5m
|
||||
#OnUnitActiveSec=20m
|
||||
OnCalendar=00/2:30
|
||||
Persistent=true
|
||||
Unit=syncoid.service
|
||||
Loading…
Reference in New Issue