cyteen
/
sanoid
mirror of https://github.com/jimsalterjrs/sanoid
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #497 from phreaker0/scripts-snapshot-grouping 

Scripts snapshot grouping
This commit is contained in:
Jim Salter 2020-11-01 17:47:11 -05:00 committed by GitHub
parent 59e8949efe adc99d0349
commit a4cde57d29
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 194 additions and 74 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

94
README.md
View File

@ -111,6 +111,100 @@ Which would be enough to tell sanoid to take and keep 36 hourly snapshots, 30 da
Show help message. Show help message.
### Sanoid script hooks
There are three script types which can optionally be executed at various stages in the lifecycle of a snapshot:
#### `pre_snapshot_script`
Will be executed before the snapshot(s) of a single dataset are taken. The following environment variables are passed:
| Env vars | Description |
| ----------------- | ----------- |
| `SANOID_SCRIPT` | The type of script being executed, one of `pre`, `post`, or `prune`. Allows for one script to be used for multiple tasks |
| `SANOID_TARGET` | **DEPRECATED** The dataset about to be snapshot (only the first dataset will be provided) |
| `SANOID_TARGETS` | Comma separated list of all datasets to be snapshoted (currently only a single dataset, multiple datasets will be possible later with atomic groups) |
| `SANOID_SNAPNAME` | **DEPRECATED** The name of the snapshot that will be taken (only the first name will be provided, does not include the dataset name) |
| `SANOID_SNAPNAMES` | Comma separated list of all snapshot names that will be taken (does not include the dataset name) |
| `SANOID_TYPES` | Comma separated list of all snapshot types to be taken (yearly, monthly, weekly, daily, hourly, frequently) |
If the script returns a non-zero exit code, the snapshot(s) will not be taken unless `no_inconsistent_snapshot` is false.
#### `post_snapshot_script`
Will be executed when:
- The pre-snapshot script succeeded or
- The pre-snapshot script failed and `force_post_snapshot_script` is true.
| Env vars | Description |
| -------------------- | ----------- |
| `SANOID_SCRIPT` | as above |
| `SANOID_TARGET` | **DEPRECATED** as above |
| `SANOID_TARGETS` | as above |
| `SANOID_SNAPNAME` | **DEPRECATED** as above |
| `SANOID_SNAPNAMES` | as above |
| `SANOID_TYPES` | as above |
| `SANOID_PRE_FAILURE` | This will indicate if the pre-snapshot script failed |
#### `pruning_script`
Will be executed after a snapshot is successfully deleted. The following environment variables will be passed:
| Env vars | Description |
| ----------------- | ----------- |
| `SANOID_SCRIPT` | as above |
| `SANOID_TARGET` | as above |
| `SANOID_SNAPNAME` | as above |
#### example
**sanoid.conf**:
```
...
[sanoid-test-0]
use_template = production
recursive = yes
pre_snapshot_script = /tmp/debug.sh
post_snapshot_script = /tmp/debug.sh
pruning_script = /tmp/debug.sh
...
```
**verbose sanoid output**:
```
...
executing pre_snapshot_script '/tmp/debug.sh' on dataset 'sanoid-test-0'
taking snapshot sanoid-test-0@autosnap_2020-02-12_14:49:33_yearly
taking snapshot sanoid-test-0@autosnap_2020-02-12_14:49:33_monthly
taking snapshot sanoid-test-0@autosnap_2020-02-12_14:49:33_daily
taking snapshot sanoid-test-0@autosnap_2020-02-12_14:49:33_hourly
executing post_snapshot_script '/tmp/debug.sh' on dataset 'sanoid-test-0'
...
```
**pre script env variables**:
```
SANOID_SCRIPT=pre
SANOID_TARGET=sanoid-test-0/b/bb
SANOID_TARGETS=sanoid-test-0/b/bb
SANOID_SNAPNAME=autosnap_2020-02-12_14:49:32_yearly
SANOID_SNAPNAMES=autosnap_2020-02-12_14:49:32_yearly,autosnap_2020-02-12_14:49:32_monthly,autosnap_2020-02-12_14:49:32_daily,autosnap_2020-02-12_14:49:32_hourly
SANOID_TYPES=yearly,monthly,daily,hourly
```
**post script env variables**:
```
SANOID_SCRIPT=post
SANOID_TARGET=sanoid-test-0/b/bb
SANOID_TARGETS=sanoid-test-0/b/bb
SANOID_SNAPNAME=autosnap_2020-02-12_14:49:32_yearly
SANOID_SNAPNAMES=autosnap_2020-02-12_14:49:32_yearly,autosnap_2020-02-12_14:49:32_monthly,autosnap_2020-02-12_14:49:32_daily,autosnap_2020-02-12_14:49:32_hourly
SANOID_TYPES=yearly,monthly,daily,hourly
SANOID_PRE_FAILURE=0
```
---------- ----------
# Syncoid # Syncoid

97
sanoid
View File

@ -331,11 +331,13 @@ sub prune_snapshots {
if ($config{$dataset}{'pruning_script'}) { if ($config{$dataset}{'pruning_script'}) {
$ENV{'SANOID_TARGET'} = $dataset; $ENV{'SANOID_TARGET'} = $dataset;
$ENV{'SANOID_SNAPNAME'} = $snapname; $ENV{'SANOID_SNAPNAME'} = $snapname;
$ENV{'SANOID_SCRIPT'} = 'prune';
if ($args{'verbose'}) { print "executing pruning_script '".$config{$dataset}{'pruning_script'}."' on dataset '$dataset'\n"; } if ($args{'verbose'}) { print "executing pruning_script '".$config{$dataset}{'pruning_script'}."' on dataset '$dataset'\n"; }
my $ret = runscript('pruning_script',$dataset); my $ret = runscript('pruning_script',$dataset);
delete $ENV{'SANOID_TARGET'}; delete $ENV{'SANOID_TARGET'};
delete $ENV{'SANOID_SNAPNAME'}; delete $ENV{'SANOID_SNAPNAME'};
delete $ENV{'SANOID_SCRIPT'};
} }
} else { } else {
warn "could not remove $snap : $?"; warn "could not remove $snap : $?";
@ -370,7 +372,7 @@ sub take_snapshots {
my %datestamp = get_date(); my %datestamp = get_date();
my $forcecacheupdate = 0; my $forcecacheupdate = 0;
my @newsnaps; my %newsnapsgroup;
# get utc timestamp of the current day for DST check # get utc timestamp of the current day for DST check
my $daystartUtc = timelocal(0, 0, 0, $datestamp{'mday'}, ($datestamp{'mon'}-1), $datestamp{'year'}); my $daystartUtc = timelocal(0, 0, 0, $datestamp{'mday'}, ($datestamp{'mon'}-1), $datestamp{'year'});
@ -514,55 +516,58 @@ sub take_snapshots {
my $maxage = time()-$lastpreferred; my $maxage = time()-$lastpreferred;
if ( $newestage > $maxage ) { if ( $newestage > $maxage ) {
# update to most current possible datestamp
%datestamp = get_date();
# print "we should have had a $type snapshot of $path $maxage seconds ago; most recent is $newestage seconds old.\n"; # print "we should have had a $type snapshot of $path $maxage seconds ago; most recent is $newestage seconds old.\n";
if (!exists $newsnapsgroup{$path}) {
my $flags = ""; $newsnapsgroup{$path} = {
# use zfs (atomic) recursion if specified in config 'recursive' => $config{$section}{'zfs_recursion'},
if ($config{$section}{'zfs_recursion'}) { 'handleDst' => $handleDst,
$flags .= "r"; 'datasets' => [$path], # for later atomic grouping, currently only a one element array
} 'types' => []
if ($handleDst) { };
$flags .= "d";
} }
if ($flags ne "") { push(@{$newsnapsgroup{$path}{'types'}}, $type);
push(@newsnaps, "$path\@autosnap_$datestamp{'sortable'}_$type\@$flags");
} else {
push(@newsnaps, "$path\@autosnap_$datestamp{'sortable'}_$type");
}
} }
} }
} }
} }
if ( (scalar(@newsnaps)) > 0) { if (%newsnapsgroup) {
foreach my $snap ( @newsnaps ) { while ((my $path, my $snapData) = each(%newsnapsgroup)) {
my $recursiveFlag = $snapData->{recursive};
my $dstHandling = $snapData->{handleDst};
my @datasets = @{$snapData->{datasets}};
my $dataset = $datasets[0];
my @types = @{$snapData->{types}};
# same timestamp for all snapshots types (daily, hourly, ...)
my %datestamp = get_date();
my @snapshots;
foreach my $type (@types) {
my $snapname = "autosnap_$datestamp{'sortable'}_$type";
push(@snapshots, $snapname);
}
my $datasetString = join(",", @datasets);
my $typeString = join(",", @types);
my $snapshotString = join(",", @snapshots);
my $extraMessage = ""; my $extraMessage = "";
my @split = split '@', $snap, -1; if ($recursiveFlag) {
my $recursiveFlag = 0;
my $dstHandling = 0;
if (scalar(@split) == 3) {
my $flags = $split[2];
if (index($flags, "r") != -1) {
$recursiveFlag = 1;
$extraMessage = " (zfs recursive)"; $extraMessage = " (zfs recursive)";
chop $snap;
} }
if (index($flags, "d") != -1) {
$dstHandling = 1;
chop $snap;
}
chop $snap;
}
my $dataset = $split[0];
my $snapname = $split[1];
my $presnapshotfailure = 0; my $presnapshotfailure = 0;
my $ret = 0; my $ret = 0;
if ($config{$dataset}{'pre_snapshot_script'}) { if ($config{$dataset}{'pre_snapshot_script'}) {
$ENV{'SANOID_TARGET'} = $dataset; $ENV{'SANOID_TARGET'} = $dataset;
$ENV{'SANOID_SNAPNAME'} = $snapname; $ENV{'SANOID_TARGETS'} = $datasetString;
$ENV{'SANOID_SNAPNAME'} = $snapshots[0];
$ENV{'SANOID_SNAPNAMES'} = $snapshotString;
$ENV{'SANOID_TYPES'} = $typeString;
$ENV{'SANOID_SCRIPT'} = 'pre';
if ($args{'verbose'}) { print "executing pre_snapshot_script '".$config{$dataset}{'pre_snapshot_script'}."' on dataset '$dataset'\n"; } if ($args{'verbose'}) { print "executing pre_snapshot_script '".$config{$dataset}{'pre_snapshot_script'}."' on dataset '$dataset'\n"; }
if (!$args{'readonly'}) { if (!$args{'readonly'}) {
@ -570,7 +575,11 @@ sub take_snapshots {
} }
delete $ENV{'SANOID_TARGET'}; delete $ENV{'SANOID_TARGET'};
delete $ENV{'SANOID_TARGETS'};
delete $ENV{'SANOID_SNAPNAME'}; delete $ENV{'SANOID_SNAPNAME'};
delete $ENV{'SANOID_SNAPNAMES'};
delete $ENV{'SANOID_TYPES'};
delete $ENV{'SANOID_SCRIPT'};
if ($ret != 0) { if ($ret != 0) {
# warning was already thrown by runscript function # warning was already thrown by runscript function
@ -578,7 +587,11 @@ sub take_snapshots {
$presnapshotfailure = 1; $presnapshotfailure = 1;
} }
} }
foreach my $snap (@snapshots) {
$snap = "$dataset\@$snap";
if ($args{'verbose'}) { print "taking snapshot $snap$extraMessage\n"; } if ($args{'verbose'}) { print "taking snapshot $snap$extraMessage\n"; }
if (!$args{'readonly'}) { if (!$args{'readonly'}) {
my $stderr; my $stderr;
my $exit; my $exit;
@ -615,10 +628,17 @@ sub take_snapshots {
} }
}; };
} }
}
if ($config{$dataset}{'post_snapshot_script'}) { if ($config{$dataset}{'post_snapshot_script'}) {
if (!$presnapshotfailure or $config{$dataset}{'force_post_snapshot_script'}) { if (!$presnapshotfailure or $config{$dataset}{'force_post_snapshot_script'}) {
$ENV{'SANOID_TARGET'} = $dataset; $ENV{'SANOID_TARGET'} = $dataset;
$ENV{'SANOID_SNAPNAME'} = $snapname; $ENV{'SANOID_TARGETS'} = $datasetString;
$ENV{'SANOID_SNAPNAME'} = $snapshots[0];
$ENV{'SANOID_SNAPNAMES'} = $snapshotString;
$ENV{'SANOID_TYPES'} = $typeString;
$ENV{'SANOID_SCRIPT'} = 'post';
$ENV{'SANOID_PRE_FAILURE'} = $presnapshotfailure;
if ($args{'verbose'}) { print "executing post_snapshot_script '".$config{$dataset}{'post_snapshot_script'}."' on dataset '$dataset'\n"; } if ($args{'verbose'}) { print "executing post_snapshot_script '".$config{$dataset}{'post_snapshot_script'}."' on dataset '$dataset'\n"; }
if (!$args{'readonly'}) { if (!$args{'readonly'}) {
@ -626,7 +646,12 @@ sub take_snapshots {
} }
delete $ENV{'SANOID_TARGET'}; delete $ENV{'SANOID_TARGET'};
delete $ENV{'SANOID_TARGETS'};
delete $ENV{'SANOID_SNAPNAME'}; delete $ENV{'SANOID_SNAPNAME'};
delete $ENV{'SANOID_SNAPNAMES'};
delete $ENV{'SANOID_TYPES'};
delete $ENV{'SANOID_SCRIPT'};
delete $ENV{'SANOID_PRE_FAILURE'};
} }
} }
} }

4
sanoid.conf
View File

@ -96,8 +96,8 @@
daily_crit = 4d daily_crit = 4d
[template_scripts] [template_scripts]
### dataset and snapshot name will be supplied as environment variables ### information about the snapshot will be supplied as environment variables,
### for all pre/post/prune scripts ($SANOID_TARGET, $SANOID_SNAPNAME) ### see the README.md file for details about what is passed when.
### run script before snapshot ### run script before snapshot
pre_snapshot_script = /path/to/script.sh pre_snapshot_script = /path/to/script.sh
### run script after snapshot ### run script after snapshot

1
sanoid.defaults.conf
View File

@ -19,6 +19,7 @@ use_template =
process_children_only = process_children_only =
skip_children = skip_children =
# See "Sanoid script hooks" in README.md for information about scripts.
pre_snapshot_script = pre_snapshot_script =
post_snapshot_script = post_snapshot_script =
pruning_script = pruning_script =