Commit Graph

883 Commits

Author SHA1 Message Date
Christoph Klaffl 85e7fca30e
Merge branch 'master' into force-delete-skip-root 2024-01-13 21:29:40 +01:00
Christoph Klaffl ca6e60b920
Merge branch 'master' into update-send-recv-options 2024-01-13 21:22:51 +01:00
Jim Salter 680bf23412
Merge pull request #699 from mr-vinn/filter-snaps
Add --include-snaps and --exclude-snaps options to syncoid
2024-01-13 14:45:03 -05:00
Christoph Klaffl 8ce1ea4dc8
fixed refactoring regression 2024-01-13 19:49:20 +01:00
Christoph Klaffl e9eb05e840
Merge branch 'master' into filter-snaps 2024-01-13 19:40:28 +01:00
Christoph Klaffl 6761004939
update possible zfs send options 2024-01-11 21:02:04 +01:00
Christoph Klaffl 4369576ac4
escape property key and value pair in case of property preservation 2024-01-09 20:40:33 +01:00
Christoph Klaffl 48d89c785e
prevent destroying of root dataset which leads to infinite loop because it can't be destroyed 2024-01-09 19:53:03 +01:00
Justin Wolf dbbaac8ac3 modify zfs-get argument order for portability 2023-12-10 21:16:42 -06:00
Jan Krieg 605b7bac1c
fix "creation"/"guid" regex detection 2023-10-29 17:46:28 +01:00
pajkastare a5a6fc0f58 Fixes jimsalterjrs/sanoid#851 2023-10-23 21:43:46 +02:00
Axel Gembe 07b6d6344c
debian: add openzfs-zfsutils as an alternative to zfsutils-linux
The package produced by ZFS 2.2.0 `make native-deb-utils` is called
`openzfs-zfsutils`.
2023-10-15 14:07:09 +07:00
Jiawen (Kevin) Chen 18ccb7df35
Fix tiny typo in README.md 2023-08-14 22:52:16 -07:00
Michael Jeanson 6b874a7e3c Fix typos in syncoid documentation
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
2023-08-03 16:17:51 -04:00
Vitalii Zurian a881d22c85
Update INSTALL.md 2023-08-01 10:05:46 +02:00
Jim Salter 61000c9da2
Merge pull request #837 from phreaker0/trim-config-values
trim config values
2023-07-25 17:29:06 -04:00
Christoph Klaffl 7ab0ac0cf6
trim config values 2023-07-25 22:56:25 +02:00
Jim Salter 91a960ffc5
Merge pull request #832 from phreaker0/prepare-2.2.0
prepare 2.2.0
2023-07-18 12:29:53 -04:00
Christoph Klaffl 538416879d
prepare 2.2.0 2023-07-18 18:09:51 +02:00
Christoph Klaffl f3d4d309b5
implemented flag for preserving properties without the zfs -p flag 2023-07-18 08:38:40 +02:00
Nick Liu a904ba02f3
enh(run-tests.sh): Sort tests with "general numeric sort"
The sort before tended to be alphabetical, which put test
`8_force_delete_snapshot` after `815_sync_out-of-order_snapshots`, but
`8` should come before `815`.

Before:

```
root@demo:~/sanoid/tests/syncoid# ./run-tests.sh
Running test 1_bookmark_replication_intermediate ... [PASS]
Running test 2_bookmark_replication_no_intermediate ... [PASS]
Running test 3_force_delete ... [PASS]
Running test 4_bookmark_replication_edge_case ... [PASS]
Running test 5_reset_resume_state ... mbuffer: error: outputThread: error writing to <stdout> at offset 0x90000: Broken pipe
mbuffer: warning: error during output to <stdout>: Broken pipe
[PASS]
Running test 6_reset_resume_state2 ... [PASS]
Running test 7_preserve_recordsize ... [PASS]
Running test 815_sync_out-of-order_snapshots ... [PASS]
Running test 8_force_delete_snapshot ... [PASS]
```

After:

```
root@demo:~/sanoid/tests/syncoid# ./run-tests.sh
Running test 1_bookmark_replication_intermediate ... [PASS]
Running test 2_bookmark_replication_no_intermediate ... [PASS]
Running test 3_force_delete ... [PASS]
Running test 4_bookmark_replication_edge_case ... [PASS]
Running test 5_reset_resume_state ... mbuffer: error: outputThread: error writing to <stdout> at offset 0xf0000: Broken pipe
mbuffer: warning: error during output to <stdout>: Broken pipe
[PASS]
Running test 6_reset_resume_state2 ... [PASS]
Running test 7_preserve_recordsize ... [PASS]
Running test 8_force_delete_snapshot ... [PASS]
Running test 815_sync_out-of-order_snapshots ... [PASS]
```
2023-04-28 01:00:03 -05:00
Jim Salter 55c5e0ee09
Merge pull request #513 from phreaker0/direct-connection
implemented option for direct connection via socat and busybox nc
2023-04-27 14:26:11 -04:00
Jim Salter c69381e5b9
Merge pull request #819 from phreaker0/fixed-default-options
fix default behaviour if run without providing arguments
2023-04-27 14:22:47 -04:00
Christoph Klaffl 1e3874aea5
Merge branch 'master' into direct-connection 2023-04-27 09:16:51 +02:00
Jim Salter 3bde229a61
Merge pull request #622 from rbike/syncoid-hold
syncoid hold
2023-04-26 14:02:15 -04:00
Christoph Klaffl 01ace52642
Merge branch 'master' into syncoid-hold 2023-04-26 19:59:20 +02:00
Jim Salter f8ff5ab0c2
Merge pull request #809 from phreaker0/handle-same-second-bookmarks
support bookmarks which are taken in the same second
2023-04-26 13:49:37 -04:00
Christoph Klaffl cc495183af
fix default behaviour if run without providing arguments 2023-04-26 19:47:49 +02:00
Christoph Klaffl 2143f629d5
Merge remote-tracking branch 'upstream/master' into handle-same-second-bookmarks 2023-04-26 19:40:24 +02:00
Jim Salter e6585e7ee5
Merge pull request #746 from mavhc/master
Removed unneeded 2>/dev/null from mbufferoptions
2023-04-26 13:00:03 -04:00
Jim Salter 6804826cf4
Merge pull request #808 from phreaker0/fix-tests
fixes some tests which fail on fast storage
2023-04-26 12:59:27 -04:00
Jim Salter ed1c542850
Merge pull request #523 from mat813/snapshot-removal
Add target snapshot deletion.
2023-04-26 12:58:29 -04:00
Nick Liu b37092f376
test(syncoid): Add test to verify out-of-order snapshot sync
See https://github.com/jimsalterjrs/sanoid/issues/815 for the original
test.
2023-04-25 17:35:45 -05:00
Nick Liu ab361017e7
feat(syncoid): Match snapshots to bookmarks by `createtxg` if possible
This is a continuation of a previous commit to sort snapshots by
`createtxg` if possible.  Now, we have to match the behavior when
selecting an appropriate snapshot based on the transaction group of the
relevant bookmark in `syncdataset()`.

Supersedes: https://github.com/jimsalterjrs/sanoid/pull/667
2023-04-25 17:07:32 -05:00
Nick Liu 8907e0cb2f
feat(syncoid): Sort snapshots by `createtxg` if possible
It is possible for `creation` of a subsequent snapshot to be in the past
compared to the current snapshot due to system clock discrepancies,
which leads to earlier snapshots not being replicated in the initial
syncoid sync.

Also, `syncoid --no-sync-snap` might not pick up the most recently taken
snapshot if the clock moved backwards before taking that snapshot.

Sorting snapshots by the `createtxg` value is reliable and documented
in `man 8 zfsprops` as the proper way to order snapshots, but it was not
available in ZFS versions before 0.7.  To maintain backwards
compatibility, the sorting falls back to sorting by the `creation`
property, which was the old behavior.

Fixes: https://github.com/jimsalterjrs/sanoid/issues/815
2023-04-28 00:43:47 -05:00
Nick Liu 8fabaae5b8
feat(syncoid): Add "createtxg" property to `getsnaps`
The `getsnaps` subroutine now retrieves the "createtxg" property of the
snapshot.

This is necessary to support the fix for
https://github.com/jimsalterjrs/sanoid/issues/815 (Syncoid: Data loss
because getoldestsnapshot() might not choose the first snapshot).
2023-04-25 14:01:54 -05:00
Nick Liu e301b5b153
refactor(syncoid): Simplify getsnaps to parse a hash rather than lines
* The part that was "a little obnoxious" has been rewritten to extract
  the desired properties in a single loop after importing each line into
  a hash rather than processing line by line with a state tracking flag.
* The `getsnapsfallback` subroutine had duplicated logic that has been
  absorbed into `getsnaps` with a recursion argument to enable the
  fallback mode.
2023-04-25 13:58:40 -05:00
Mathieu Arnold 2f706a4ae1
Batch snapshot deletion.
This is to prevent a problem with a large amount of snapshots which
exceed the allowed shell command length.
2023-04-08 10:09:11 +02:00
Mathieu Arnold f711e6bf28
Add a note about when snapshot deletion is done. 2023-04-08 09:45:01 +02:00
Mathieu Arnold ecd1400539
Handle output/errors of those zfs destroy commands.
If there was an obsolete remote syncoid_hostname_* snapshot that did not
get removed at the correct time, for some reason, like, maybe, network
problems, it would have been cleaned up in pruneoldsyncsnaps just before
this code, and we would get a strange error message saying:

    could not find any snapshots to destroy; check snapshot names.

Also, when using --quiet, do not output anything, as failing to remove
an obsolete snapshot is not really a big problem.
2023-04-08 09:45:01 +02:00
Mathieu Arnold 63dd819ec5
Rename option. 2023-04-08 09:45:00 +02:00
Mathieu Arnold 2d89434ac3
Add target snapshot deletion. 2023-04-08 09:42:58 +02:00
Christoph Klaffl 9e04911945
Merge branch 'master' into direct-connection 2023-03-24 09:16:13 +01:00
Christoph Klaffl 91d96a4c87
support bookmarks which are taken in the same second 2023-03-24 09:00:07 +01:00
Christoph Klaffl a7f9f08f47
fixes some tests which fail on fast storage 2023-03-21 16:44:38 +01:00
Christoph Klaffl 0b27059133
Merge remote-tracking branch 'upstream/master' into filter-snaps 2023-03-21 16:35:20 +01:00
Christoph Klaffl f3152d6f6c
Merge remote-tracking branch 'upstream/master' into syncoid-hold 2023-03-21 15:44:04 +01:00
Jim Salter 86fffca306
Merge pull request #807 from phreaker0/pr-merger
PR Merger
2023-03-21 02:04:45 -04:00
Christoph Klaffl e5b1d0fd4f
Merge branch 'pr796' into pr-merger 2023-03-20 22:31:40 +01:00
Christoph Klaffl b3d32d9394
Merge branch 'pr788' into pr-merger 2023-03-20 22:31:19 +01:00