Commit Graph

965 Commits

Author SHA1 Message Date
Jim Salter 8e1d11e0b2
Merge pull request #572 from JakobR/dataset-dependencies
Replicate all dependencies of a dataset first
2026-02-18 17:58:51 -05:00
Jim Salter fc010c9118
Merge pull request #1077 from phreaker0/fix-zero-named-bookmarks
fix replication with '0' named bookmarks
2026-02-18 17:13:29 -05:00
Christoph Klaffl dcb3978fed
Merge branch 'master' into fix-zero-named-bookmarks 2026-02-18 23:12:03 +01:00
Jim Salter ba495e58af
Merge pull request #626 from 0xFelix/ignore-failed-create-bookmark
syncoid: Compare existing bookmarks
2026-02-18 17:08:18 -05:00
Jim Salter 04ca8f4e58
Merge pull request #1071 from r-ricci/print
don't print cache updates on stdout
2026-02-18 17:07:53 -05:00
Jim Salter 31104a4488
Merge pull request #1009 from bjoern-r/snaplist-from-configured-datasets
get snpshots from configured list of datasets only
2026-02-18 17:07:02 -05:00
Jim Salter 20abb530c8
Merge pull request #1078 from phreaker0/fix-009-test
fix test with latest zfs versions
2026-02-18 17:06:25 -05:00
Jim Salter 9f76aab5d6
Merge pull request #1051 from ifazk/busyfix
Fix iszfsbusy to not match pool prefix
2026-02-18 17:04:44 -05:00
Jim Salter d763e45dd4
Merge pull request #1007 from Deltik/fix/815
syncoid: Sort snapshots by `createtxg` if possible (fallback to `creation`) (redux of #818)
2026-02-18 17:02:00 -05:00
Jim Salter b4a5394d1f
Merge pull request #1079 from phreaker0/doc-weeklies
add weeklies documentation
2026-02-18 17:00:11 -05:00
Christoph Klaffl 3b606768fa
Merge branch 'master' into fix/815 2026-02-18 22:59:26 +01:00
Jim Salter 3d866c9473
Merge pull request #1054 from ifazk/unused-forcerecv
Remove unused forcerecv variable
2026-02-18 16:59:00 -05:00
Jim Salter 07e32cad71
Merge pull request #1044 from numerfolt/master
Change syncoid snapshots date and time divider from ":" to "_"
2026-02-18 16:49:55 -05:00
Christoph Klaffl 4415b36ba8
fix error handling for fallback (old behaviour) and codestyle 2026-02-18 20:36:42 +01:00
Christoph Klaffl 730bce6d38
add weeklies documentation 2026-02-18 15:20:23 +01:00
Christoph Klaffl 2875e10adb
fix test with latest zfs versions 2026-02-18 15:07:10 +01:00
Christoph Klaffl 3a1a19b39b
fix replication with '0' named bookmarks 2026-02-18 14:57:38 +01:00
Jim Salter 1ae8f26e82
avoid bug with snapshots named 0
Updated error handling to check for empty string instead of false for snapshot retrieval.
2026-02-18 00:19:37 -05:00
Jim Salter d9fe50b564
Merge pull request #1074 from ifazk/create-bookmark-last-snapshot
Fix bookmark last snapshot sync early exit
2026-02-17 13:04:37 -05:00
Jim Salter d8ff385691
Change comparison from '==' to 'ne' for error checks
Zpool status reports error counts with unit suffixes after the first 999 errors in any given column; as originally written, the monitor-health plugin used strictly numeric comparison checks which would error out when encountering a value like "1.09K" for checksum errors.
2026-02-17 11:53:42 -05:00
Ifaz Kabir 89d3cbcda3 Fix bookmark last snapshot sync early exit 2026-02-17 10:37:28 -05:00
Roberto Ricci 2343089a08 don't print cache updates on stdout
Fixes 393a4672e5
2026-01-31 19:12:56 +01:00
Jim Salter 74b61d149f
Enhance disk space check script functionality
Updated the script to accept warning and critical levels as decimal arguments. Added support for excluding 'squashfs' filesystems from the df command output.
2026-01-19 23:51:15 -05:00
Jim Salter 536cceea95
Enhance comments for check_all_disk_space script
Updated comments to clarify script functionality and usage.
2026-01-19 23:30:35 -05:00
Jim Salter 361e6839d1
Add disk space check script for ZFS filesystems
This script checks the available disk space on all mounted filesystems, producing Nagios-ready exit codes and brief, human-readable text.
2026-01-19 23:23:34 -05:00
Felix Matouschek dcae5ce4b5 syncoid: Compare existing bookmarks
When creating bookmarks compare the GUID of existing bookmarks before
failing the creation of a duplicate bookmark.

Signed-off-by: Felix Matouschek <felix@matouschek.org>
2026-01-10 11:36:25 +00:00
Ifaz Kabir 225c9f99d1 Remove unused forcerecv variable 2025-12-10 21:10:32 -05:00
Ifaz Kabir 4543705ffc Fix iszfsbusy to not match pool prefix 2025-12-03 23:49:22 -05:00
Bjoern 29f05ff5c4 fix dataset filter if autoprune is set 2025-11-18 12:14:06 +01:00
Bjoern Riemer 1c6d7d6459 onl query not ignored datasets for snapshots 2025-11-09 22:44:20 +01:00
numerfolt 4c9fba2277
Change date and time divider from : to _ 2025-09-29 22:04:29 +02:00
Björn 24b0293b0f
fallback to old zfs list snapshot method in case of error 2025-08-25 17:52:48 +02:00
Jim Salter dbcaeef1ac
Remove Alpine Linux ps installation instructions
Removed instructions for installing ps on Alpine Linux.
2025-08-24 17:30:21 -04:00
Jim Salter 2c98f0501c
Merge pull request #634 from asche77/patch-1
Update INSTALL.md to include OmniOS installation instructions
2025-08-24 11:29:35 -04:00
Jim Salter 0f3a9c94d9
Merge branch 'master' into patch-1 2025-08-24 11:28:40 -04:00
Jim Salter a6728e48de
Merge pull request #996 from jjakob/fix_directtimeout
syncoid: fix directtimeout in directmbuffer mode
2025-08-24 10:56:44 -04:00
Jim Salter 393a4672e5
Merge pull request #1023 from phreaker0/fix-file-handle-conflict
fixed file handle conflict
2025-08-24 10:50:58 -04:00
Christoph Klaffl efd52f416d
fixed file handle conflict 2025-08-12 14:19:47 +02:00
Nick Liu 1952e96846
fix(tests/common/lib.sh): Support set -e in test scripts
The `systemctl is-active virtualbox-guest-utils.service` command returns
a non-zero exit status when the service is inactive, causing early exit
in tests that use `set -e`.

This change suppresses that error with `|| true` to prevent test failure
when the service check returns non-zero.

Fixes this test:

```
root@demo:~/sanoid/tests/syncoid# cat /tmp/syncoid_test_run_011_sync_out-of-order_snapshots.log
+ set -e
+ . ../../common/lib.sh
+++ uname
++ unamestr=Linux
+ POOL_IMAGE=/tmp/jimsalterjrs_sanoid_815.img
+ POOL_SIZE=64M
+ POOL_NAME=jimsalterjrs_sanoid_815
+ truncate -s 64M /tmp/jimsalterjrs_sanoid_815.img
+ zpool create -m none -f jimsalterjrs_sanoid_815 /tmp/jimsalterjrs_sanoid_815.img
+ trap cleanUp EXIT
+ zfs create jimsalterjrs_sanoid_815/before
+ zfs snapshot jimsalterjrs_sanoid_815/before@this-snapshot-should-make-it-into-the-after-dataset
+ disableTimeSync
+ which timedatectl
+ '[' 0 -eq 0 ']'
+ timedatectl set-ntp 0
+ which systemctl
+ '[' 0 -eq 0 ']'
+ systemctl is-active virtualbox-guest-utils.service
inactive
+ cleanUp
+ zpool export jimsalterjrs_sanoid_815
+ rm -f /tmp/jimsalterjrs_sanoid_815.img
```
2025-06-18 00:15:44 +02:00
Björn 510becee2f
get snpshots from list of datasets
query the existing snapshots only from the configured datasets to avoud spinning up disks that are not in the config
2025-06-17 13:32:59 +02:00
Jim Salter 940a84e21f
Merge pull request #1008 from aabccd021/master
Fix readme formatting
2025-06-12 09:53:57 -04:00
aabccd021 680194fa33 Fix readme formatting 2025-06-12 12:58:26 +07:00
Nick Liu f2e1e4f8a0
test(syncoid): Add test to verify out-of-order snapshot sync
This commit adds a regression test for the out-of-order snapshot
replication issue.

The new test case manipulates the system clock with `setdate` to create
snapshots with non-monotonic `creation` timestamps but a correct,
sequential `createtxg`. It then runs `syncoid` and verifies that all
snapshots were replicated, which is only possible if they are ordered
correctly by `createtxg`.

See #815 for the original test.
2025-06-11 13:01:00 -05:00
Nick Liu 258a664dc0
fix(syncoid): Harden bookmark replication against timestamp non-monotony
* Replaced direct sorting on the `creation` property with calls to the
  `sortsnapshots()` helper subroutine. As with other usages, this
  ensures that when `syncoid` searches for the next snapshot to
  replicate from a bookmark, it preferentially uses the monotonic
  `createtxg` for sorting.
* Refactored the variables holding bookmark details from separate
  scalars (`$bookmark`, `$bookmarkcreation`) into a single hash
  (`%bookmark`). This allows for cleaner handling of all
  relevant bookmark properties (`name`, `creation`, `createtxg`).
* Fixed a code comment that incorrectly described the snapshot search
  order. The search for a matching target snapshot now correctly states
  it proceeds from newest-to-oldest to find the most recent common
  ancestor.
2025-06-11 12:40:57 -05:00
Nick Liu be52f8ab1e
feat(syncoid): Sort snapshots reliably using the `createtxg` property
System clock adjustments from manual changes or NTP synchronization can
cause ZFS snapshot creation timestamps to be non-monotonic. This can
cause `syncoid` to select the wrong "oldest" snapshot for initial
replication or the wrong "newest" snapshot with `--no-sync-snap`,
potentially losing data from the source to the target.

This change adds the `sortsnapshots()` helper that prefers to compare
the `createtxg` (creation transaction group) property over the
`creation` property when available. The `createtxg` property is
guaranteed to be strictly sequential within a ZFS pool, unlike the
`creation` property, which depends on the system clock's accuracy.

Unlike the first iteration of `sortsnapshots()` in #818, the subroutine
takes a specific snapshot sub-hash ('source' or 'target') as input
rather than both, ensuring createtxg comparisons occur within the same
zpool context. The first iteration that took the entire snapshots hash
had no mechanism to sort target snapshots, which could have caused
issues in usages that expected target snapshots to be sorted.

Most snapshot sorting call sites now use the new `sortsnapshots()`
subroutine. Two more usages involving bookmarks are updated in a
different commit for independent testing of a bookmarks-related
refactoring.

Fixes: #815
2025-06-11 12:25:14 -05:00
Nick Liu a6d417113b
fix(syncoid): Rework snap/bookmark fetching with feature detection
The original snapshot fetching relied on a complex state-dependent
`getsnaps()` subroutine with a separate `getsnapsfallback()` for older
ZFS versions. The first refactor attempt in #818 simplified this but
introduced performance regressions by using `zfs get all`, which was
inefficient for large datasets.

This commit avoids that overhead by integrating proactive `zfs get`
feature detection through a new `check_zfs_get_features()` subroutine
that determines the command's capabilities by testing for `-t` (type
filter) support and the availability of the `createtxg` property.
Results are cached per host to avoid redundant checks.
`check_zfs_get_features()` came from the #931, which this change
supersedes.

The `getsnaps()` and `getbookmarks()` subroutines now use this
information to build optimized `zfs get` commands that query only
necessary properties. As before in #818, the parsing logic is refactored
to populate property hashes for each item, eliminating the old
multi-loop state-dependent approach and the need for mostly duplicated
fallback logic.

This resolves both the original complexity and the performance issues
from the first attempted fix. Now there is a foundation for fixing the
snapshot ordering bug reported in #815.
2025-06-11 12:10:59 -05:00
Jim Salter 8d4abf14b2
Merge pull request #1006 from phreaker0/prepare-2.3.0
prepare 2.3.0
2025-06-11 08:10:52 -04:00
Christoph Klaffl becec66320
prepare v2.3.0 2025-06-09 23:21:44 +02:00
Adam Fulton aa2c693e62
fix(syncoid): regather $snaps on --delete-target-snapshots flag 2025-06-05 23:31:47 +02:00
Christoph Klaffl b794da6f14
Revert "Merge pull request #818 from Deltik/fix/815"
This reverts commit 7c225a1d7b, reversing
changes made to acdc0938c9.
2025-06-05 23:23:30 +02:00