Commit Graph

948 Commits

Author SHA1 Message Date
Jim Salter 9f76aab5d6
Merge pull request #1051 from ifazk/busyfix
Fix iszfsbusy to not match pool prefix
2026-02-18 17:04:44 -05:00
Jim Salter d763e45dd4
Merge pull request #1007 from Deltik/fix/815
syncoid: Sort snapshots by `createtxg` if possible (fallback to `creation`) (redux of #818)
2026-02-18 17:02:00 -05:00
Jim Salter b4a5394d1f
Merge pull request #1079 from phreaker0/doc-weeklies
add weeklies documentation
2026-02-18 17:00:11 -05:00
Christoph Klaffl 3b606768fa
Merge branch 'master' into fix/815 2026-02-18 22:59:26 +01:00
Jim Salter 3d866c9473
Merge pull request #1054 from ifazk/unused-forcerecv
Remove unused forcerecv variable
2026-02-18 16:59:00 -05:00
Jim Salter 07e32cad71
Merge pull request #1044 from numerfolt/master
Change syncoid snapshots date and time divider from ":" to "_"
2026-02-18 16:49:55 -05:00
Christoph Klaffl 730bce6d38
add weeklies documentation 2026-02-18 15:20:23 +01:00
Jim Salter 1ae8f26e82
avoid bug with snapshots named 0
Updated error handling to check for empty string instead of false for snapshot retrieval.
2026-02-18 00:19:37 -05:00
Jim Salter d9fe50b564
Merge pull request #1074 from ifazk/create-bookmark-last-snapshot
Fix bookmark last snapshot sync early exit
2026-02-17 13:04:37 -05:00
Jim Salter d8ff385691
Change comparison from '==' to 'ne' for error checks
Zpool status reports error counts with unit suffixes after the first 999 errors in any given column; as originally written, the monitor-health plugin used strictly numeric comparison checks which would error out when encountering a value like "1.09K" for checksum errors.
2026-02-17 11:53:42 -05:00
Ifaz Kabir 89d3cbcda3 Fix bookmark last snapshot sync early exit 2026-02-17 10:37:28 -05:00
Jim Salter 74b61d149f
Enhance disk space check script functionality
Updated the script to accept warning and critical levels as decimal arguments. Added support for excluding 'squashfs' filesystems from the df command output.
2026-01-19 23:51:15 -05:00
Jim Salter 536cceea95
Enhance comments for check_all_disk_space script
Updated comments to clarify script functionality and usage.
2026-01-19 23:30:35 -05:00
Jim Salter 361e6839d1
Add disk space check script for ZFS filesystems
This script checks the available disk space on all mounted filesystems, producing Nagios-ready exit codes and brief, human-readable text.
2026-01-19 23:23:34 -05:00
Ifaz Kabir 225c9f99d1 Remove unused forcerecv variable 2025-12-10 21:10:32 -05:00
Ifaz Kabir 4543705ffc Fix iszfsbusy to not match pool prefix 2025-12-03 23:49:22 -05:00
numerfolt 4c9fba2277
Change date and time divider from : to _ 2025-09-29 22:04:29 +02:00
Jim Salter dbcaeef1ac
Remove Alpine Linux ps installation instructions
Removed instructions for installing ps on Alpine Linux.
2025-08-24 17:30:21 -04:00
Jim Salter 2c98f0501c
Merge pull request #634 from asche77/patch-1
Update INSTALL.md to include OmniOS installation instructions
2025-08-24 11:29:35 -04:00
Jim Salter 0f3a9c94d9
Merge branch 'master' into patch-1 2025-08-24 11:28:40 -04:00
Jim Salter a6728e48de
Merge pull request #996 from jjakob/fix_directtimeout
syncoid: fix directtimeout in directmbuffer mode
2025-08-24 10:56:44 -04:00
Jim Salter 393a4672e5
Merge pull request #1023 from phreaker0/fix-file-handle-conflict
fixed file handle conflict
2025-08-24 10:50:58 -04:00
Christoph Klaffl efd52f416d
fixed file handle conflict 2025-08-12 14:19:47 +02:00
Nick Liu 1952e96846
fix(tests/common/lib.sh): Support set -e in test scripts
The `systemctl is-active virtualbox-guest-utils.service` command returns
a non-zero exit status when the service is inactive, causing early exit
in tests that use `set -e`.

This change suppresses that error with `|| true` to prevent test failure
when the service check returns non-zero.

Fixes this test:

```
root@demo:~/sanoid/tests/syncoid# cat /tmp/syncoid_test_run_011_sync_out-of-order_snapshots.log
+ set -e
+ . ../../common/lib.sh
+++ uname
++ unamestr=Linux
+ POOL_IMAGE=/tmp/jimsalterjrs_sanoid_815.img
+ POOL_SIZE=64M
+ POOL_NAME=jimsalterjrs_sanoid_815
+ truncate -s 64M /tmp/jimsalterjrs_sanoid_815.img
+ zpool create -m none -f jimsalterjrs_sanoid_815 /tmp/jimsalterjrs_sanoid_815.img
+ trap cleanUp EXIT
+ zfs create jimsalterjrs_sanoid_815/before
+ zfs snapshot jimsalterjrs_sanoid_815/before@this-snapshot-should-make-it-into-the-after-dataset
+ disableTimeSync
+ which timedatectl
+ '[' 0 -eq 0 ']'
+ timedatectl set-ntp 0
+ which systemctl
+ '[' 0 -eq 0 ']'
+ systemctl is-active virtualbox-guest-utils.service
inactive
+ cleanUp
+ zpool export jimsalterjrs_sanoid_815
+ rm -f /tmp/jimsalterjrs_sanoid_815.img
```
2025-06-18 00:15:44 +02:00
Jim Salter 940a84e21f
Merge pull request #1008 from aabccd021/master
Fix readme formatting
2025-06-12 09:53:57 -04:00
aabccd021 680194fa33 Fix readme formatting 2025-06-12 12:58:26 +07:00
Nick Liu f2e1e4f8a0
test(syncoid): Add test to verify out-of-order snapshot sync
This commit adds a regression test for the out-of-order snapshot
replication issue.

The new test case manipulates the system clock with `setdate` to create
snapshots with non-monotonic `creation` timestamps but a correct,
sequential `createtxg`. It then runs `syncoid` and verifies that all
snapshots were replicated, which is only possible if they are ordered
correctly by `createtxg`.

See #815 for the original test.
2025-06-11 13:01:00 -05:00
Nick Liu 258a664dc0
fix(syncoid): Harden bookmark replication against timestamp non-monotony
* Replaced direct sorting on the `creation` property with calls to the
  `sortsnapshots()` helper subroutine. As with other usages, this
  ensures that when `syncoid` searches for the next snapshot to
  replicate from a bookmark, it preferentially uses the monotonic
  `createtxg` for sorting.
* Refactored the variables holding bookmark details from separate
  scalars (`$bookmark`, `$bookmarkcreation`) into a single hash
  (`%bookmark`). This allows for cleaner handling of all
  relevant bookmark properties (`name`, `creation`, `createtxg`).
* Fixed a code comment that incorrectly described the snapshot search
  order. The search for a matching target snapshot now correctly states
  it proceeds from newest-to-oldest to find the most recent common
  ancestor.
2025-06-11 12:40:57 -05:00
Nick Liu be52f8ab1e
feat(syncoid): Sort snapshots reliably using the `createtxg` property
System clock adjustments from manual changes or NTP synchronization can
cause ZFS snapshot creation timestamps to be non-monotonic. This can
cause `syncoid` to select the wrong "oldest" snapshot for initial
replication or the wrong "newest" snapshot with `--no-sync-snap`,
potentially losing data from the source to the target.

This change adds the `sortsnapshots()` helper that prefers to compare
the `createtxg` (creation transaction group) property over the
`creation` property when available. The `createtxg` property is
guaranteed to be strictly sequential within a ZFS pool, unlike the
`creation` property, which depends on the system clock's accuracy.

Unlike the first iteration of `sortsnapshots()` in #818, the subroutine
takes a specific snapshot sub-hash ('source' or 'target') as input
rather than both, ensuring createtxg comparisons occur within the same
zpool context. The first iteration that took the entire snapshots hash
had no mechanism to sort target snapshots, which could have caused
issues in usages that expected target snapshots to be sorted.

Most snapshot sorting call sites now use the new `sortsnapshots()`
subroutine. Two more usages involving bookmarks are updated in a
different commit for independent testing of a bookmarks-related
refactoring.

Fixes: #815
2025-06-11 12:25:14 -05:00
Nick Liu a6d417113b
fix(syncoid): Rework snap/bookmark fetching with feature detection
The original snapshot fetching relied on a complex state-dependent
`getsnaps()` subroutine with a separate `getsnapsfallback()` for older
ZFS versions. The first refactor attempt in #818 simplified this but
introduced performance regressions by using `zfs get all`, which was
inefficient for large datasets.

This commit avoids that overhead by integrating proactive `zfs get`
feature detection through a new `check_zfs_get_features()` subroutine
that determines the command's capabilities by testing for `-t` (type
filter) support and the availability of the `createtxg` property.
Results are cached per host to avoid redundant checks.
`check_zfs_get_features()` came from the #931, which this change
supersedes.

The `getsnaps()` and `getbookmarks()` subroutines now use this
information to build optimized `zfs get` commands that query only
necessary properties. As before in #818, the parsing logic is refactored
to populate property hashes for each item, eliminating the old
multi-loop state-dependent approach and the need for mostly duplicated
fallback logic.

This resolves both the original complexity and the performance issues
from the first attempted fix. Now there is a foundation for fixing the
snapshot ordering bug reported in #815.
2025-06-11 12:10:59 -05:00
Jim Salter 8d4abf14b2
Merge pull request #1006 from phreaker0/prepare-2.3.0
prepare 2.3.0
2025-06-11 08:10:52 -04:00
Christoph Klaffl becec66320
prepare v2.3.0 2025-06-09 23:21:44 +02:00
Adam Fulton aa2c693e62
fix(syncoid): regather $snaps on --delete-target-snapshots flag 2025-06-05 23:31:47 +02:00
Christoph Klaffl b794da6f14
Revert "Merge pull request #818 from Deltik/fix/815"
This reverts commit 7c225a1d7b, reversing
changes made to acdc0938c9.
2025-06-05 23:23:30 +02:00
Christoph Klaffl b9bcb6a9d3
Merge branch 'cache-add' into prepare-2.3.0 2025-06-05 22:44:59 +02:00
Christoph Klaffl f0a2b120d9
Merge branch 'blacklist-encryption' into prepare-2.3.0 2025-06-05 22:42:49 +02:00
Christoph Klaffl a546b7d162
Merge branch 'fix-warning' into prepare-2.3.0 2025-06-05 22:42:22 +02:00
Christoph Klaffl b1f191ff8f
Merge branch 'pr-964' into prepare-2.3.0 2025-06-05 22:42:10 +02:00
Christoph Klaffl 003dd4635a
Merge branch 'patch-1' into prepare-2.3.0 2025-06-05 22:40:47 +02:00
Christoph Klaffl 1915ea29a2
Merge branch 'ignore-duplicate-template-keys' into prepare-2.3.0 2025-06-05 22:40:11 +02:00
Christoph Klaffl 6bda64508b
Merge branch 'ossimoi/master' into prepare-2.3.0 2025-06-05 22:39:38 +02:00
Christoph Klaffl 5109a51b68
Merge branch 'fix-debian-package' into prepare-2.3.0 2025-06-05 22:37:29 +02:00
Christoph Klaffl 67b9dec294
Merge branch 'fix/918' into prepare-2.3.0 2025-06-05 22:37:14 +02:00
Christoph Klaffl 44a9b71d5f
Merge branch 'dev/sde/remove-force-prune' into prepare-2.3.0 2025-06-05 22:36:12 +02:00
Christoph Klaffl 27fc179490
implemented adding of taken snapshot to the cache file and a new parameter for setting an custom cache expire time 2025-06-05 21:59:30 +02:00
Jernej Jakob 749490830f
syncoid: fix directtimeout in directmbuffer mode
If --insecure-direct-connection contained 4 parts (including the
',mbuffer' at the end), the 3rd part (timeout) was silently ignored
and left at the default 60s.

Do not ignore the timeout part even in directmbuffer mode.
2025-03-31 14:54:46 +02:00
Jim Salter 6beef5fee6
Update INSTALL.md
add cd /tmp to Debian and Ubuntu package build instructions, to avoid newbies getting confused by limited permissions under eg /root

add reference to dependency on Capture::Tiny in sanoid and syncoid
2025-02-11 19:54:53 -05:00
Christoph Klaffl 7062b7347e
blacklist encryption property from preserving 2025-01-24 14:01:59 +01:00
Christoph Klaffl 4a9db9541d
fix warning in edge cases ("Use of uninitialized value in numeric comparison") 2024-12-03 19:32:47 +01:00
Christopher Morrow f4e425d682
Add Install instructions for EL9 systems
Added to INSTALL.md the command to add the `crb` repo for Rocky Linux 9 and AlmaLinux 9.
Necessary for perl-Capture-Tiny package.
2024-11-27 18:24:08 -08:00