Repository navigation
Commit 55652d4
Add --era: find changed data with dm-era instead of checksums
For keeping a copy that nobody else changes up to date (e.g. backups of
VM images) when the source filesystem sits on a Linux dm-era device.
Instead of reading both files and computing checksums, the generator
sends an "era request" -- a sum head with no strong sums, followed by
the destination file's user.rsync.era xattr ("<era> <time>
<era-device-id>") -- and the sender looks up which blocks of the source were written in
that era or later (era.c, using era_invalidate on a metadata snapshot).
It answers with block references for unchanged blocks and literal data
for the rest, so only changed source data is read and the receiver
neither reads nor rewrites unchanged blocks. The whole-file checksum
covers only the literal data, on both sides.
With every file in --era mode the sender also sends the era the
destination will be current with: the era started by the snapshot used
for it, or for a normal transfer the newest logged era. The receiver
removes the xattr before changing a file and sets it again once the
checksum matches, so an interrupted file has no era and gets a normal
update next time. Each file is therefore only re-sent the changes made
since it was itself last updated. Each era device has a log of the
eras our snapshots started and when (/var/lib/rsync/era/<device-id>,
also the lock serializing runs); a file's era is only accepted if the
log has it with the same time, so era numbers that start over after the
era metadata is recreated cannot match.
-X neither copies nor removes the xattr. A run takes at most 8
snapshots per device; after that a file reuses the bitmap of the
newest older era, a superset.
A file without an era, or whose era is from another device or not in
the log, uses normal sums. If the sender cannot handle a requested file
(not on dm-era, modified during the snapshot, ...) it declines with one
byte; the receiver leaves the file untouched and asks for a redo, which
uses the normal algorithm and never sends another era request. A
missing era_invalidate on the sender, or a destination without user
xattrs, is a hard error.
--era is only built for standard Linux (not Android or other systems)
and is needed on both sides. It implies --inplace and
--no-whole-file, conflicts with --whole-file, --sparse, --append,
--write-devices and batch mode, and requires protocol 30.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>1 parent ea64520 commit 55652d4
12 files changed
Lines changed: 1243 additions & 20 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | | - | |
| 52 | + | |
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
143 | 148 | | |
144 | 149 | | |
145 | 150 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| |||
679 | 680 | | |
680 | 681 | | |
681 | 682 | | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
682 | 690 | | |
683 | 691 | | |
684 | 692 | | |
| |||
0 commit comments