Backup speed benchmark: rsync vs borg vs restic vs kopia
Backups are important, specially when you need to recover the data. But backups also need to be fast otherwise you are consuming CPU, DISK and NET to do backup and not the relevant stuff.
Why the backups are important ?
- human errors
- disk failures (SSD, HDD, DVDs)
Which data should I backup ?
- personal/private data, photos, videos, software configurations..
- If you use Docker, persistant volumes, docker images in rare situations
Is better to do backups or snapshots or mirroring RAID-1 ?
Snapshots save the current state on a disk, backups saves a specific state on the disk and protect from disk failure (if the backup is on another disk)
Mirroring RAID-1 takes 2 equal disks and protects from one disk failure but do not guarantee a correct state per se.
Which tools should I use to do the backups ?
rsync/syncthing are just for synchronization, you need btrfs/zfs snapshots if you want to save efficiently a state
proper backup tools like kopia, restic and borg are the correct software specially if your filesystem do not support snapshots, the target disk is an untrusted space like the cloud where is better to use the encryption to access the data.
Do I need zstd compression to transfer the changes ?
It depends during my tests, many small files, small changes the compression decreased the overall backup speed.
Can I backup an online database ?
Yes but it's risky, the database should be offline or you can dump the data from an database into a file and then backup it
borg init -e none firstname.lastname@example.org:/backup/borg borg create --compression zstd,1 email@example.com:/backup/borg::Monday . borg create --compression zstd,1 firstname.lastname@example.org:/backup/borg::Tue . borg diff email@example.com:/backup/borg::Monday Tue
gui pika webui https://torsion.org/borgmatic/docs/how-to/set-up-backups/
restic -r sftp:firstname.lastname@example.org:/backup/restic init restic -r sftp:email@example.com:/backup/restic --verbose backup . restic -r sftp:firstname.lastname@example.org:/backup/restic backup .
official gui included
rsync -az --zc zstd -e ssh . email@example.com:/backup/rsync #68s on Raspberry Pi 4: 5:00 ns compress (5:08 with zstd) rsync -az --zc zstd -e ssh . firstname.lastname@example.org:/backup/rsync #3s
is Raspberry Pi 4 useful ?
The same rsync command took on a Rpi4 with m.2 disk and ethernet took 5mins instead of 28s (on a AMD minipc)