While reading many of the blogs and posts here about self hosting, I notice that self hosters spend a lot of time searching for and migrating between VPS or backup hosting. Being a cheapskate, I have a raspberry pi with a large disk attached and leave it at a relative’s house. I’ll rsync my backup drive to it nightly. The problem is when something happens, I have to walk them through a reboot or do troubleshooting over the phone or worse, wait until a holiday when we all meet.
What would a solution look like for a bunch of random tech nerds who happen to live near each other to cross host each other’s offsite backups? How would you secure it, support it or make it resilient to bad actors? Do you think it could work? What are the drawbacks?
Syncthing is not a backup tool and may very well destroy all your data on its own (though this is rare).
I have local incremental backups and rsync to the remote. Doesn’t syncthing have incremental also? You have a good point about syncing a destroyed disk to your offsite backup. I know S3 has some sort of protection, but haven’t played with it.
They do have versioning: https://docs.syncthing.net/v1.27.7/users/versioning
Of course, you actually have to use that, it has to work, and you have to have a strategy for reverting the state (I don’t know if they have an easy way to do that – I’ve never used the versioned side of things).
I have had some situations where Syncthing seems to get confused and doesn’t do its job right. I ran into this particularly with trying to sync runelite configurations and music. There were a few times I had to “force push” … and I vaguely recall one time where I was fighting gigs of “out of sync” in both directions on something and just destroyed the sync and rebuilt it to stop … whatever it was doing.
Don’t get me wrong, it’s a great tool for syncing things between computers; but I would not rely on it for backup (and prefer having a backup solution on top of the synced directories). There are real backup tools out there that are far better suited to this sort of thing. I suggested Kopia, you should get some integrity checking using its builtin sync (as it won’t be able to figure out what to sync if your origin is corrupted); you won’t get that with a straight up rsync or a syncthing, they’re not application-aware enough to know they’re about to screw you over.
Restic has a similar feature but I’ve always found Restic’s approach much more frustrating and not-at-all friendly for anyone less than a veteran in systems administration. Kopia keeps configuration in the repository itself, has a GUI for desktop use that runs jobs for you automatic, automatically uses the secrets manager appropriate for your operating system, etc … Restic you kind of have to DIY a lot of basic things and the “quick start tutorial” just kinda ignores these various concerns.
Even if you plan to just use cron jobs, Kopia will do sane things with maintenance. Restic last I checked you still need to manually run maintenance tasks and if any job maintenance or otherwise fails, you need to make sure to unlock the repository (which if you haven’t set up notifications … well now you’ve got a silent backup failure and your backups aren’t running).
I just kept running into a sea of “oh this could be bad” footguns with Restic that made me uncomfortable trusting it as my primary backup. I’m sure Restic can be a great tool if used in expert hands with everything appropriately setup; but nobody tells you how to do that … and I get the feeling a lot of people are unaware of what they’re getting into.
The folks making Kopia … they seem like they really know what they’re doing and I’ve been very happy with it. We’re moving from rsnapshot to Kopia at work now as well (rsnapshot is also fairly good you’ve got a bunch of friends with NASes that support hard links and SSH, but it’s CHATTY and has no deduplication, encryption, data integrity verification is basically left to the file system – so you better be running ZFS – etc).
Duplicati’s developer is back too, so that might be something to keep an eye on … but as it stands, the project has been bit rotting for a while and AFAIK still has some pretty significant performance issues when restoring data.
Oh, fair point. Perhaps rclone.org then! :O
rclone or rsync is probably better but see my reply a few comments down (the very long one) about protocol aware cloning vs just cloning things at the file system