Hopefully this helps someone down the road determine if syncthing is for them. I was skeptical. But so far (1.5 weeks later) I’m convinced. it’s a great tool, maintained by a group of obviously highly skilled developers, and nice helpful people on the forums.
I have four distinct use cases:
- User data sync (dotted black arrows);
- Data backup into a “live” environment (red one way arrows);
- Photo and video editing without network lag (yellow-ish arrow);
- Phone camera backup (solid black arrows).
Use case 1: user data sync
- I kept running out of space on onedrive, and don’t want to pay for more. Same for any of the other tools that sync user data (think Windows folders: documents, downloads, picture etc.). I found data transfers to be slow (we live rurally and internet service is high speed but not FAST - uploads take a while). I have a 42 TB primary server that I want to use.
- I find with syncthing (running as synctrayzor on all windows machines) the transfer is nearly instant so that within 10-15 seconds all items are present on all machines. Using a central server I’m able to version on the server and not on the client machines, thus saving space on limited size OS SSDs.
- synctrayzor only runs when the user is logged in, and is user specific. setting up separate folders on the server allows syncing between the same user on different machines, but with carefully controlled sharing and careful folder names (!!) you can be sure not to cross between logins.
- You also gain the ability to share ACROSS users on windows machines if you want to - for instance my wife and I have separate logins for windows, but I could share the server “photos” folder under me with her login as well.
- Additionally if I want to I can drop something into the server folder and next time the client logs in, the sync will pull down the file onto the local drive. I’ve used this when the dance teacher shared music with the parents for the routines to practice at home. I put the file on the server and both the kids have access to it on whatever laptop they’re using that day.
Use case 2: data backup into “live” environment
- I used to run my backups from the always on VM (contained on the primary server) to push files around. It worked great for years, but I was never entirely happy as there were occasional permission issues, disconnected drives or share issues, and it just felt “clunky”.
- I am running syncthing in linuxserver’s docker container on my unraid servers. Primary server is 42TB, backup is 22TB. As you can see from the diagrams above, there is a bit of folder nesting going on. But since I want everything that matters captured (the entire “private” share, for instance), I have a one way send on the primary to a send/receive folder on the secondary (more on that in a second).
- The backup server must be “live”, in that I should be able to turn it on whenever I want and have a functioning version of plex, and access to all my files (that matter).
- The backup server is set to wake up in the early morning hours. When it does, the syncthing docker automatically starts, scans, syncs, and then shuts itself off once the drives spin down 30 minutes later. Every day. So I have a live version but it’s not burning any electricity most of the time.
- Version is set to a longer time on the backup than on the primary (30 days vs 5 days) so if something goes wrong I’ll still have it on the backup.
- I used send/receive on the backup server so that I could control from the primary if something goes wonky on the backup. the primary is ALWAYS right. So rather than having to log into the backup to “revert changes”, I can do it form the primary by “override changes”. Forgive me if my verbiage isn’t right, I’ve only seen it once or twice.
use case 3: photo and video editing
- we take lots of pictures, and lots of video. My big pain was always waiting for video to load over the network (sure it’s gigabit, but still have to wait). the phones now back up and sync to my workstation hard drive (a separate 2TB physical spinner) where I do all my editing and review. Thumbnails generate almost instantly and there is no lag when playing video. Renames are instant, then reflected across the remainder of the syncs. then I copy back to the proper network share location (fully organized by person, date, etc. in folders…) so they are available to family elsewhere running plex. Once it’s moved into it’s final place, … (see use case 4)
- version cleanup is set to 180 days on the local hard drive because I have the space. It’s a second to last ditch rescue in case something goes horribly wrong.
use case 4:
- I wanted a robust backup solution for the phone camera and video. I had used “foldersync” for years on the android devices and it was always pretty good, but not “fantastic”. Sometimes it would fail, or something would go wrong (like android would put it to sleep), and it relied on the home wifi connection to work, and the two way sync was scary. So now I have syncthing running on the androids and it’s so far been flawless. Once I"ve finished editing the pictures (use case 3), and move them into the final proper spot on the server, they get cleared off the phones.
- we also use google photos so all the history is still there on the devices, just not locally.
offsite backup
- I am paranoid. I’ve lost data and pictures once before, in 2001. the next day I bought a new computer with a raid5 card and 4 hard drives. spent a fortune. Ever since I’ve always looked for great ways not to lose the stuff that matters. In addition to everything I’ve described I backup nightly to a cloud backup provider. Their program runs on the Windows VM and does its thing every night. In the morning I check my email and see that it ran. It’s important.
a final note
- The main point is that syncthing makes it easy to set up, easy to monitor, and easy to automate across all sorts of devices. I have 4 use cases, and i’m sure there are many many more. But so far syncthing is able to handle everything I’ve thrown at it, and the errors I experienced were because I didn’t know what I was doing. I had issues with nesting and syncing the syncthing folders. sooo much great info in this thread: complex sync structure - bad? that it doesnt’ need to be repeated here.
- I’ve also set the rescan interval to one day (86400 seconds) rather than the defaul 1 hour. Most things are captured by the watching, and anything that’s not will be caught up later. Plus it lets the drives spin down to save some energy - important when 19 drives are busy spinning a way under the stairs…
I wish you all the best of luck with it, and have a great day! Tiwing.
1 post - 1 participant