Implementing 3-2-1 Backup Strategy with Raspberry Pi for Time Machine and Sync to Cloud
This is a four-part guide describing a Raspberry Pi-based solution for macOS Time Machine backup with subsequent sync to cloud. It satisfies the requirements of the 3-2-1 backup strategy. Carnegie Mellon University paper is an excellent source to learn about that. Also, there is a myriad of articles available on the Internet covering this topic.
Below are a few things to get out of the way before proceeding:
I assume you have functioning Raspberry Pi hardware. I recommend Raspberry Pi 4 for improved USB and Ethernet IO performance compared to the previous generations.
You also will need an external hard drive. A spinning HDD is preferred since today majority of Macs ship equipped with SSDs, and 3-2-1 backup strategy advises using at least two different media to eliminate data loss due to similar reasons. I assume you have a way to connect the external hard drive to the host Raspberry Pi. That is a fancy way to say you need a proper USB cable.
Since we are talking about Time Machine, there should be a macOS computer in your possession with a working SSH client.
You are not afraid of working with Terminal and comfortable with the command line.
The Time Machine backups are assumed to be created encrypted so that there is no need to deal with encryption on the Raspberry Pi end or in cloud.
Overview
Below is a high-level diagram of what we are building including components and data flow:
Components
These are the garden variety of personal Mac computers in need of backing up documents, photos, music and such.
Raspberry Pi with an external hard drive attached. That is the core of the system. It accepts hourly backups from the Macs and stores them to the external drive. Then it orchestrates nightly backup data sync to cloud.
The cloud designation on the diagram is symbolic. It can be one of multiple commercially available cloud storage service providers, like Dropbox, Amazon S3, Google Drive, etc. Or it can be your friend’s file server. As long as the data is pushed somewhere outside of your own home, we are good. Having said that, I will be using “cloud” when referring to off-premisses destination.
Initially, I tested the setup with Dropbox (described in part 3), then switched to using Backblaze B2. In my experience, the system performs equally well with either cloud service.
Data Flow
Client Macs’ Time Machines are configured for hourly backups, except for the time between 1 AM and 3 AM.
The Raspberry Pi syncs its local backup data to cloud nightly at 2 AM. That is accomplished via a scheduled cron job and consists of two steps. First, a read-only snapshot of backup data is created thanks to backup volume residing on top of Btrfs - one of Copy-on-Write file systems available on modern Linux OSs. Btrfs makes this task straightforward and very fast. Then, the snapshot is synchronized with a remote cloud location using Rclone, a command-line program for cloud storage management.
Note: The 1 AM - 3 AM window with no Time Machine activity is necessary to guarantee consistent state of backup data on the Raspberry Pi before sending the data to cloud. Stopping Time Machine one hour prior to cloud sync allows for any in-progress backups to finish before 2 AM. There might be better ways to achieve the same (which I might explore in the future), but for now I am happy with the time sharing solution to this “readers-writers problem.”
Table of Contents
The remaining three parts of this guide cover the following topics:
Setting Up Raspberry Pi for Time Machine provides step-by-step instructions to configure a Raspberry Pi as a NAS recognized by macOS as a Time Machine backup destination.
Setting Up Raspberry Pi for Cloud Sync gives directions to “connect” the Raspberry Pi to a cloud provider. with Dropbox as an example. This part also covers automating snapshot creation and configuring cron.
Tuning Time Machine on Mac Clients introduces a free application for macOS called TimeMachineEditor and shows how to set it up to ensure no Time Machine activity within a certain period of time.
Let me close this intro with a quote attributed to Glenn Thomas Jacobs (aka Kane):
"Pursue your dreams but have a backup plan."
Keep it Safe. Back Up!
