Offsite Backups

No matter how modern and redundant the data centre: in addition to the host's backups, customers should always run their own offsite backups.

A cautionary tale: the OVH data centre fire

In 2021, the SBG2 data centre of French provider OVHcloud burnt down. Thousands of servers, data of countless customers: gone. Anyone without a geographically separated backup location had a serious problem. As a hoster, we could only too easily relate to what those colleagues and their customers were going through.

What we recommend

• Customer-owned backup in addition: The hoster runs backups, but the customer should always keep an independent backup copy. That way the data is still reachable when the hoster account gets locked, the hoster goes bankrupt, or the hoster's data centre is gone.
• Geographic separation: Backups don't belong in the same building as the primary systems.
• Client-side encryption: Data leaves the controlled zone encrypted, so the backup provider cannot inspect it.
• Automated and monitored: Unverified backups aren't backups. Monitoring must detect when a backup fails or shrinks unexpectedly.
• Practice the restore: A backup that has never been restored is a wish, not a guarantee.

How we implement this

Which backup provider and which tool fit depends on data volume, compliance requirements, RTO/RPO and, not least, on cost. As an example: for backups within Switzerland we like to use Infomaniak Swiss Backup as the destination. The exact destination is secondary: what matters are the properties listed above.

LFOps provides the linuxfabrik.lfops.duplicity Ansible role: file-based, encrypted backups using duplicity to an object store, plus the duba wrapper script for parallel backups.

For pure object-store-to-object-store sync, tools like rclone or aws-cli are widely used outside of LFOps.

More on this

• OVH fire: Heise report (German)

We can help

Need help building a geo-redundant backup strategy? Have a look at our Service & Support plans and get in touch.