Apr 15, 2020
Solving the mystery of a Master Archive & From Backup with Love.
Solving the mystery of a Master Archive & From Backup with Love, were to be the last article of three that focuses on how Qvisten Animation has built its production environment based on Open-Source software. However, as the topic Backup became a bit vast, I divided Backup and Monitoring into two. Here is, from Backup with Love.
From Backup with Love.
The first thing to consider regarding backup is how much the data is worth. You can backup everything local, on-tape, in cloud, and on an external server. But the cost is tremendous when working with terrabytes of material. I, therefore, split the process up into three categories, In-Production, Finished Production and Archive.
In-Production,
backup has the highest value and is therefore stored using the 3–2–1 backup strategy. I read up on from Backblaze.
The 3–2–1 backup strategy means that you must have three copies of the data, two local and one external. Luckily for us, our IT Partner, hovedkvarteret has multiple dark fibers to our locations, so we managed to give the strategy a little twist. Our TANK storage is therefore off-site as well at Hovedkvarteret. With locked racks, camera surveillance, and soon 40GbE Direct connection to our office.
In-Production setup.
Our production server, SPEEDY, takes daily snapshots of all on-going productions, using the Periodic Snapshot Tasks in FreeNAS. From there, I’m using FreeNAS’ Replication Tasks, to synchronize the snapshots from SPEEDY to TANK. It’s a super simple setup. It’s thoroughly explained in the FreeNAS Manual.
Now we have two copies. The next one, the third copy, is a fancy thing.
Here the production can themself decide what cloud Backup they want. Trough FreeNAS, we are using the Cloud Credentials that use rClone. It’s rsync for cloud storage, and by setting up Cloud Sync Tasks, we can send encrypted data off-site to a cloud Partner.
Finished production.
As the production is getting close to changing state to Finished production,
the main goal is to separate the data.
What are deliverables, and what is production files?
Deliverables are the data we deliver and own. Therefore it has current and historical value. Deliverables are hosted in a secure DAM system.
Production files have current value if there are any changes or fixes. But it does not have a high historical value as files do get out of date quickly.
At this time, the SPEEDY Server needs space for a new production, and as we copied snapshots from SPEEDY to TANK from day one, we have all historical and current production files on TANK.
On SPEEDY, we clean up all the snapshots and replication tasks and verifies the production files on the TANK storage server. Then we use ZFS Destroy at the dataset at SPEEDY.
In parallel, we work with the Media Manager to copy all deliverables to our DAM system.
At the end of this operation, we have only a single copy of production data, but multiple copies of the deliverables in the DAM System. When we have time, we transfer a copy of the of the latest production files to the archive.
Archive
As deliverables are hosted on the DAM system, we move only the production data to LTO Tape, for longterm archive storage. LTO is still the cheapest storage out there. It is a high startup cost for software and drives, but after that, you pay less than 150USD for 12TB storage.
Archive, price comparison.
A box of 30x LTO-8 Tapes, with 12 TB Storage each is 2500,- USD
That’s a total of 360 TB RAW Storage.A HPE StoreEver LTO-8 Ultrium 30750, costs ca 3400,- USDTotal cost 5900,- USD If you add an Open-Source Backup system like Bareos, then there is just your cost on top. ( PS: Here is a nice how-to on FreeBSD )---------------------------------
Some studios are using Archiware P5. If you want to go down that route, then there is an additional software cost around 4500 USD, then you are at 10 400,- USD---------------------------------Cost
Open-Source, self-serviced = 16.4 USD per TB.
Archiware P5, supported = 28.9 USD per TB.5 years storage time
Open-Source, self-serviced = 3,28 USD per TB.
Archiware P5, supported = 5,78 USD per TB.
AWS Glacier 5 years example
360 000 GB ( 360 TB ) Storage in AWS Glacier
costs a total of 98 594,- USD for 1825 days, ( 5 years )
