In the XXI century, storage is a big deal. Preserving personal data and memories is hard, but it is hell considering privacy and ethics of possible solutions. Naturally, I am very interested in this, while being no developer, nor engineer, nor having enough time to dedicate to this issues, therefore I will be noting some thoughts here.
- Durability: I want to be able to effortlessly access all of my data now as 30 years from now.
- Reliability: my storage system has to be absolutely fail-proof (of course, no system is. I need to achieve the best reliability I can afford). I cannot lose any of my data again.
- Scalability: the amount of pictures collected and stored by everyone in the world is growing at a tremendously fast pace, my pictures are no exception. I would prefer not to end up stuck in a resources consuming framework which is great with 2TB of pictures, but makes 10TB of pictures unmanageable.
- Affordability: I should not spend too much on this. In the end, shooting pictures is not my job (yet).
- Accessibility: if possible, I would like to be able to access all of my pictures from anywhere anytime.
- Portability: the system I choose must be as bare-bones and simple as possible, so that, if in the future I want to switch to a better one, I can do it as seamlessly as possible.
- “Replicability”: why should my system work for me only? It would be ideal, by documenting everything, to make my final solution “replicable”, hence available to anyone as a source of inspiration. I am writing these words for this reason.
- Can I rely on Flickr to Piwigo alone to host all of my JPGs, even my private ones? No
- is it reliable? How much do I risk of losing my data? Not too much. Not more reliable than any server around the world.
- is it secure? Are my private pictures somehow server-side encrypted? No encryption. Not the best option to backup pictures
- Does it make sense to use my Cubbit's 4TB for long-term picture storage? It does, as long as some features will be included
- Should I use portable Hard Disks, as the ones I already have, or internal hard disks to plug to Linuxplosion? None of the two, if the Cubbit solution works
- +++ Linking Ecologia Digitale article
The solution I found out to be working greatly up to now is the one described in Importing workflow, which consists in setting a window of a few hours per week while Linuxplosion is available to synchronize pictures and run tasks via SSH. In this way, storage is safely and redundantly backed up at home. This solution requires too much maintenance time and effort, it is fragile and unreliable on the go.
Cubbit is the most promising long-term solution, but it still lacks some crucial features. Furthermore, it is not optimized for pictures (yet): albums and photo galleries are limited to folders and sub-folders, there is not even any support for tags.
Photoprism is arguably the best Google Photos alternative. Nevertheless, it requires a relatively powerful hardware to run. It is the best possible solution in terms of usability, but not in terms of costs and eco-friendliness. Some installation resources
- This lecture on Backups, from MIT’s Missing Semester is useful to understand the reasoning behind safe backups. Note that what The Missing Semester evolves around is code, hence small if not tiny plain text files, while our interest is in huge amounts of data, such as high quality videos and images.
- An engineer’s guide to cloud capacity planning
On Linuxplosion, I am considering to configure a RAID 1 storage system. Below, some notes about this.
- 💽 How to Setup Software RAID with MDADM Command on Linux Ubuntu in 2021 💾, a YouTube video of September 2020
- Ubuntu 20.04 Mdadm Setup Guide a YouTube video
!Personal storage management notes