Backups

General, minimal principles

• Any backup is better than no backup. Even if you're totally confused and disorganized, at least attach a USB stick to the computer, copy important files to the stick, and put it somewhere safe.
  
  
• Organize your data and give it good names. If important data is scattered all over your disk and thumb drives etc, and every file is named something like "Document 745", and every folder is named something like "New Folder 271", you are lost.
  
  
• Know what data is important to you and where it is (what device, what folder, on paper, etc).
  
  
• Copy to at least two places, at least one of them off-site. Maybe to an external hard disk and to a cloud service.
  
  
• In general, simpler is better. I'd rather waste some space and do things the simple dumb way, instead of doing fancy incremental compressed backups and risk having them not work the way I expected. With simple methods, you easily can see what did/didn't get backed up, easily retrieve and test files, easily restore.
  
  
• If you have the space, don't overwrite old backups, just add new copies under a directory with a date-stamp in the name. If you accidentally delete a section of a file, and don't notice it for a while, you want to have an older version in backups.
  
  
• Backups should be encrypted, so someone can't steal/copy all your data. Or at least encrypt your sensitive files.
  
  
• Keep those backup devices offline (not mounted as disks) during normal use of the computer, so malware can't take out your files on main disk and backup locations at the same time. Even better, keep external devices unplugged when not using them.
  
  
• Keep main data and backups in multiple locations, so a single disaster (fire, flood, theft) can't take out your files on main disk and backup locations at the same time.
  
  
• Do the backups at a frequency that makes sense to you. If you rarely change important files on disk, maybe back up once every month or two. If you're making important changes to only a few files, maybe back up those few files every couple of days. If you're working on some enormous project such as writing a thesis or a book, maybe make special daily backups to multiple locations, and keep multiple versions.
  
  
• Test the backups every now and then. Copy all or a sample of the files to a temp directory on your main disk, and try to open a sample of the files, see if they're okay.
  
  
  
• Cloud backups are fragile in that the service could go out of business, disable your account, silently stop accepting files when you hit a limit or forget to pay, etc.
  
  
• What can fail, and how would you restore from backups ? Consider scenarios: entire system stolen, system hard disk fails, you delete a bunch of OS files and system won't boot any more, you delete a bunch of your important personal files.
  
  Lesson I learned when my laptop's charger died:
  
  Backup disks from my Linux system turned out not to be readable on Win10 systems.
  
  [Strategy 1:]
  
  Backup disks are external disks formatted as NTFS, then with a VeraCrypt container file with an ext4 filesystem inside. I was able to decrypt the container with VeraCrypt on Win10 to make local disk N: appear. But then neither Linux Reader nor Ext2Fsd nor Ext2explore on Win10 would recognize the decrypted (ext4) volume presented by VeraCrypt. Filed a bug report with Linux Reader, and within two days they had fixed the issue with their application !
  
  Looks like I should use exFAT inside (and maybe outside too). Linux kernel 5.4 is first version to officially support exFAT, with additional support coming in 5.6 or 5.7. I'm using 5.4. "sudo apt install exfat-fuse exfat-utils"
  
  [Strategy 2:]
  
  To change my backup drive to exFAT, I had to attach it to my Linux system, copy the data to elsewhere (took an hour or more), unmount the drive in file-explorer, use GParted to make one big unformatted partition on the drive, then use Ubuntu "Disks" (AKA gnome-disks) to format the partition to exFAT (edit partition type to set to "NTFS/exFAT/HPFS (0x07)" and Format Partition to Type "Other" and then exFAT). That went quickly. As usual the drive shows up as 1 TB raw and 931 GiB usable (which is 976 GB). (article)
  
  Copied some files (VeraCrypt apps for Win10 and Linux) to the partition. Then I used VeraCrypt to make an encrypted container filling the rest of the partition's space, formatted as exFAT (specified no files bigger than 4 GB, portable to multiple OS's). That took over 12 hours !
  
  [Strategy 3:]
  
  Decided I was making things needlessly complex; started over. Used GParted to leave the drive with no partitions at all. Used VeraCrypt to make an encrypted volume (on the drive /dev/sdb, not a partition /dev/sdb1), formatted as exFAT (specified no files bigger than 4 GB, portable to multiple OS's). Able to select "Quick Format", which means it was created in a matter of seconds. Mounted it through VeraCrypt. Copied my backup data onto the drive, finding that some of my node.js development libraries have symlinks in them, had to "skip all" those. Also ext4 allows chars ':"?*' in filenames but exFAT doesn't, had to fix those. (article; also Btrfs doesn't allow '/'; don't know if ZFS has restrictions) And ext4 supports symlinks, exFAT doesn't. Someone says exFAT doesn't support Linux file permissions, but I don't see how that could be.
  
  [Strategy 4:]
  
  Decided to change to Btrfs on my backup drives, and got VeraCrypt devs to add Btrfs as an option. Used GParted to leave the drive with no partitions at all. Used VeraCrypt to make an encrypted volume (on the drive /dev/sdb, not a partition /dev/sdb1), formatted as Btrfs (specified no files bigger than 4 GB, use only on Linux). Able to select "Quick Format", which means it was created in a matter of seconds. Mounted it through VeraCrypt. Copied my backup data onto the drive.
  
  ("sudo blkid" is the best way to see what type a filesystem is. In output of blkid, exFAT displays as 'SEC_TYPE="msdos" TYPE="vfat"'; NTFS displays as 'TYPE="ntfs" PTTYPE="dos"'.)
  
  Also, in VeraCrypt click the "Volume Tools ..." button and select "Backup Volume Header ..." and save the backup to somewhere else, with a decent name.
  
  [Strategy 5:]
  
  Decided I didn't care about Windows compatibility, I wanted to simplify, and I wanted stronger encryption. Changed from VeraCrypt to LUKS2, with Btrfs inside.
  
  Don't save recovery codes or backup copies of encryption keys only in the cloud account associated with the device (e.g. saving BitLocker key in Microsoft account). If something goes wrong (account disabled, or device damaged), you may get into a situation where you can't recover.
  
  

• SaaS services (O365, GSuite, etc) do not back up your data. Read their TOS; they don't promise to preserve your data. And they don't give you a "restore" button. They have backups to protect their company and keep the service running. You are responsible for backing up your data.
  
  
• Ransomware is now targeting backups. You need to properly secure your backups. They should be offline, air-gapped, encrypted. If you have a live backup server on your LAN, maybe use a different OS and different authentication from your main system.
  
  

Data you could back up

• Important files you've created on disk.
  
  
• Application internal files on disk (email data file, browser profile, password manager database, RSS reader feed list, RSS reader database, IDE config, etc).
  
  
• Standard installer for your operating system, on a bootable device.
  
  
• Image of entire disk, including operating system and applications.
  
  
• Data on cloud services (in an email service, in Facebook, in WhatsApp, etc).
  Probably the most critical data to backup is your Contacts/Friends list, so you have it in case you have to create a whole new account. To back up your Facebook Friends list, maybe go to your Friends page and then use a browser add-on such as Link Gopher to capture all the links.
  
  
• Non-digital data: paper records, ID cards, passports, access cards, financial cards, memories, procedures only you know. Digitize it.
  
  

sudo blkid | grep -v squashfs >saved.blkid.txt
cp /etc/fstab saved.etcfstab.txt
lsblk --fs --list --paths >saved.lsblk.txt
sudo fdisk --list >saved.fdisk.list.txt
sudo inxi -Fmpx >saved.inxi.txt
tar --create --file saved.dot-ecryptfs.tar ~/.ecryptfs/*
# .ecryptfs files let you use ecryptfs-recover-private to recover access

Have backups, don't just keep your data online

Google, from DrStephenPoop on reddit:

> BACK UP YOUR DATA

And not just what's on your hard drive.

Do not trust the cloud!

Google recently ended my account for an unidentified TOS violation. I am not sure what I did. I just logged into gmail one day and instead of an inbox I saw a message saying my account had been disabled. I lost:

8 years of email contacts

6 years of favorited YouTube videos

About a dozen videos I made with my brother that were uploaded to YouTube.

All my Drive/Doc files including original writing.

My passwords to several sites, including banking and insurance sites.

Three albums I had purchased from Google Play.

Here's the kicker: I was a google believer. I am one of the 5 or so non-developers who actually owns a first generation Chromebook. I believed in the cloud!

Use and enjoy Google's services, but do NOT rely on them. Even though you buy their computers and purchase music from them, you are STILL not the consumer with google. You are the product (sold to advertisers). So when you are shut out from their garden, you have no customer service to appeal to, or to even find out why you got tossed. You might as well be staring at an angel with a flaming sword, wondering where your pants are.

> Didn't you contact Support ?

When you get the "your account has been disabled" screen, they give you a link to voice your grievance. After submitting, you get a message that says something to the effect of: "If we find we have reason to contact you, we will contact you."

You can also go the community forums and plead for help. Sometimes someone associated with google will actually say: "I'll have people take a look at this." Despite all my pleas, I never got a response. That is as far as support goes. You are not a customer. You are the product, and you are merely a commodity. Have you ever heard of "commodity support"?

Tienlon Ho's "Can You Live Without Google?"
Gonzalo Sainz Trapaga's "A new and innovative way for Google to kill your SaaS startup"
Desirea Calvillo's "When You Get Locked Out of Your Google Account, What Do You Do?"

Facebook, from someone on reddit:

A few days ago my Facebook account was disabled suddenly and without warning. I've gone through what I thought was a fairly routine appeals process - filled in the form they link you to when you try to log in and included a scan of my photo ID as they requested to prove I'm a real person etc. However, I just received an email from Facebook saying the following:

> ... Upon investigation, we have determined that you
> are ineligible to use Facebook. ... Unfortunately, for
> safety and security reasons, we cannot provide
> additional information as to why your account
> was disabled. This decision is final. ...

This is really bizarre and quite upsetting - it's easy to forget just how much we rely on this service. If I can't get my account reactivated, that's six years of content (and memories) lost, and a huge blow to my ability to keep in contact with some friends and family.

The only possible reason I can think of for my account being disabled is what I was doing at the time - sending some photos to someone through the private messaging system. Some of the photos were (mildly) adult in nature (at her request!) which could be deemed a breach of the Community Standards if you look at it in strict black and white terms ("Facebook has a strict policy against the sharing of pornographic content"). However I can't bring myself to believe that there is someone monitoring private message attachments and instantly banning people if they see boobs. Beyond that, I genuinely can't conceive of a reason as to why my account was singled out for anything.

Any advice would be appreciated as to what I should do next - I am not yet willing to just give up and lose all of that content. I have replied to the email, though I doubt anyone will read it, but beyond that there's really no other contact options I can see, and Googling this problem does not produce much beyond more horror stories like this.

Google from /u/sugarbreach on reddit:

I am writing this to warn Google users to back up their data, and to realize that everything you take for granted can be taken away in an instant.

About a week ago I attempted to log into my Gmail account and was greeted with a page saying my account was disabled. It says that it was disabled due to a perceived violation of the terms of service and product specific polices. I have read and reread the google terms of service, and I know I haven't done anything to violate them. The only possibility I can think of is that someone may have hacked into my account. I have been an enthusiastic gmail user since it first came out in beta, and you had to be invited to get an account. I have relied on google apps to make my life easier. I have filled in their account recovery form, and even tried calling members of the Gmail team, but have had no luck. I also have posted on the gmail help forum, but an expert there said he contacted google and there was nothing he could do and google wouldn't tell him anything "for privacy reasons".

This has created the ultimate real-life nightmare, and has turned my life upside down, a few examples of which are listed below.

All of my contacts were linked to this account. I now do not have access to emails, phone numbers, addresses, etc.

My google voice telephone number is no longer working. I had this phone number on my business cards and email signature, and now when someone dials the number, they are given an error recording. "We could not complete your call, please try again".

My youtube account with many videos I cherished of my children are now gone.

I have all of my photos backed up to the account for nearly my entire life, as I thought this was the safest place to keep them (the cloud!) I have photos of my beloved grandparents who have since passed away, and the thought that I can no longer access these photos makes me sick. I also have thousands of pictures from vacations and of my children that I fear are gone forever.

A nice chromebook that I purchased to access all of the google apps is now almost useless since my account has been disabled.

I have multiple documents in my google drive that I have spent hours of work on, and can no longer access them.

I placed an enormous amount of faith and trust into google's products and services, as millions of people have worldwide. It is a shame that something this important in someone's life cannot even warrant a response from a live person at Google.

I have been very depressed because my entire life was encased in google's products, and now everything is gone.

Again, I am writing this to warn others that this can happen to anyone at any time, so it would be wise to back up treasured items in your google account. Ironically, google provides the means to do this through their "takeout" app, which I did not learn about until after my account was disabled. If there is anyone out there reading this that can offer any guidance for getting my account reinstated, I would sure appreciate it!

Apple, from someone on reddit:

[To someone who lost files:]

Apple has two backup options, iCloud and iTunes. iCloud backups generally do not backup content (see section 3 re: Your iCloud Backup includes information about the content you buy, but not the content itself.) from apps, including Books, but they do backup your purchase history. If any [lost]] files were purchased through Apple, you can likely recover them by re-downloading them. Files you manually store or re-direct from your Mac or iOS device in iCloud are not considered to be using iCloud backup. They are stored in iCloud drive, but they do not recover automatically as part of Apple's restore process.

Conversely doing a backup via iTunes completes a snapshot of the device in the moment, including current content. If you set up a backup cycle, iTunes uses a feature called incremental backups to ensure future snapshots only capture data that's been changed or added since the previous backup. For the future, using both in conjunction with one another is key to ensuring your data is backed up. Apple's iCloud service is useful in some regards, but on mobile devices it's only asset is syncing purchase history and device settings.

Dashlane, from someone on reddit:

Dashlane Deleted Me. A Cautionary Tale:

Consider this a word of warning to Dashlane users or those considering it.

We all have a lot of passwords to remember, right? Why not an application to remember them. You know, for all us folks out there with memory problems, or enough to keep track of already. Seems like the perfect solution, no? Not so fast:

I have an account with Dashlane, or should say they have my account. All of my accounts. I have no access. I have recently had to reformat my computer due to Windows 10 file corruption (that's another story). In the process I lost my backup passwords list. Now I am locked out of all of my important emails and accounts because Dashlane did not recognize my device. I am unable to receive the authentication token required to log in because Dashlane has all my passwords. Thus, I am unable to sign in to the email account. I have tried to recover my email account as well, to no avail (yet another story). I have tried my best to answer their customer support questions via email; one I just made for the purpose, to the best of my knowledge, and submitted my ID drivers license. All of which their customer support found unsatisfactory. You need to remember quite a bit of the top of your head and have a memory like a steel trap to be authenticated. Short of a urine sample, or blood draw, I am unsure of how I can prove I am who I say I am. (Sorry for the dramatism, but it is extremely frustrating)

I know my circumstances may seem a bit extraordinary. The perfect storm, if you will. however, I am certain I am not the only one. Nor will I be the last. Nevertheless, now I am locked out of all other accounts, financial and social. All due to not having access to my passwords. And absolutely no further recourse. I am exploring legal recourse, and I will update you on this when I'm done talking with my lawyer. This is financially devastating. So consider this a word of caution, users. Don't fall as hard as I am falling right now. I have been digitally deleted, all in the name of and despite security reasons. So don't make the same mistake I did, I recommend writing down your passwords and keeping them in a safe place.

Credit-card chargebacks on Google and others, from justAnotherLedditor on reddit:

So early Black Friday sales happened last month and I picked up a Google Pixel 7 since my previous phone was nearing 6 years old and starting to die every few hours.

Due to some funky error, whether I accidentally put two phones in the cart, I don't know or remember. I ended up getting double-charged and realized I got shipped two phones.

I contacted Google Support to start a return for a refund on one of them, and the first support person was great ... up until the next dozen support staff throughout this stupid journey.

Turns out that the package I shipped back to them never made it back. I spoke with support and I got the most generic responses ever from a person that doesn't speak English (once they stopped making generic replies, it was quite evident).

They escalated the problem to a supervisor. The supervisor told me that they would do an investigation, would take about a week.

Beginning of this week, investigation ended. They say the package was indeed most likely lost but the representative I spoke to said I could just chargeback with my credit card. So I did.

Today, my Google account was banned. 15 years of history gone.

I went on the support chat for the umpteenth time and they told me because I did a chargeback, the rules are that my account will be banned. I asked why they suggest for me to do a chargeback, when they could have just refunded themselves, and they said the support I spoke to should never have suggested it but rules are rules.

Been trying to fight this but looks like Google support is utter trash. After looking online, it seems like this is their most stupidest policy, and it exists across most other platforms too. [Other people say same at Steam, EA.]

Jon Christian's "Deleting the Family Tree"
DanDeals' "PSA: Don't Mess With The Google!"
Alex Hern's "Pixel phone resellers banned from using Google accounts"
"A few reasons not to organise on Facebook"
Killed by Google

Matthew Miller's "SIM swap horror story: I've lost decades of data and Google won't lift a finger"
David Murphy's "I Lost Nine Years of Photos by Locking Myself Out of My Google Account"
Leo Notenboom's "A One-step Way to Lose Your Account ... Forever"

Paraphrased from someone on reddit 11/2019:
"As a prank, a friend changed the name of our WhatsApp group to something obscene. WhatsApp then banned the group and the accounts of everyone in the group ! My account has been banned !"
[Related: don't let unknown people add you to groups; you could get suspended or banned for being added to a malicious group. In Android app, relevant setting is Settings / Account / Privacy / Groups.]

Paraphrased from someone on reddit 12/2019:
"My Facebook account got banned (maybe for creating two accounts ?), and then a week later my WhatsApp account got banned too, I assume because my Facebook account got banned."

What happens to your Android phone if your Google account gets banned ? Your phone is logged in to that account all the time.

New Google TOS quoted by someone on reddit 11/2020:
"If your account is inactive in Gmail, Drive or Photos for more than two years, Google 'may' delete the content in that product. So if you use Gmail but don't use Photos for two years because you use another service, Google may delete any old photos you had stored there. And if you stay over your storage limit for two years, Google 'may delete your content across Gmail, Drive and Photos.'"

Some photo-storage or photo-gallery sites feel free to downgrade the resolution of large photos or all photos, assuming that human viewers won't be able to see the difference. Know the exact policy of the site before using it, if this matters to you. The downgrading may happen as owner uploads, or may happen when a user "views" and/or when the owner "exports". You may have to test carefully to understand what is done.

Cloud-connected devices maybe can be remote-wiped or have files deleted by the manufacturer.
WD My Book Live disaster



If you lose a cloud account, you can lose stored data, your calendar, remaining time on a subscription, any accumulated credit or "reputation" or gift cards, network link that makes some device (such as Amazon Echo, Google Home, etc) work, playlists, contact list, media you had bought or stored there, etc.

Do NOT use Facebook or Google or Apple or Microsoft as your login to lots of other web sites. Not only does it let your activity get shared to Facebook or etc, but if Facebook or etc ever deactivates your account for some reason, you've lost access to those other sites too.

Do NOT use Google's online password manager (holding passwords you've saved in Chrome or Android). If Google ever deactivates your account for some reason, maybe you've lost access to those other sites too, I'm not sure.

Do NOT use Facebook or Google or Pinterest or Amazon or etc as the sole, critical host of your business, if you can avoid it. They give the "appearance of ownership", but in fact you do not own the platform, you have "digital tenancy". If the service ever deactivates your account for some reason, your business is dead. And content you write on them (in FB Pages, Amazon items for sale, etc) probably is in a non-standard format and hard to move to elsewhere. If you absolutely must use such a service as your critical host, plan for the possibility that they may drop you. Keep backups, have a separate web site and email, have pages on other services, etc.

Do NOT rely on a high page-rank in Facebook or Google, or a high reputation rating in Amazon or iTunes or YouTube or AirBNB or Yelp or something, as the critical asset of your business, if you can avoid it. The algorithms behind those can change at any time. A couple of bad reviews from users can harm you greatly.

Do NOT use a free email account supplied by your ISP or cell-phone service provider. If you ever change service provider for some reason, you may lose that email account.

Maybe some people don't consider their email/messenger to be "cloud data", but it is. If you're saving 10 years of past messages in GMail or WhatsApp or something, it may be valuable to you, and it may be used or deleted by a hacker if your account gets hacked. It also may be hard to back up, and may be hard to move to elsewhere. I'm a big believer in keeping your email account as close to empty as feasible. Clean it out !

Alberto de Murga's "How to back up your Git repositories"

If you're running a business on a cloud service (Facebook, eBay, Shopify, Etsy, GMail, Amazon, AirBNB, etc), back up your data. The service may or may not be backing it up for you. Even if they are backing it up, getting it restored may take a while. And if they turn off your account for some reason, you need that data so you can move to another platform and continue to serve your customers. These services give the "appearance of ownership", but in fact you do not own the platform, you have "digital tenancy". If there's a way to use a custom domain name that you own, that's safer than using one provided by the service: if the service fails then you can make the domain name point to some new server. Same is true of a phone number, especially a VOIP number: you don't really own it, the provider owns it, and you can lose the number through disuse or failure to pay or some other mishap.

Do you actually "own" the things you think you own ? If a friend set up your domain registration or email account for you, is it in their name or yours ? If an employee administers the company email accounts on GMail, is the employee's personal account the only administrator for the whole company ? If someone gave you a used computer or phone or something, whose name is on any accounts or subscriptions associated with it ? If your relationship with your spouse or partner is failing, whose name is registered as the owner of various accounts ?

If you do lose access to something important, be wary of threats in search results. Lots of sites have been set up to provide "Facebook Support phone number" or "Unlock your banned WhatsApp account" or similar in search-engine results. But these big vendors with free services (Google, Facebook, WhatsApp, etc) deliberately do not HAVE a phone support number you can call. They have hundreds of millions or billions of free users; the LAST thing they want is for users to be able to call humans at their company. Any search result that gives you such a phone number is trying to connect you to a scammer. At best, they'll try to sell you something. At worst, they'll install ransomware, steal your money, and sell your information.

Other things to back up

Do "backups" of old non-electronic data, such as family photos and diplomas and such. Scan them and back up the images.

From Justin Carroll on an ITRH podcast:

Kinds of information (for you and everyone in family, and pets) you should have backed up and available (carry with you) in event of a disaster:

• Biographical (driver's license, passport, birth certificate, wedding license, divorce decree, firearm licenses, military history, etc).
• Medical records (prescriptions, vaccination record, test results, etc).
• Ownership and Financial records (titles of house, vehicles, insurance policies, bank accounts and statements, photos and info of expensive items, credit reports).
• Other (family photos, etc).

Lisa Rowan's "Keep These Financial Records in Your 'Go Bag'"

Do a "backup" of your own memory: in a simple text file, write a summary autobiography. Dates and places you lived, went to school, worked, traveled, etc. Names of friends, roommates, coworkers, etc. Memory fades over time.

Destinations to back up to

• External encrypted hard disk or flash drive attached through USB.
  
  Some choices:
  
  

  • USB disk with hardware encryption (e.g. WD Passport).
    
    
  • USB disk or flash drive with software encryption (VeraCrypt, LUKS2).
    
    

  Prices are very low these days (2021): 1 TB USB hard drive for maybe $70, 1 TB USB flash drive for maybe $180. But pay attention to type of USB connection, and transfer speed. Some info about speeds and types: article1, article2
  
  
  
• Cloud drive (AKA Cloud Storage; really intended for sharing files among devices).
  
  Usually appears as just another disk drive in your system. You copy files to/from it using normal file-manager, or any backup or sync application that expects to see a disk drive.
  
  You really should have the cloud drive "mounted" (appearing in your system) only when you're actually doing a backup. If you leave it mounted all the time, any malware could access that drive and destroy your backup files.
  
  Safest to use a client that comes with the OS or is independent, not the client from the cloud company.
  
  Some choices:
  
  

  • Dropbox
  • GMX
  
  
  
• Cloud backup service.
  
  Usually doesn't appear as a drive in your system; you connect to it through a client (custom or generic) speaking some protocol (custom or generic) to the server.
  
  Safest to use a client that comes with the OS or is independent, not the client from the cloud company.
  
  Some choices:
  
  

  • MEGA
    
    Apparently you get 50 GB free when you create an account, but then it reverts to 15 GB after a month. If you have uploaded more than 15 GB in that first month, you will get warnings/pleas to upgrade, but the files won't be deleted. You won't be able to update/upload any files until you either upgrade or get back under the 15 GB limit.
    Apparently if you don't log in for 3 months, you will get account-deletion warnings in email. So make sure you keep a valid email address on the account.
    MEGAsync client app has an "Export Recovery Key" feature; what you get looks like a password.
    
    On Ubuntu, I think I installed the client by some custom script. Not sure.
    
    If something goes wrong, MEGAsync client may create ".debris" directories in your system with old file copies in them. "find ~ -name .debris -print"
    
    I used MEGAsync for a while, but had a couple of bad experiences where somehow it appeared that the file on my laptop (what I considered the master) was older than the copy in MEGA.nz (what I considered a backup), and MEGAsync synced the old file down to my laptop, and I lost data. Seemed to happen with VeraCrypt containers in use; I would forget to dismount them, and MEGAsync would see them as old. I couldn't see a clear way to specify "only upload to cloud, never sync down to laptop". So I removed MEGAsync client ("sudo apt remove megasync").
    
    But I can still log in to the MEGA.nz web site, click "file upload", and select a file to be uploaded. Safer.
    
    Also, someone said you can run the MEGAsync client, but not designate any folders as "syncing". Then right-click on a folder and select "upload" when you want it to be uploaded. Safer.
    
    So, re-installed by following instructions in Manikandan's "How to Install and Setup MEGA on Ubuntu". Installed with Nautilus integration.
    
    Better yet, instead of using MEGA's client, use an open-source client: rclone or duplicity
    
    
    
  • Sync
    
    
    
  • pCloud
    
    
    
  
  
  

Backups to the cloud

If you do backups to the cloud, don't leave those backups accessible from your machine via a "cloud drive" that is always mounted (shows up as drive H: or something). If you get hit by malware, it may affect files on all accessible drives, including your backups in the cloud.

Apparently, automatic cloud backups of your phone data can expire and be deleted if you don't use your phone for many months. Android backups in Google Drive Backup are deleted if you don't use the phone for 2 months ? iPhone backups in iCloud are deleted if the iCloud account is not used for 6 months ?

A factor to consider: today's cloud backup may be encrypted so well that no one can crack it. But that encrypted data may still be available somewhere in the cloud 20 years from now, and maybe 20-years-future technology WILL be able to crack today's encryption.

Some photo-storage or photo-gallery sites feel free to downgrade the resolution of large photos or all photos, assuming that human viewers won't be able to see the difference. Know the exact policy of the site before using it, if this matters to you. The downgrading may happen as owner uploads, or may happen when a user "views" and/or when the owner "exports". You may have to test carefully to understand what is done.

Eric Griffith's "Back Up Your Cloud: How to Download All Your Data"
Adam Dachis's "How to Protect Your Data in the Event of a Webapp Shutdown"

• There is a difference between backups (retain for maybe 2+ years) and archiving (retain for maybe 10-50 years).
  
  
• There are special cloud-archiving services, but it may be slow and expensive to get your data out. Amazon S3 Glacier
  
  
• Flash and SSD are bad for archiving, because they have to be powered-on every 6 months or so to retain data.
  
  
• HDD is bad for archiving, because the bit sizes are made as small as possible.
  
  
• LTO tape is best for archiving. Wikipedia's "Linear Tape-Open"
  
  
• M-DISC (DVD form-factor, compatible with DVD drives) is a newish technology that may be great for archiving. Wikipedia's "M-DISC"
  
  

Ways to manage the backup process

Type of backups

• Full image: a block-level copy of the whole raw disk/partition contents.
  
  Good: Everything is copied, even hidden data and bootloader etc. Restoring is simple, everything gets copied back in one operation.
  
  Bad: Takes a lot of space and time, even if few files have changed. Restoring is an all-or-nothing operation, you can't restore just part of the system.
  
  
• Full file-level: all files under some directory are copied, as files.
  
  Good: Simple and clear. You can do it manually if you wish. You can see and test what you did. Restoring is flexible, you can copy back only the files you need. Uses only standard default OS applications.
  
  Bad: Takes a lot of space and time, even if few files have changed. May miss hidden stuff or bootloader. Will miss partition table.
  
  
• Incremental file-level: changed files under some directory are copied, as files, and stored as deltas from previous versions of the files.
  
  Good: Fast and space-efficient. Restoring is flexible, you can copy back only the files you need.
  
  Bad: Something has to decide what files have changed, which is tricky and maybe slow. May miss hidden stuff or bootloader. Restoring requires specific software, usually.
  
  
• Incremental filesystem-level: filesystem marks "snapshot" points, and keeps track of state of everything at each snapshot.
  
  Good: Fast and space-efficient. Every detail of filesystem can be saved/restored.
  
  Bad: You have to decide when to take a snapshot, and you may forget or do at inappropriate times. I think all things that have changed will be saved/restored; no detailed control. Not supported by older filesystems. A newish feature, so may have bugs.
  
  

In incremental file-level backup, how to decide if a file has "changed"

One or more of:

• Metadata:
  • File size.
    (Danger: some files such as encrypted container files never change size.)
  • Modified time.
    (Danger: some database or encryption apps may not change modified time.)
  • Inode number.
  • Access permissions / ownership.
  
  
  
• Contents:
  • Hash.
  • Byte-by-byte comparison.
  
  
  

Ways to do the backup

• Copy files across manually, using Windows Explorer or similar.
  
  
• Use general-purpose backup software. See "Linux Software" section.
  
  
• Use backup software specific to a particular (destination) cloud backup service.
  
  
• Use backup software specific to a particular (source) cloud service.
  
  Alan Pope's "Safely Backup Google Photos" (to Linux)
  
  
• Use backup software specific to a particular filesystem type.
  
  For Btrfs: digint / btrbk, or Timeshift.
  
  

Note that a "sync" feature (as in Google Drive, Dropbox, Sync.com) is not a backup. If something is deleted or corrupted on one end of it, that thing will be deleted or corrupted on the other end too. Usually. And if one system gets a bad date/time setting, the "sync" may copy old files over new files. Be careful.
David Murphy's "Why Did iCloud Delete All of My Photos?"

/r/techsupport's "backuptools wiki"

Do before starting to make a backup

Scheduled backups or not ?

• Scheduled:
  
  Good: Don't have to remember to do a backup.
  
  Bad: Destination has to be available at all times (or else the software has to be able to mount/dismount it). No way to clear caches before backing up ? No way to quit applications or unmount filesystems before backing up ? You may not notice if backups are failing.
  
  
• Manual:
  
  Good: Can clear caches and quit apps and dismount filesystems before backing up. More likely to notice any failure right away. You can choose important times to do backups, such as right before and right after some major change.
  
  Bad: Easy to forget to do backups, let them slide for too long.
  
  

Restore

• Will you have all the login information and encryption keys and digital certificates and license keys you'll need ?
  
  
• You might need a special recovery disk to re-install a complete disk image.
  
  
• You might need to re-install any special drivers.
  
  
• Do you have a list of all the applications you'd need to re-install ?
  
  
• Do you have notes about how to do any tricky things, such as tweak network settings or app settings ?
  
  

Linux Software

• rsync:
  
  Available in just about all Linux's by default, and ported to everywhere else.
  Used as a back-end by many backup apps.
  Very good at resuming if the process is interrupted, and sends only updated files.
  
  Copies files from source to destination. If destination is always the same, you're doing a "sync"; if destination is different, you're making a backup, essentially.
  
  I used to use it to do backups to cloud, but then one day I lost data. I think time/date must have glitched somehow, and some files were copied the wrong direction.
  
  
  
  
• rclone:
  
  rclone
  
  This does an incremental file-level backup. What gets used to decide if a file has changed varies by type of destination. For example, with Mega, rclone only copies a file to Mega if the file size has changed. But locally, rclone also can filter via modified-time.
  
  CLI only. Available in just about all Linux's by default, and ported to everywhere else. Knows how to connect to many cloud services.
  
  I use "rclone copy", which can only copy from local to remote.
  
  

  
  # Installed it through Ubuntu's software store; it's a Snap.
  rclone --version	# v1.36
  rclone --help
  
  # https://rclone.org/mega/
  rclone config
  # type n for "new remote"
  # give it a name of "remote"
  # type number for MEGA
  # WHOOPS: this version doesn't support MEGA
  snap remove rclone
  
  # same with deb in Ubuntu MATE 20.04 repo
  # same with deb in Kubuntu 20.10 repo 12/2020, version 1.50.2
  
  
  # Go to https://rclone.org/downloads/ and download the "Intel/AMD - 64 Bit - deb"
  # Double-click on .deb file and install.
  rclone --version	# v1.52.n or better
  
  # https://rclone.org/mega/
  rclone config
  # type n for "new remote"
  # give it a name of "remote"
  # type number for MEGA (21 ?)
  # type username of your MEGA acct
  # type password of your MEGA acct
  # type n to not edit advanced config
  # type s to set configuration password
  # type a to add password
  # best to use MEGA acct password ?
  # type q to quit to main menu
  # type q to quit config
  
  # list top-level dirs
  rclone lsd remote:
  # FAIL: have to turn off 2FA on MEGA account
  
  # list top-level dirs with modified times
  rclone lsd remote:
  # list top-level dir names only
  rclone lsf --dirs-only remote: | sort
  # list all dir names only, recursively
  rclone lsf --dirs-only -R remote: | sort
  
  # show quota information about remote account
  rclone about remote:
  
  # Copy changed files from source to remote.
  # Recursive down dirtree.
  # Copies if contents have changed, not if only timestamp has changed.
  rclone sync ~/MYDIRNAME remote:MYDIRNAME --progress
  
  # Apparently rclone has no way to read its commands from a file.
  # But if doing multiple commands, set variable RCLONE_CONFIG_PASS
  # to avoid getting prompted for password multiple times.
  

  
  Simplified version of bash script I use:

  
  #!/usr/bin/env bash
  
  if [ -b /dev/mapper/veracrypt1 ] || [ -b /dev/mapper/veracrypt2 ]
  then
  	ls /dev/mapper/veracrypt*
  	echo Dismount VeraCrypt volumes before doing backup
  	read -p 'OK? '
  	exit
  fi
  
  export RCLONE_CONFIG_PASS
  read -sp "What is rclone password ? " RCLONE_CONFIG_PASS
  
  set -o verbose
  
  rclone lsd remote:
  
  rclone copy ~/DIR1 remote:DIR1 --progress
  rclone copy ~/DIR2 remote:DIR2 --progress
  
  rclone about remote:
  
  set +o verbose
  
  read -p 'OK? '
  

  
  dobackup-pushtomega.sh script I actually use.
  
  Test your script VERY carefully. I found this MAJOR issue with using rclone with Mega.nz: "Mega does not support modification times or hashes yet." from https://rclone.org/mega/ , which means rclone only copies a file to Mega if the file size has changed. Some files (e.g. VeraCrypt and LUKS containers, and probably VM images) never change size; for other files some edits may leave the sizes unchanged. You may think these files are being backed up, but they're not. [Someone said "apparently mega encrypts all files locally before uploading them, which makes hashing not work".]
  
  Adding "--no-check-dest" will force copying always. But there seems to be no way to check for changed hashes and modified-times only on the client side. Maybe use "--max-age 7d" and then do a backup at least once a week ?
  
  Also, there is a distinction between "sync" and "copy": if you rename a file from A to B, with "sync" you will end up with only file B on the backup, but with "copy" you will end up with both files A and B on the backup.
  
  Linux Uprising's "How To Encrypt Cloud Storage Files With Rclone"
  
  
  
  
  
• Timeshift:
  
  Timeshift is meant to back up the OS, not /home, and is set that way by default (at least in Mint).
  "You can restore a snapshot from the grub command line." [Not sure this is true.]
  
  "Caution: If you do a backup, and later change your system password, you won't be able to restore from that backup using the Snapshot feature. Neither old nor new password will work."
  
  Linux Mint Installation Guide's "System snapshots"
  TechHut's "Timeshift Tutorial" (video)
  Paul Kim's "A quick guide to setting up Btrfs Timeshift on Fedora 33"
  
  Backups are incremental; first one may take 20 GB, each subsequent one probably takes less than 2 GB, depending on how much you've changed in the "system".
  
  Since (in Mint 19) Timeshift by default backs up the OS to a folder on /home, which probably is on the same disk as the OS, it's really making snapshots, not backups. If the disk fails, you've lost both the active OS and the Timeshift "backups".
  
  [Method 1:] If you do Timeshift snapshots of OS, and then later the OS becomes un-bootable: boot live image of same version of OS from USB stick (with no internet), maybe back up /home to some external drive to be safe, do absolute minimal install of OS to hard disk without touching /home, reboot into live session again, run Timeshift and "restore" your latest snapshot to the newly-installed system on hard disk, reboot into hard disk.
  
  [Method 2:] If you do Timeshift snapshots of OS, and then later the OS becomes un-bootable: boot live image of same version of OS from USB stick, do chroot or arch-chroot to root filesystem on hard disk, login as root user, run Timeshift and "restore" your latest snapshot to the filesystem on hard disk, reboot into hard disk.
  
  Timeshift normally uses rsync to make copies of files, but on a Btrfs filesystem it can do Btrfs snapshots instead.
  
  Garuda distro has a GRUB entry that lets you restore from Timeshift backup. Similar on other distros too, if doing Btrfs snapshots ? Install "grub-btrfs".
  
  
  
  
  
• Backup Tool:
  
  Installed in Mint by default.
  Meant to back up your home directory (makes a TAR of it), and can also be set to save a list of the software you installed through Mint's Software Manager.
  
  
  
• Backer:
  
  GUI app that uses rsync.
  Only latest version of each file will be saved on each backup medium, I think.
  Backer
  
  
  
• Veeam:
  
  Veeam Agent for Linux
  
  
  
• BorgBackup:
  
  BorgBackup
  BorgBackup docs
  Vickie Li's "Backing Up With Borg"
  Teknikal's_Domain's "BorgBackup"
  Bruce Byfield's "Command Line - BorgBackup"
  "It does encryption, differential backups, de-duplication, and you can mount archives. Downside: can only use SSH-capable backend and run borg server on the destination."
  Some clients: borgmatic, Vorta, Emborg.
  Some servers: BorgBase.
  
  
  
  
  
• Back In Time:
  
  GUI front-ends for it are available.
  bit-team / backintime
  Chris Hoffman's "How to Back Up Your Linux System With Back In Time"
  
  
  
• restic:
  
  restic docs
  Shahriar Shovon's "How to Install and Use Restic on Ubuntu 18.04"
  
  
  
• Clonezilla:
  
  Boot "Clonezilla Live" from a USB and make/restore image backups.
  Clonezilla
  Matei Cezar 's "How to Clone or Backup Linux Disk Using Clonezilla"
  
  
  
• rdiff-backup:
  
  This does an incremental file-level backup, using size, modified-time, inode number to decide if a file has changed. (I turn off the inode-number-checking.)
  
  rdiff-backup
  Aaron Kili's "rdiff-backup - A Remote Incremental Backup Tool for Linux"
  Patrik Dufresne's "What's new with rdiff-backup?"
  Patrik Dufresne's "Manage your Linux backups with Rdiffweb"
  
  

  
  sudo apt install rdiff-backup
  
  # put this script in a dir on the backup drive and run it from THERE:
  mkdir MYRDIFF
  rdiff-backup ~/MYDIR1 MYRDIFF/MYDIR1	# back up a dir tree
  
  # flags I use: --use-compatible-timestamps --no-compare-inode --create-full-path --print-statistics
  
  # If you want to back up just a single file ~/MYDIR1/FILENAME1:
  rdiff-backup --include '**FILENAME1' --exclude MYDIR1 ~/MYDIR1 MYRDIFF/MYDIR1
  

  
  When backing to a USB disk, if none of your files have changed since last backup, the scan of all files is VERY fast.
  
  What exactly is being tested to determine "file has changed" ? I think only metadata: size and modified-time. Not checking file contents. (Which means you should do a "touch" on VeraCrypt container files beforehand to make sure they get backed up.)
  
  Since the backup archive is compressed/incremental, you can't easily open a couple of files to see if they are valid. You'd have to restore to some temporary location and test them there.
  
  
  
  
• rsnapshot:
  
  rsnapshot
  Alan Pope's "Straightforward Linux Backups with rsnapshot"
  
  
  
• duplicity (CLI only).
  
  duplicity
  Duplicity is available as a deb or snap.
  Ubuntu Community Help Wiki's "DuplicityBackupHowto"
  "duplicity --help | less"
  URL format for MEGA back-end: "megav2://user[:password]@other.host/some_dir"
  
  Deja Dup is a GUI front-end for duplicity.
  It's available as a deb or snap.
  
  
  
• Deja Dup:
  
  Deja Dup is a GUI front-end for duplicity.
  Deja Dup
  Jack Wallen's "Total System Backup and Recall with Deja Dup"
  Chris Hoffman's "How to Back Up Ubuntu the Easy Way with Deja Dup"
  Someone said: no way to have "profiles" to do backups to different locations.
  
  
  
• Cryptomator:
  
  Requires a cloud service (Google Drive, Dropbox, etc) to store the files on.
  Appears to you as a disk drive.
  Cryptomator
  
  
  
• FSArchiver:
  
  FSArchiver
  
  
  
• git to some git repository:
  
  
  
• Simple "cp" commands to external hard disk:
  
  Since the backup is plain files, you easily can open a couple of files to see if they are valid. [Encryption and compression are done at the device/filesystem levels.]
  
  Scripts I use to back up configuration files and my personal documents: dobackup-pulltodisk.sh and dobackup-pulltodisk-extras.sh
  
  
  

Miscellaneous

If you're concerned about your backups functioning for 10, 20, 50 years

Don't focus so much on "what media to use ?". Focus on "when should I make a perfect copy from my current media to new media ?". Where "new media" could be same type as current media, or some new type as the current type nears EOL.

For example, if you have backups on floppy disks, you need to copy to new floppy disks as the old ones threaten to degrade, or copy to USB flash sticks as the market threatens to stop making floppy drives and floppy disks.

Then in 10 or 20 years, you'll be copying from your USB flash sticks to whatever the new medium is (DNA or something ?).

Uninterruptible Power Supply (UPS)

Ray Woodcock's "Choosing, Configuring, and Testing a Battery Backup (UPS)"
Cheapskate's Guide's "My Cheap Solution for Keeping My Website on Line"