Hard drives have greater and greater capacity, and all for less and less cost, every day.

But their underlying reliability - the MTBF (Mean Time Between Failures) seems little unchanged from a decade or more ago.  The new high capacity hard drives equate to a greater headache in terms of data backup and subsequent data recovery after the inevitable hard drive crash occurs.

Fortunately, there are now some excellent solutions that - at long last - makes backup easy and foolproof, and some prudent precautions you can take to anticipate a hard drive crash and solve the problem at a time of your choosing rather than when the hard drive decides to suddenly fail.

Protecting Your Data is Both Easier - and More Essential

Our data storage 'needs' have grown, probably in line with Moore's Law (ie doubling every two years or so).  Most people now have a 100+ GB hard drive in their computer, whereas a decade or so ago it would have been a 100 MB hard drive, and a decade or so before that, hard drives were still an optional upgrade rather than an essential part of a computer.

Although the cost of drives are now unbelievably low (in terms of cents per Gigabyte) and the size of the drives unbelievably small, the reliability issue remains a weak point - and indeed this is probably because of the other two improvements.

Even good hard drives 'fail' thousands of times a minute.  They misread data all the time, but most of the time, they are able to correct these errors without pausing in their stride.  It is only when their performance degrades beyond the built-in levels of fault tolerance that the problems become noticeable and threatening.

For a fascinating 'look under the hood' the video on this site is interesting, albeit technical.

If manufacturers thought they could justify a higher price in terms of more reliability/longevity, they might be motivated to do so, and if they thought that people would accept a more bulky drive that was more reliable, again they might do that, but their feeling at present is that hard drives are 'throwaway' items that are best produced as inexpensively as possible.

Sure, hard drives may be throwaway in terms of the cost of buying a new one, but in terms of the inconvenience of losing the data on a drive (and the hassle factor of reinstalling all your software on a new drive), they are enormously expensive and very undervalued items.

Of course, the more data we store, the more data we must backup and protect.  Fortunately, the old-fashioned ways of backing up data - ways which were cumbersome, time consuming, expensive, and relied on us to voluntarily participate in the process - have been largely replaced by convenient automatic and inexpensive solutions that now finally and truly leave us with no excuse not to have our data all thoroughly backed up, all the time.

Read on for how to manage and control your own data storage.

RAID - Safety in Numbers

If you have a laptop, you probably only have space for one hard drive in your computer.  But if you have a regular desktop style computer, you probably are vaguely aware that inside the computer's case there is room for two, three, four - maybe even more hard drives.  However, the chances are you only have one hard drive installed - with the drives being so huge in capacity these days, who needs two, right?

It is true that in the past, there were capacity and performance reasons for having more than one hard drive in one's computer, and it is also true that these days, both issues are of less importance.  However, there is now a reason for having two or more drives in your computer that takes advantage of the ever-lower costs of hard drives and related controllers.

This reason is to have an automatic backup system whereby all your data is written onto two drives simultaneously.  If either of the two drives fail, your computer simply swaps over to the other drive and asks you, politely and non-urgently, to please replace the failed drive one of these days.

The replacement is also an angst-free operation.  Pull the old drive out.  Plug the new drive in.  Nothing to reload or reinstall or recover.

For you as the computer user, all the magic of the process is completely hidden.  Everything is the same to you in terms of file management, programs, controls, and so on.  Apart from probably seeing one more message flash over the screen when the computer is booting up, everything else will be the same.

This amazing ability has been around for a long time, and in its most sophisticated form is referred to as using a RAID device.  A RAID (which stands for Redundant Array of Inexpensive (less frequently and less correctly the word 'Independent' is sometimes used) Disk (drives) ) is simply two or more hard drives connected to a controller card (which is built in to the mother board of many computers these days, otherwise which can be purchased and added to the computer subsequently).

The RAID controller does all the behind-the-scenes housework of checking the drives and keeping duplicate copies of everything on both drives, and of taking a failed drive offline with hopefully no interruption of service or data loss.

There are different types of RAID array.  Some offer data safety, others offer faster disk performance, some offer a mix of both.  The one of most relevance for backing up data is also the most inexpensive and simple form of RAID array, and is referred to as a RAID 1 array - a pair of mirrored drives.

Your action item is simple :  Next time you buy a desktop computer, consider asking for it to contain a RAID 1 controller and two drives rather than one.

This might add $200 to the cost of the computer, but it will almost completely eliminate your fear of future data loss from a corrupted hard drive.  There is nothing new for you to learn, no new complexities or software programs or anything.  It just happens, automatically in the background, the same as the rest of the computer's hardware operations.

Automatic Off-Computer Backup

If you don't have a RAID 1 computer, you of course need some sort of backup procedure.  Even if you do now have a computer with a RAID 1 capability, you still need off-computer backup too.

Why is that?  Isn't the RAID 1 supposed to solve all problems, make the coffee for you in the mornings, and take the cat out at night?  Alas, no.  You remain exposed to two potential problems - the first is a failure in the RAID controller card itself that could, as part of its failing, corrupt data on both drives, and the second is an external type of problem (more about this in a minute).

Clearly, if you're not mirroring your data on two drives in your computer, you have no backup for your computer's data and must have some way of having an up-to-date copy if everything, just in case something goes wrong.  And the list of things that could go wrong is broader than you might think.

Reasons Why You Need Off-Computer Backup

You need a copy of all your information off your computer for several reasons.  What happens if your computer is lost or stolen?  If your data is backed up on your computer, then the loss of the computer means both your main and your spare set of data have both gone too.

The same is true if instead of losing your computer, it is destroyed in a fire or other awful event like that.

So your backup copy of your data needs to be somewhere unrelated to the computer itself.  At the very least, it needs to be where a thief wouldn't find it and take it as well as your computer.

Ideally if it can be completely away from the location of the computer, then you can have fire, floods, famine, or just about anything else where the computer is (was) but still have a copy of your data somewhere else, unharmed by whatever misfortune befell your main computer.

Automatic Backup

Remember the bad old days of setting a schedule to back your computer up, and feeding it a collection of floppy disks, one after the other after the other?  Or running a tape drive?  Or burning CD copies?

All those processes are now obsolete and have been replaced by much more convenient programs that will be all the time monitoring your hard drive, and any time they see a change to any file, they will then instantly take a copy of it and shuffle that copy off to your backup device/location.

If you are not yet using an automatic backup program, you should - no, not should.  You must get one and start using it as soon as possible.  It is so easy and simple, there is no reason or excuse not to do this.

Local Backup

The easiest type of automatic backup system is to go to Costco or Amazon and buy a ridiculously huge external hard disk that comes complete with the automatic backup software.  These drives can be either a local drive - ie, connected to the computer by a short length of probably USB cable - or a network drive (often called a NAS - Network Attached Storage - device), which can be located somewhere else in your house/office, but connected to your computer via the network (either wireless or wired).

Clearly, it is better to have the drive somewhere away from the computer, in a non-obvious place so that thieves would be less likely to find it and take it as well as your computer at the same time.

A network attached drive has another advantage too.  You can have multiple computers all sharing it for backing up purposes.  So instead of needing to buy a backup drive for each computer, you just need one and you can use it for two or three different computers (different drives and software have different limitations on how many computers they will allow to share one networked drive).

If you are choosing the networked drive option you need to understand how many users the drive will allow to be connected to it, and if the drive can be connected by Wi-fi or only wired ethernet cable.  And get a really big one - there's little difference in price, and it gives you more ability to grow into the future.

Here's the page on Amazon listing most of their NAS drives.  I have a 3TB Western Digital device myself.

Cloud Backup

The ultimate in backing up away from the physical location of your computer is to back your data up into the 'internet cloud'.

These days there are a lot of companies that offer automatic backup services, copying your data off your computer and onto one of their servers, somewhere else in the country or world.  If you have a fast internet upload speed, this can be a great way of getting your data safely stored somewhere else.  (Note that stated internet speeds are usually based on the download speed - your upload speed might be very much slower, and can be measured by a service such as this.)

Two established examples of cloud backup services that operate automatically on your computer are Mozy (nothing for free, their least expensive package a 50GB allowance for $6/month) and Sugar Sync (5GB for free, 30GB for $5/month).

Google, just a couple of weeks ago, announced a product - Google Drive - that gives 5GB for free and 25 GB for $2.50 a month.

So there we were, discussing a local backup drive with 3TB (ie 3,000 GB) of capacity - that makes 5GB seem awfully puny, doesn't it!  Indeed, your computer probably has a 100+ GB hard drive on it, and for sure, if you check, you'll see it has way way more than 5GB of data currently on it.  But maybe you don't need to back up all that data.

How Much Cloud Storage Do You Need?

It is one thing to be choosing between your own local storage device that has a 1TB or 2TB or even 4TB or 8TB capacity, and with only a few tens of dollars separating the cost of the smallest and the cost of the largest of these devices.  In such cases, obviously you are well advised to get more than you need right now, because - as we all know - our data storage needs have a habit of growing and growing.

But when you're having to pay some dollars every month for every GB of storage online, you need to take a more careful look at what you need to store.  The good news is you actually need to store much less than you thought.

First of all, there is no point in storing anything to do with your operating system, because you can't just copy all that stuff back to your computer if you need to.  You have to re-install it from your original disks rather than copy the installed files from backup.

The same is true of most other programs.  Don't copy any program and configuration files for any of the programs you have, because they all have to be re-installed too.

If you downloaded programs in installation packages to then be installed, you would only need to keep copies of these if they were protected downloads that you can't re-download in the future for no extra payment.

Now for your actual data files.  Most people will find that the largest part of their data files are videos, music tracks, and pictures.  A single two hour HD movie could use up your entire 5 GB of free storage somewhere - or, to look at it another way, it would be cheaper to buy DVDs and Blurays than to pay for the cost of remotely storing such movies.

We suggest that the large sized types of files such as mentioned above - files which are probably seldom accessed and never changed - be backed up onto some sort of local device.  Your cloud backup should only be for data files - spreadsheets, documents, and those sorts of things, and your local email files if you use a local email client rather than a cloud email client.

You'll probably find that 5GB is more than sufficient for these types of files (although if you're like me, you might find that your old emails will spill over 5GB by themselves - what I do there is I have old email archives of old emails that I store locally, and only keep the ever-changing current email file stored remotely).

The Downside to Remote Cloud Storage

There is a downside to using a cloud based service as well.  You are reliant on being able to access the cloud storage service any time you need to - both for uninterrupted convenient backing up, and of course, most of all for being able to retrieve your backed up files in the future.  This makes you reliant on both convenient fast internet access and the continued uninterrupted operation of the remote service.

There are some surprising risks and downsides to this, well covered in this article about the dangers of cloud based computing.

So, by all means use cloud based backup services, but use them as well as, rather than instead of, your own backing up procedures.  Remember, a bird backup in the hand is worth two in the bush cloud.

Some people are also uncomfortable with the privacy of their data.  All the leading remote backup services use encrypted communications for your data to be remotely backed up, but the unavoidable fact is we regularly hear of credit card companies having their 'secure' computers hacked and our personal information compromised, so there is some element of risk whenever we store information outside of our control.

Disk Health Checkups

Do you have annual checkups at your doctor?  Do you go to your dentist from time to time?

What about your car?  Is that regularly given lube jobs?

And what about your computer?  When was that last given some preventive maintenance?  These days many of us rely on our computer more than our car, and whereas it is easy to rent a replacement car, there's no way we can rent a replacement set of data if we lose it.

Fortunately, in most cases, there is software you can use to test your hard drives and in many cases these programs will warn of incipient failure.  Most modern hard drives have some built in error recovery capabilities, and by seeing how much error recovery the hard drive is having to do, it is possible to measure how well the drive is performing.

Hard drives also have 'spare' space on them that can be pressed into service when some parts become 'bad' and being able to see the amount of spare space that has already been used up gives another measure of the condition of the drive.

There are three types of hard drive testing software.

The first is a program provided by your hard disk manufacturer.  Most hard drive manufacturers will provide software to test their drives.  Look on their website, or if you can't find it, ask their support people.  This page has a list of major drive manufacturers and the disk utility software they provide.

The second type of hard drive testing might be provided by the manufacturer of your computer.  Usually the most rigorous tests are pre-boot tests, but maybe there is a utility program that can be run after your computer is up and running, too.

Again, if you can't find this software, ask your computer's support people about it.

A third type of hard drive testing is an independent third party program.  There are many of these, and some are free while others are not.

One of these programs stands head and shoulders above all others.  It is the Spinrite software, sold by Gibson Research.  It does a better job of diagnosing the health of your drive, of recovering bad data, and sometimes of restoring your disk back to health again too.  Very highly recommended (and Steve Gibson, the developer of Spinrite, is one of the all-time 'good guys' of the internet, having done pioneering research on issues like spy way and security weaknesses which he has generously released into the public domain).

One type of software which won't really help you is that within your operating system (presumably Windows).  The Checkdisk utility won't really tell you if your disk is alive and well, or on its last legs.  Don't rely on this to help you anticipate disk crashes.

Not to sound obsessive, but if you don't choose Spinrite, you might want to use both your hardware manufacturer's utilities and also your hard drive manufacturer's utilities too.  Sometimes one might detect something where the other does not.

Note that you should always do the complete or extended or advanced testing.  The quick testing might pick up serious errors, but it will not necessarily identify more subtle errors that may be evolving.

How Regularly Should You Test Your Harddrive?

How often should you test your harddrive?

If Spinrite gives it a two thumbs up type clean bill of health, we'd suggest maybe once every half year or so (add it to your list of things to do with each Daylight Saving transition, perhaps).

As the drive ages, you'll want to keep a more careful eye on it.

Any time your computer starts misbehaving, you'd be well advised to include a detailed drive test as part of your troubleshooting.  Sometimes hard drive problems can manifest themselves in strange ways, due to the complex interaction as between a computer's hardware and the software which controls it and the additional software which interfaces between the computer and you.

Summary

You can protect your data, within your computer, by adding a RAID 1 disk controller and extra disk to your desktop but not laptop computer.

You can protect your data, outside your computer, with a cable connected remote hard drive nearby, or a LAN connected NAS drive somewhere else in your computer network.

You can also protect your data completely off-site via a cloud backup service.

Automatic backup software ensures your backups are always up to date.

These various different approaches are all simple and easy and inexpensive.  You no longer have any excuse not to have 100% backup and protection for your computer data.

Related Articles, etc

If so, please donate to keep the website free and fund the addition of more articles like this. Any help is most appreciated - simply click below to securely send a contribution through a credit card and Paypal.

Originally published 25 May 2012, last update 30 May 2021

You may freely reproduce or distribute this article for noncommercial purposes as long as you give credit to me as original writer.