Some computer storage problems and solutions

This is a post on some computer storage issues and some things I learned while addressing them.

.

Drobo to TerraMaster?

For ten years, I have been backing up to a Drobo.  This was a black box (literally) with five drives in a RAID array, so that it operated faster than single disks and was also safer, as a disk or two could die and you could just replace them without losing data. 

But after twelve years, the Drobo died.  It appeared to be in a startup loop.  I found a report of this and it appeared to be a terminal problem but in any case, support from Drobo is almost non-existent and I would have had to pay $US100 to get their technical people to consider the problem.  So I decided to replace the Drobo with a Chinese five-disk RAID device called a TerraMaster D5-300, which I got for about about half the price of a replacement Drobo.

(I have about 6TB of image data to back up so it’s a sizable task as well as an important one.  NVMe SSDs give great performance but are too expensive to store lots of data so it’s still back to the old hard disks or HDDs for that.)

.

Backup, Computing, Drobo, Photography, RAID, Terramaster

This is what it looks like.

Unlike Drobo, Terramaster offers very good assistance which I found invaluable.  The unit also comes with a piece of paper with a link to the Terramaster site where they explain how to set up (including a short video).

.

Problems with RAID and hard drives

First I tried the new RAID using 3TB HGST hard drives from the old Drobo but I ended up with partition errors, the RAID was in RAW format and when I dismounted the RAID, the individual disks were in RAW format.  Now this is not like RAW files for images; a RAW disk has no file system that Windows can read.  My backup program Acronis couldn’t recognise the disks, Windows Explorer didn’t recognise them, Window utility chkdsk couldn’t fix them and nor could Disk Manager.  After some research I found a free utility Paragon Partition Manager Community Edition that allowed me to to convert the RAW hard drives to NTFS and recover them. 

I still wasn’t able to create a RAID, though.  I tried twice using six HGST 3TB drives and three times using nine WD Red 2TB drives (from a RAID on my previous PC).  Each time I encountered partition errors, RAW format RAIDs and disks, and bad sectors on some of the disks.  I discovered that Windows chkdsk (Check Disk) is a superficial utility that may approve a disk as OK that still has bad sectors on it.  However, I discovered some free utilities that though slower are far more thorough than chkdsk.  I used DiskGenius and WD Data Lifeguard Diagnostic for Windows (and for those with Seagate disks, there is also Seatools by Seagate (but may not work on non-Seagate drives)).  (WD Lifeguard Diagnostic has also recently been replaced by another product but it’s only for SSDs).

After five unsuccessful attempt to create the RAID and losing some disks, I gave up and returned the RAID enclosure.  Instead I am backing up to additional single hard drives located inside my PC.  This only makes sense because I also backup to the Cloud using CrashPlan for Small Business.  Otherwise, in the event of a fire, I would lose all my files.  (I also have local backups to external disks using a disk caddy but that doesn’t get updated very often).

I ended up with two dead HGST 3TB hard drives out of six and five dead WD Reds out of nine.  The HGST drives may well have been killed by the Drobo.  The Drobo’s startup loop meant it kept starting up and closing down, so that the hard drives were repeatedly waking up and crashing, which could well have caused physical damage and made some of them unusable. 

The HGST drives were 8 or 9 years old but they are enterprise drives with a longer life and I don’t think that was the problem.  My WD Reds were mainly 6 to 8 years old though three, including one that failed, were only one year old.  A “normal” hard drive is said to have a life expectancy of 3 to 5 years, so was it just that the most of the WD Reds were too old?  I decided to also test my other old hard drives from previous RAIDs and backups.  These included four 2TB WD Greens of 10 to 11 years old, four 1TB WD Blacks mainly 12 years old and one 1.5TB WD Green of 9 years old.  All of those tested OK including two with a single bad sector that was repaired.

So I suspect that those five WD Reds (most of which were in a RAID last year) did not die of old age but were killed by the TerraMaster.  Perhaps it would have worked with newer disks but I wasn’t about to purchase a whole lot of them to find out.  Perhaps I had a defective RAID enclosure.  Most people report no problems and find it works well though a small minority have reported problems similar to the ones I encountered.

.

Hidden issues with WD Reds

Speaking of WD Reds, they are the most obvious disk for most people to use in a RAID but there is a hidden issue with them.  A couple of years ago there was just the WD Red but now there is Red, Red Plus or Red Pro.  The old Reds are the same as the Red Plus.  The new Reds are best avoided because they may be less reliable and are not much cheaper.  However, WD weren’t open about this change and disks sold as Reds can either be new Reds or new Red Pluses (refer here for which models are which).  The Red Pros are faster than the Red Pluses (7200rpm vs 5400rpm) but the Red Pluses are likely to last longer and the speed is not so important for backup.  The enterprise drive above the Reds is the WD (HGST) Ultrastar, which replaces the WD Gold.

.

CF and SD Cards

Coincidentally, I encountered a bad sector when trying to download some images from a CF card.  CF and SD cards are said to have a life of around ten years and some of mine are as old as 13 years.  So I also went through and tested all my CF cards and found two out of seven unusable due to bad sectors.  That may not mean your camera will tell you you’re about to lose some images though, so it’s a good idea to occasionally test your cards, especially the old ones.  It’s probably better to throw away a CF or SD card if it has any bad sectors.  For testing these cards I used DiskGenius because it has a good graphical interface.  (I will soon also test my SD cards, which can be up to 10 years old).

.

To check a disk you right-click to Verify or Repair Bad Sectors or find that on the Disk menu.

Hopefully, you have a disk with no bad sectors.

 

… But this one is unusable whether the camera thinks so or not, and is to be thrown away.

If you have problems with lost files on a CF or SD card, the best option I know of is RescuePro Deluxe, which comes free with Sandisk Extreme and Extreme Pro cards.  It’s not owned by SanDisk though and it recently changed hands.  They now make it more difficult for you to find the free download link and try to encourage you to go for the paid version.

.

Next post : Normal service resumes at Bukhara, Uzbekistan….

 

Technical Posts

 

Links to technical posts on this Blog….

.

 

Technique

RAW Processing

Lightroom

Printing

Computers

Backup

Equipment

Computer problems – RAID and backup

I may make very few posts during the next six weeks or so.

One reason is that I’ve been spending a lot of time rebuilding my computer data storage.  First the backup of my travel images became corrupted so I tried to recreate it.  Then one of the backup drives failed.  Then one of my data drives failed.

My main data drive was a RAID 5 array with four 1TB drives.  This means that three of those drives were for storage, with “striping” or writing across drives for speed, and the remaining drive was for backup.  My backup system is a Drobo S, with five 3TB drives, functioning similarly to RAID 5.   I had a lot of spare space there so when the drive failed it just rebuilt to the remaining four drives but that made it unusably slow for backing up for several days in the meanwhile.

Then, when one of the drives in the RAID 5 data drive failed,  I managed to backup Antarctica, Japan and New York and just two TIFF files from the North Atlantic trip before it died completely.   This was good because I had no backup of the images I restored and for the travel files I didn’t manage to restore, I still had all the RAW files on the laptop.

I now have all backups restored.  I lost about 250 TIFF files from the North Atlantic trip where I worked in Photoshop CC, Vivesa, Silver Efex Pro, SNS HDR or AutoPano Pro.  Those I can always regenerate from the RAW files though.  However, it appears I failed to create a backup for Blues Festivals in 2011 and 2012 and I have lost all those images, about 7,000 in all.  The best 1,000 or so are published to the web on JAlbum (in reduced size) but losing the source files is unfortunate and there’s no redress.

.

The original file for this image is now lost. This is Darren Jack at Thredbo Blues Festival 2012.

.

The conventional wisdom is that you should have three copies of your data, including one in a remote location in case say your house burns down.  I thought I could get away with two because they were both in systems that can usually cope with drive failures.  I almost did get away with that but then the main problem was that I was missing a backup.

I have rebuilt the data drive with four new 2TB drives as a RAID 10 array.  This means that two of those drives store data while the other two are backups (mirroring the data).  The drives are also “striped”, writing between each other for speed.  That can cope with losing up to two drives, provided they are not both drives of a mirrored pair.

I have many spare internal drives and a “disk caddy” to access them.  So I will create a second set of backups on those drives.  Finally, I will backup selected images to the Cloud.  These will probably be those with three stars or more in Lightroom, only essential images to minimise expense.

It’s also important to have a backup of your computer’s image, i.e. your C Drive including all the hidden system files.  You can use that to restore all your functionality and programs if you get a virus or your C Drive crashes and even (with a higher version of Acronis) to a different disk that replaces your C Drive.  This didn’t come to play in this situation but has a couple of times in the past.  I have saved images from both Acronis and Windows.  There’s a link here that shows you how to do it on Windows 8 which also has the benefit that Windows will use this image if you elect to Refresh PC.  Otherwise refreshing the PC just wipes everything.

I lost ten days with all of that.  I had about ten posts prepared in advance but not any more.  I’ve started preparing posts on St Kilda where I took about 400 images and another 420 sailing around the islands.  This will take a while and there’s also a lot to say.  Then this coming weekend I’m off to cover the Thredbo Blues Festival and it will take me at least two weeks to process the images I take there.  After that I’m in India for most of February (more on that soon).  I may not get much opportunity to post from there.  There may be very few posts between now and early March.