The Need for Speed

No time to prepare my usual travel post because I’m rebuilding my computer, using new technology with the aim of a very fast upgrade.  Here instead is a detailed article about that.  (For those not interested in this, my next post should be on St Martin/ Sint Maarten in the Caribbean.)
.

Asus Hyper M.2 x16 Card, Backup, Computer Upgrades, Computers, Equipment, Laptops, Motherboards, Photography

The super-fast tongue

(Panther Chameleon, Peyrieras Nature Reserve, Madagascar).

From the images I took, at 9 frames per second, I estimate the full extension of the tongue takes about 1/5th of a second.

.

Context

With the sudden impact of the COVID-19 crisis, many people are looking to upgrade their PC, either for working from home or because they are now going to spend a lot of time there.  I am also going through this process, though both my cause and my objective is different.  I need to replace the motherboard of my PC and I am looking to upgrade my PC specifically for high-end photographic processing.

I have both Nikon and Fuji cameras.  My main Nikon camera is 47MP which produces very large files (52MP) and for either Nikon or Fuji I may perform computer-intensive operations to combine many exposures.  This can be for increased depth of field, extending tonal range, creating panoramas or some combination of those.  Processing infrared images and scanning film can also be demanding on resources.

Many of these operations I either was not performing or with much smaller files when I last upgraded my motherboard in late 2016.  Therefore I prefer to take account of these greater tasks and the possibility they may increase in the future in ways I now don’t anticipate.  I hope that if I have a more efficient computer I may be able to free up some of the time I now spend processing.

I’m writing this hoping that anyone can easily understand and that it will also be useful to people with detailed knowledge.  I’m definitely aiming at a high-end PC but I’ll also cover alternatives for people who have lesser financial resources or lesser processing needs.

I have already written an article on specifying Computers for Photography which is more general than this one and which contains links to other informative sites.  This article includes new technology that is not covered in any of those links.

 

Priority for Speed

My main objective is a search for processing speed.  As linked to in my earlier article, Image Science suggests that the order of priority for spending money on a computer is:

  1. Monitor and Calibrator
  2. Disks
  3. Fast Ram
  4. Video Card
  5. Processor
  6. Motherboard
  7. Case

 

Types of Hard Disks

I’m going to ignore the monitor and calibrator here (See information on that in the other article). The next priority is disks and the reason for that is that different types of disks and different combinations of them have very different speeds.

  • HDD or spinning disks are those big heavy things that have been around for decades. An example is the Western Digital (WD) Black, which you might use to store data.  Another example is the WD Blue (previously WD Green), which spins at 5400rpm instead of 7200rpm and is slower.
  • Next step up is SSDs, small and oblong but squarish, rather like a big USB stick for a different purpose. They are faster and more expensive than HDDs and these days many people use them as their C Drives or boot disks.  They can fit into an HDD drive slot with an adaptor, or they can have their own.
  • More recent are the M.2 SSDs which are long and thin and smaller than a “normal” SSD. They require a motherboard that supports them and have their own attachment points.
    • HDDs and ordinary SSDs are SATA disks (using the SATA standard for data flow). An M.2 SSD can also be a SATA disk in which case they have about the same speed and cost as a “normal” SSD.
    • SSDs can alternatively follow the PCIe standard. PCIe M.2 SSDs are more expensive than SATA SSDs and significantly faster.
  • As well as these different kinds of hard drives, your motherboard may allow you to combine groups of the same model and capacity of hard drive (in a RAID). This can be either for increased speed or increased security or both.  You’d use WD Reds for an HDD RAID because they have special reliability provisions.  Individually, they are the same speed as WD Blues.  For SSDs in a RAID, you don’t use a different model of SSD.  Let’s say we have four hard disks.  There are four different kinds of RAID.
    • RAID 0 writes to all disks simultaneously and appears as one disk. The four disks make it four times as fast as a single disk but if one disk fails, you lose all the data on the RAID.
    • RAID 1 appears as two disks but each of those virtual disks contains two disks, one of which has a copy of the other disk. You lose no data if a disk fails and the speed is the same as a single disk.
    • RAID 5 appears as one disk, simultaneously writes to three disks and uses one disk for data validation. You should able to recover your data if a disk fails.  It is three times faster than a single disk.
    • RAID 10 appears as one disk, simultaneously writes to two disks and uses two disks for data validation. You should able to recover your data if a disk fails and maybe also if two disks fail, depending on which ones.  It is twice as fast as a single disk.
    • Some people express concern about the reliability of RAID.  I think that has improved greatly over the last 10 years and it doesn’t concern me.  I do keep a spare disk though to swap in immediately in case one fails and I have a portable evaporative cooler in my study to reduce the risk of disk failure when it gets very hot, above 35ºC say.

 

 

The Hard Disk Race for Speed

So how do these disks compare for speed?  Let’s have a race to find out.

.

Asus Hyper M.2 x16 Card, Backup, Computer Upgrades, Computers, Equipment, Laptops, Motherboards, Photography

Serpent Column and Thurmose III’s Obelisk at Constantinople Hippodrome.

.

This is an image from the Hippodrome of Constantinople (ie it’s from Istanbul).  This is where they did their chariot racing, at least until Constantinople was occupied during the Second Crusade in 1202.  The first event of the ancient Greek Olympics is also believed to be a chariot race.  Chariot racing was the pinnacle of speed in the ancient world.

In the background is the Obelisk of Thutmose III, Pharaoh 1479-1425 BC.  It was brought to Constantinople in 390AD, and only the top third survives.  In the foreground is the Serpent Column.  This was originally erected in Delphi in 478 BC to commemorate the final defeat of the Persian army at the Battle of Plataea.  It originally had three serpent heads that supported a golden bowl but the bowl was lost or stolen during the Third Crusade and the heads fell off in 1700.  (The upper jaw of one remains in the Istanbul Archaeology Museum).  Both of these were on the Spina, or the ridge in the middle of the Hippodrome that the chariots raced around.

These days we have various kinds of football clubs (though currently inactive due to COVID-19).  The Hippodrome crowds divided into Blue and Green factions (and originally there were also Red and White).  They were also a bit like criminal gangs, and disputative to the extent of starting civil war.  The Constantinople Hippodrome was 750 metres long and 220 metres wide (including seating).  A lap was about a kilometre and a race was 7 laps, so the race was over 7 kilometres.  There were two very tight turns at the end of the track for each lap.  For the main races, the chariots were pulled by four horses.

 

And we’re off and racing!

The Hippodrome is U-shaped and the mouth of the U houses the starting bays.  We have six chariots lining up for the race, representing different hard disks.  We have a WD Blue HDD, a WD Black HDD, a 4 disk RAID 5 array of WD Red HDDs,  a Samsung 860 EVO SSD, a Samsung M.2 PCIe 970 EVO Plus SSD and a 4 disk RAID 0 array of Samsung M.2 PCIe 970 EVO Plus SSDs.

The 7 kilometre race is over in 8 minutes 25 seconds, won of course by the 4 disk Samsung M.2 PCIe 970 EVO Plus SSD RAID 0.  After all, it is a new chariot pulled by four strong horses.  In that time, the WD Blue has covered 90 metres.  I guess you can’t expect too much from a donkey and a cart.  In the same time, the WD Black covered 125 metres, the WD Red RAID 5 array 265 metres and the Samsung 860 Pro SSD 460 metres.  All of these have been lapped seven times and didn’t make the first turn, at 500 metres.  The single Samsung M.2 PCIe 970 EVO Plus SSD did better, completing one and three quarter laps, or 1,750 metres, and was lapped only six times.  (This is approximately the relative times but they will vary somewhat with different models).

 

Costs of Horses and Chariots (or Hard Drives in modern parlance)

Of course there’s a cost for better and more high tech chariots, quicker stronger horses and more accomplished charioteers, or in general, for increased performance.  If we take one terabyte disks as an example, a WD Blue HDD costs about $70, a WD Red HDD $95, a WD Black HDD $135,  a Samsung 860 EVO SSD $260 and a Samsung M.2 PCIe 970 EVO Plus SSD $350.  (Current prices in Australian dollars, though they may change quickly in these times).  There are always compromises but square wheels don’t help much in chariot races while only the generals can afford horses from the Arabian Peninsula.

So this is why disks are the most important component in computer speed these days.  The kind of disk you choose and whether you combine them can have a huge effect.

 

A more usual approach to a fast photographic PC

Now the normal approach to upgrading the motherboard for a reasonably fast photographic PC would be as follows:

  • You’d want a motherboard with DDR4 RAM and it would be good to have at least two M.2 PCIe ports.  There are lots of choices, one is an Asus Prime H370-A for around $200. It needs to be able to fit into your case though (This one is an ATX motherboard.  You can have micro ATX, ATX and extended ATX cases).
  • Then the CPU, though you really don’t need the fastest and most expensive CPU out.  You might get an Intel i7-9700 CPU, perhaps an i7-9700F for about $550.
  • You’d also need an appropriate CPU cooler from maybe $60 to $150.  (Also needs to fit in the case).
  • Then you might add say 32GB RAM from a good brand such as G.Skill or Corsair for between $230 and $300. (You could also get by with 16GB though 32GB is better for Lightroom and Photoshop).
  • You’ll also need a suitable graphics card but it doesn’t need to be a high end one, perhaps a GTX 1660 with 6GB memory for around $360.
  • Then you’d need to add hard disks which you might be able to transfer from your existing PC. With this motherboard you could add up to six SATA disks, which could include an SSD as a boot drive (C Drive).  Then you have two M.2 PCIe SSDs as fast processing drives.  Alternatively, don’t have the SSD and have one M.2 PCIe SSD as your boot drive and a larger one as your processing drive.

 

The approach I’ve taken

That’s not what I’ve done.  I’ve opted for seriously fast processing disks, including four M.2 PCIe SSDs combined in a RAID array.  This is the chariot and four horses that can blitz the field at the Hippodrome.  To do that I’m using an Asus Hyper M.2 x16 card.  You can see a detailed overview of it in this review (which has six pages, in case that’s not obvious).

– CPU, Motherboard and PCIe Card

Now the card is only $90.  That makes it sound like a cheap option and people have made that mistake but it has four ports and to use all four you need an expensive CPU and motherboard.  The slot on the motherboard that your graphics card goes in is called a PCIe slot.  Specifically, you need a CPU that supports at least 44 “PCIe lanes” so you can have your graphics card in the first full length PCIe slot and the Hyper M.2 x16 card in the second, working for all M.2 PCIe SSDs.  So in terms of Intel CPUs, neither an i7 9700 nor an i9 9900 fit the bill and I went for an i9 10900X ($1070).  For the motherboard I went for an Asus Prime X299-A II ($600).

– HDDs

On my current computer I have 6.7TB of data, including 6TB for images.  I have a 2TBx4 RAID 10 array which gives me 4TB of storage plus a 6TB WD Black and a 500GB M.2 PCIe SSD.  I currently store 2.9TB on the RAID 10 for images, 3.4TB on the WD Black for older images and other files, and 365GB on the M.2 SSD for the Lightroom Catalogue.  I also have a 256GB SSD for the boot drive.  Note that it is essential to have at least 10% free space on a drive and preferably 20% or it may become unreliable.

On the new computer, I can have up to 8 SATA drives.  Unlike the old PC, I can’t have single SATA drives as well as a SATA RAID, so I’m adding two WD Red 2TB drives that I already have to create a RAID 5, array using six 2TB drives giving 10TB of storage. This should survive a drive failing and I’m getting another 2TB WD Red so I can swap it in if that happens.  The single 6TB WD Black will now be surplus.

– M.2 PCIe SSDs

On the new computer, I can also have up to three M.2 PCIe SSDs on the motherboard, plus the Asus card gives me another four M.2 PCIe SSDs.  I can also combine the M.2 PCIe SSDs on the Asus card in a RAID array using Intel Virtual RAID from CPU or VROC.  So I’ll have four 500GB Samsung 970 EVO Plus SSDs on the card in a RAID 0 array, plus a separate 970 EVO Plus and also my current 960 EVO in motherboard slots.  I’ll use the separate 970 EVO Plus SSD for the Lightroom Catalogue, while the 960 EVO will now become the boot disk.

RAID 0 gives a very fast drive but if any of the component SSDs fall over, I lose all the data. This may be a bridge too far for some people but here is why I am comfortable with it.

  1. I have everything backed up to a local Drobo and to the Cloud.  The Drobo is a black box of drives connected to the PC while for my Cloud backup I use CrashPlan though for most people BackBlaze would be sufficient.  For more on backup see my article Backup for Photographers.
  2. I will have a copy of the RAID 0 images on the SATA RAID 5 drive.  For both Lightroom and Capture One, I save changes to the catalogue rather than to sidecar files and I don’t intend to store my catalogues on the RAID 0.  So to see those images and changes, I could simply point the catalogue to the SATA RAID 5 drive.  The only files I would need to restore from backup would be JPEGs and TIFFs I export from Capture One to Lightroom, and TIFFs I have generated in other programs such as Photoshop, Zerene Stacker, Autopano Giga or SNS-HDR.
  3. The SSDs have a 5 year warranty so failures should be infrequent.  In the event of a failure I can swap in the 970 EVO Plus SSD from the motherboard or should be able to buy one locally the same day.
  4. When I regenerate the RAID 0 it would be quite quick to repopulate it.  I would start by copying the images from the RAID 5 drive (which as an internal RAID drive operates faster than from backup), then restore other files from local or Cloud backup.

.

There is also an option for people with older PCs to add an Asus Hyper M.2 x16 card to get an M.2 slot even where the motherboard doesn’t already have one.  Your CPU won’t support a VROC RAID but depending on your CPU and the PCIe lanes on your motherboard, the card will show one or two M.2 SSDs, though not the full four.

.

Asus Hyper M.2 x16 Card, Backup, Computer Upgrades, Computers, Equipment, Laptops, Motherboards, Photography

Iceland Farmhouse uploading to The Cloud.

 

Intel, AMD and Complications

My preference would be to go for RAID 5 rather than RAID 0 on the Asus card. You could then replace a disk if one fails without losing data, at the cost of some loss of speed.  However, Intel make things unnecessarily complex and expensive and seem to want to make you purchase an additional hardware key, which is a dongle you plug into the motherboard.  Only Intel SSDs will work at RAID 0 without the key and if you want RAID 5 with Intel SSDs, you need an Intel Key that costs $200.  The Samsung SSDs have a reputation of being faster and more reliable than Intel ones.  However, you also need a standard hardware key for RAID 0 ($170) or a premium one for RAID 5 ($400).

AMD motherboards don’t have this problem, you don’t need to pay extra for the RAID, so I checked out an AMD alternative, specifically an Asus Prime X399-A motherboard ($460) and an AMD Threadripper 2920X CPU ($800) which is quite a bit cheaper than the Intel alternative.  The problem for me, mainly for my SATA RAID requirement, is that even though you don’t need a hardware key, they only support RAID 0, 1 and 10 and not RAID 5.  I guess I could have gone that way but I would have had to buy a new set of larger disks for a SATA HDD RAID 10 array which may have ended up costing more than the Intel option.

 

Concluding Comments

The prices I cite above are what I find at the current time.  Where they will go remains to be seen.  In the short term, the decline of the Australian dollar should see prices rise significantly.  Conversely, the effects of a potential World Depression are anyone’s guess and supply might become a problem for some components.  In the normal course of events, technology bounds along and prices get cheaper after a year or two.  That may be delayed.

You can buy an off the shelf PC or get a custom one.  I choose to specify a custom one myself and get a commercial person to install the components.  If specifying components yourself, as well as reading all manufacturer specs and the manual, it is a good idea to consult the Qualified Vendor List (QVL) to make sure the components are compatible.  However, if we go to full lockdown, it may be difficult to get installation service or even to get delivery of parts.

I don’t claim to be an expert, I just investigate what seems to be appropriate, do my research and follow my own path.  I’m also not responsible for what you may encounter if you follow my example.  Not many people would need a PC like this one, it all depends on your processing requirements and your budget.  What you already have may be quite sufficient.

Given that many of us are now facing an enforced period at home, some of you may be considering upgrading your PC.  Also consider my earlier article, Computers for Photography which is more general and contains useful links.

Feel free to make comments or ask questions, though, to which I will respond to the best of my abilities.

 

 

Appendix 1: About computers

 

Appendix 2:  Lessons of my upgrade history

My PC is ten years old but all that’s left of the original build is the case, the power supply and a couple of CD drives.

Original spec January 2010:

  • Gigabyte GA-X58A-UD7 motherboard
  • Intel i7-965 chip
  • Gigabyte PCIE 2.0 GTS250 (x2) video cards (not in SLI; one for each monitor)
  • 12GB DDR3 RAM, 1600mhz
  • 80GB Intel X25-M SSD for C Drive
  • WD 1TB Caviar Black
    • 4 for RAID 5 array
    • 1 for additional disk (Photoshop cache, etc)
  • Silverstone FT02 case
  • Silverstone Strider Plus (ST1000-P) 1000W power
  • CD/ DVD Player x2

The RAM failed at some stage and I replaced it with 24GB (6x4GB) DDR3 Corsair Vengeance 1600MHz

October 2014: replaced Intel SSD with Samsung 850 Pro 256GB as the Intel SSD had become too small

January 2015:  Lost two HDD in the RAID 5 and one in the Drobo within a week when temperature was very hot (>35ºC)

  • Lost all data on the RAID 5
  • Replaced WD Blacks in the RAID5 with 2TB WD Reds for a four HDD RAID 10 array
  • Installed a portable evaporative cooler in my study to keep it cool on hot days (the house one ineffective there)
  • Restored the Drobo easily enough and recovered data but discovered a hole in my backups not covered by my next-line remote HDDs.  Could not recover all files.
  • Some time later worked out how to recover the lost files from 1:1 Lightroom previews using a Lightroom plugin by Jeffry Friedl.

December 2016:  My CPU melted down, due I was told to faulty installation of the CPU cooler.  Needed to replace it and my motherboard.  Replacement included these components:

  • Gigabyte Z170X-Gaming 7 motherboard
  • Intel i7-6700K 4.0GHz CPU
  • Gigabyte (NVIDIA) GTX 1060 3GB Graphics Card
  • G.Skill Ripjaws V 32GB (8GBx4) DDR4 2133 RAM
  • Samsung 500GB 960 EVO M.2 SSD

June 2017:  Subscribed to CrashPlan for Cloud backup.  Stayed with it when it moved to CrashPlan for Small Business

March 2020:  System working fine but replacing motherboard because it is no longer compatible with RAID due to Intel software change.  Also M.2 SSD seems incompatible with critical Windows 10 upgrade, probably associated with Intel RAID software change. New motherboard and components as above.

  • If you’re specifying components for yourself it’s important to drill down in the details of each component in the Manufacturer’s site to make sure they are compatible.
  • I was initially going to specify two 32GB kits to get to 64GB RAM but discovered this is not a good idea.  Both motherboard and memory manufacturers warn against it.  For example, I discovered a blog post by the Technical Marketing Manager of Asus who says that 80% or people who do this have problems and most vendors don’t know to advise against it.  (Perhaps that’s why my first lot of RAM failed some years ago.)  So I went for a single 64GB kit.
  • I’ve purchased Samsung M.2 PCIe SSDs that may not work in the Asus Hyper M.2 x16 card without a Hardware Key dongle.

Early April 2020:

  • My new PC is up and running but I still have some work to do.
  • I have a fresh install of Windows so I have to reinstall all my software, which will take a while
    • That was a mistake on my part.  My backup program Acronis has a capacity to restore a computer image to different hardware but I omitted to set that up.
    • I have seen people advocate fresh installs of Windows, so maybe it’s a good idea anyway..
  • As a matter of priority I’ve backed up a system image and created a boot USB.
  • When I went to create the SATA RAID I discovered that what Asus call a RAID 5 is actually a RAID 5E.  This means it has two reserve drives rather than one, which is fine, but means I need a extra drive to get my required disk space.  Then I’ll need to restore all the data to them, which may take a day or two.
  • Most user reviews I read (though not all) suggested that non-Intel SSDs will work at RAID 0 on the Asus card without a hardware key, so I didn’t order one.  However, that didn’t work and also the SSDs are not working properly on the card as individual drives.  Onlt two of the four are available for formatting as drives.  So I’ll exchange them for Intel ones, which hopefully should work fine.
  • Most online user comments I found (maybe 90%) also suggested that the Qualified Vendors List (QVL) is just a list of components the manufacturer has tested and other choices are probably fine.  I went for a 64GB memory kit that was on the QVL, but it’s only giving me 32GB.  I’ll exchange it since it is on the QVL.  In the meantime the 32GB RAM from my old motherboard works although it is not on the QVL and is at the minimum spec for the motherboard.
  • So, it’s a complex process and sometimes a bit slow but with the prospect of useful results.

6 comments on “The Need for Speed

  1. waaay too technical for me. I’m sure glad you know what you’re doing…..

    Liked by 1 person

  2. George Mac says:

    Hello Murray, way over my head as well 😀 I just bought an all singing and dancing Dell PC because the nice people there seem to know about these things and put together a few nice machines. It’s an XPS 8930 and works rather well. Quiet too. Enjoy the build.

    Liked by 1 person

  3. Robert Hayes says:

    Hi Murray! One hell of a task you experienced. And like George up there, it’s way over My head too. When I have a computer problem, I take it to a guy I call the Wizard. He’s earned it! :). My last upgrade went from a 3 tb hard drive to an 8tb. It’ll take at least a week to fill it. 🙂
    Stay safe with the virus out there.

    Liked by 1 person

    • Murray Foote says:

      Well, I did use to work in IT as a systems developer though I didn’t specialise in hardware and my qualifications are actually in Economic History. We’re in lockdown here and there were no new cases in the ACT yesterday so we’re fine for the moment.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s