LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

CalDigit HDPro: RAID 5 and Beyond!

COW Library : CalDigit : Shane Ross : CalDigit HDPro: RAID 5 and Beyond!
CalDigit HDPro: RAID 5 and Beyond!
Creative COW Review


Review

Bob Zelin

Shane Ross
Los Angeles, CA US
©Shane Ross and CreativeCOW.net

Article Focus:
Creative Cow Leader Shane Ross shares the great news: both performance and protection with CalDigit's new HDPro RAID! Supporting RAID 0, 1, 5, and 6, the HDPro even works with your laptop. It's the answer to Shane's prayers. He thinks it might be the answer to yours too.

 

Not long ago...January 2007...I was at MacWorld in San Francisco. There was a LOT of buzz going around about a great many things. The iPhone was announced, Leopard was demoed, a company converted a MacBook into a tablet computer...there was no end to the cool things to see. But all the companies that were there showing off storage solutions...RAID solutions...were hit by one big question.

 

"Does this do Raid 5?"

 

Well, I can tell you, that a VAST majority of the answers were "Sorry, no. Only Raid 0 or Raid 1." This turned many people away. The only people who could say "Why yes, we do!" were the Fibrechannel solutions. But, as you might suspect, those solutions were EXPENSIVE. Restrictively so for many of us.

 

My how technology flies.

 

Here we are...8 months later...and the companies who were at MacWorld (and NAB) heard the cry. One such company was CalDigit. Their offering, the HDPro, offers what everyone desires...hardware Raid 5 support. For a device to offer “Hardware RAID” it must have 2 things: An IO Processor (CPU) and a Cache Memory. This unit has both.

 

 

What is Raid 5?

Well, let us first start with the lower numbers.

 

RAID 0 is what is called performance RAID. What you do is take two or more drives and "stripe" them together so they act as ONE DRIVE. "Striping" essentially means you format multiple hard drives together as one large "virtual" hard drive for your system."

 

For example, if you take four 250GB drives and stripe them RAID 0, you'd have a 1TB Raid. In doing this, you spread out the information that is being written onto all the drives so that you get faster data transfers. This means that you can capture uncompressed video formats and play them back. The more drives you add, the faster the raid, the better the performance.

 

RAID 1 is known as a protection RAID. You take 2 or more drives (need to be in pairs) and group them so that they again appear as one drive, but this time the information is MIRRORED. If you took two 250GB drives and striped them as RAID 1, you'd have a 250GB drive, but the information would be secure because the information on one drive is copied to the other drive as well. The drives are exact duplicates of each other. The good thing is that your media is safe, the drawback being that your drive performance isn't as great, and you cannot really work with uncompressed formats.

 

RAID 5 is the best of both worlds. Performance with protection. RAID 5 stripes both data and parity information across three or more drives. It doesn't have one dedicated parity drive that contains all the information, but rather distributes the data and parity blocks across all the drives in the array. "What the heck did you just say Shane? What does all that mean?" To tell you the honest truth...I don't understand the inner workings of the engineering. I am not an engineer, I am a creative editor. But I DO know that it basically means that your footage and the backup information is not on one drive, but spread across many. So that if one drive fails (and drives are known to do this, more often than we care to admit) your information is still safe.

 

If you want to know more about the RAID types, go to the Wikipedia. Enough with the schooling...let's get to the HDPro itself.

 

Getting started
First off, I must mention that currently the HDPro ONLY WORKS WITH PCIe MACHINES (PCI-X support due by mid-September I hear). This means it works only on the newer MacPros, and the older G5 Quads withe a PCIe slot. It also works with any similarly configured PC.

Now, I don't have one of these machines, I have a PCI-X G5 with a Kona LH capture card. I needed a PCIe machine with capture card in order to do this test. So for the purpose of testing, I was loaned a MacPro (Dual-core 2.66Ghz model) with a Decklink Multibridge Pro capture card.

When I went to unpack the unit, I encountered something similar to what I encountered when I unpacked the S2VR Duo unit. The hard drives were not inside the unit. They were all tucked into little cut out sections of the packing foam that encased the drive chassis. This is clever, for it helps prevent damage that can occur by jostling the unit. We know how gently delivery services are with packages, right?

 

So I pulled out all the drives from their safe cubbies, removed them from the plastic wrap, and installed them into the chassis. They were all conveniently labelled A0, A1, A2 so I knew what order to install them.

 

Another interesting thing is that the card that you install inside the MacPro is a VERY small and basic card. It is a 4-lane one port card, with no real electronics to speak of. The card doesn't have the controller on it. Nope...it is simply a BRIDGE from the board to an external connection that a PCI Express cable connects to. The controller is INSIDE the chassis.

 

To quote the website, "This direct link eliminates latency introduced by the conversion of other interfaces to PCI Express and provides superior bandwidth, availability and deployment flexibility over earlier-generation SCSI and Fibre Channel technology."

 

The reason for this is two-fold. First, it doesn't contribute to the heat inside your MacPro...since you will have plenty with your DOUBLE-WIDE graphics card and your capture card. And second, it allows for controlling MULTIPLE units with the aid of a PCIe switch (not out yet). Meaning that your machine can control more than one hdpro box...you can add more units as the need arises. Just like I can now with my eSATA boxes and my port multiplier card. Room to grow.

 

The test unit I received was a 4TB unit. Eight 500GB drives. When I fired up the unit it appeared on the desktop, ready to go. It was already configured...Raid 5 and everything. So I ran the AJA test with the following result:

 

Read Write 1 

 

331 Write / 372 Read. RAID 5!! Plenty fast for uncompressed HD...a couple streams of uncompressed HD. Niiiiice.

 

 

 

Doing it myself
But, me being me and not trusting the integrity of a drive format after shipping...and, well, to see what is involved in setting up a RAID 5, I did it myself. CalDigit includes a handy little pamphlet on how to do this, so I won't go into the boring details. But I will mention that it did take 3 hours to build the RAID...which is slightly faster than I hear RAID 5 takes to build on an XServe.

 

Do note that the size of your Raid determines how long it will take to build. 8TB will take longer to build than 2TB. Makes sense.

 

So, I let it do it's thing while I ate dinner, played with the kids, and tucked them into bed.

 

When the building was done, I ran the AJA test again:

 

Read Write 2

 

311 Write and 402 Read....again, niiiiice. The Write was a little slower, but who really needs more than 250 anyway? I am not CAPTURING multiple streams...so 300 to 340 is fine. The Read increased, which was cool. And I suppose the average will be about 320 Write and 385 Read...which is VERY nice indeed.

 

I captured a few different ways. First DVCPRO HD native via firewire. No sweat...but I can do that with my G-Raid. So then I captured uncompressed 8-bit 720p...no problems. Uncompressed 10-bit 720p...no problems. Uncompressed 10-bit HD...passed again without one dropped frame.

 

 

OK, time to abuse this thing.
Again I set it up to capture uncompressed 10-bit, and about 5 minutes in...I YANKED A DRIVE OUT! Just so you know, this isn't something you can do by accident. There is a pinhole that you need to push an included tool into (or a paperclip if you lost the tool) and that pops open the handle that you then use to disengage the drive and pull it from the chassis. When I did this it did give me this nice little warning:

 

removal 

 

But, it kept merrily match capturing...not missing a beat. When it was done I saved my project and set about fixing what I intentionally broke. I pushed the drive back in and went into the RAID SHIELD software and unlocked the drive and the software went about rebuilding the Raid. Again, that took about 3 hours. But the footage I captured was still intact and played fine.

 

 

Testing playback
OK, now let's look at playback. I was able to get 9 streams of DVCPRO HD playback (both firewire and Decklink encoded DVCPRO HD)...I resized my footage to look like the opening to THE BRADY BUNCH. The first 4 streams gave me no render bar whatsoever. When I hit 5 streams I encountered the dark green render bar, which means high quality playback. Eight streams got me the light green render bar....low quality playback...and that held up until I added the 10th stream...when it went RED. So i stuck with nine.

Since it was getting late, and I had work in the morning, I just let it play this footage as I went to bed, setting Final Cut Pro to loop the footage.

 

(Did I mention I was testing this with Final Cut Pro? No? Sorry.)

 

I woke up, went to work, came back home, had dinner, then went "oh yeah, I have something going on in my office." I turned my monitors on and there the footage was, happily playing along. No dropped frames.

 

But hey...seriously....DVCPRO HD footage?  Was that a REAL test?  OF COURSE NOT!  So I went to the 8-bit uncompressed 720p footage.  I started layering that.  I got the dark green render bar after two layers, light green after 3 and the red render bar when I hit five.  So four streams it is. 

 

I tried 10-bit uncompressed and couldn't get more than one.  But really...uncompressed 10-bit...I am not offline editing with this codec.  I am FINISHING with it, so one stream is plenty.  I am offlining with DVCPRO HD, and getting nine RT streams is a nice thing to have.  OH, and I was able to add the 3-way color corrector to seven streams and stay RT.

 

Time for bed again, so I resized the uncompressed 8-bit footage into four windows, and pressed play. This time I let it run for 3 days. Every time I came back, it was running.

 

On the third day when I got back from work, I walked in, opened the unit, and yanked a drive. I turned on the monitor and again, saw the warning, but playback wasn't affected. Kept going and going and going. So I let it go. Went to bed. Woke up the next morning to see it still humming along...footage playing. I stopped...saved...quit...and set the raid to rebuild again. And again, when it was done, all my footage was still there...safe and sound.

 

The HDPro unit was running for the entire 8 days that I had it...I didn't turn it off. When I finally did, and I removed a drive and felt it...is was warm to the touch, but not overly hot. The RAID SHIELD software has temperature monitoring capabilities, so you can always check and see how hot things are. OH, and I forgot to mention that you can set up e-mail notification so when a drive does happen to die and you are...somewhere else...you'll get an e-mail telling you a drive failed. Very handy.

 

shield

 

When I was at the CalDigit booth at IBC, I mentioned that it would also be really nice if the unit had an audible warning when a drive failed.  Currently you can set it up to e-mail you if anything unusual is happening, but it would be nice to have an audible warning as well. 

"Oh, we have that!" was the reply.  And Alex Chen, the owner, showed me where this was.  In the RAID SHIELD software, you open the controller and then click on PREFERENCES.  In there you will find an option called BEEP.  Check that and when you hit OK you hear the unit beep twice to indicate that the feature is active.  So when a drive fails, or the unit starts to overheat or other issue occurs, the unit will beep.  Sweet!

Here's something else that impressed me
For some reason I thought it was common among other RAID 5 solutions, but I hear it is rare: after I yanked out the drive and then put it back in and started the Raid rebuild, I realized that I could continue to work with the drive as it rebuilt the raid! 

 

The performance of the raid dropped from 400MB/s to 190MB/s, but I was still able to access footage on the timeline and edit!  It did the rebuild in the background! This means that there is ZERO downtime with this.  VERY handy feature.

 

AND the units aren't only for the Mac.  It’s cross platform, so it works on the PC as well.  It works with Final Cut Pro, Premiere Pro (we ran a PC with CS3 on it playing footage, and Matrox uses the CalDigit units in their booth), and recently it was found to work with an Avid as well.

 

OH! OH! OH! AND...oh I love this...AND CalDigit makes an Express34 card for this unit, so you can use it with your LAPTOP! You won't get nearly the numbers you get with the tower, but the numbers you do get are QUITE impressive:

 

 

 

168 Write and 195 Read. PLENTY for uncompressed SD and even uncompressed 8-bit HD work. And for DVCPRO HD and the new ProRes format, that would be plenty for several streams of playback...ON A LAPTOP!

 

 laptop

 

Man...time for me to get a MacPro...or MacBook Pro.

 

 

Feel free to discuss this review in the Creative COW CalDigit forum.



If you found this page from a direct link, please visit our forums or read other articles at CreativeCOW.net





Comments

Re: CalDigit HDPro: RAID 5 and Beyond!
by Josh Zimmerman
Hello,

I am running a HDPro2 recently I had a drive go bad(drive 1) and replaced it then formatted to RAID 5. Now I will turn on my machine and start transferring data and after about 10-15 mins drive 5 gets"offlined" and the drive disappears from my desktop. I can restart and everything works again until I start transferring data again. Is this my drive going bad? I have updated all the drivers so that should be fine... Any ideas?


Related Articles / Tutorials:
CalDigit
CalDigit AV Drive with USB 3

CalDigit AV Drive with USB 3

Hard drives rarely feel fast enough, but CalDigit's new AV Drive ($258) is one of the first to use the new USB 3 interface, which is much faster than the USB 2, FireWire 800 and eSATA interfaces of yesteryear. You'll pay a little more for the AV Drive, but its speed easily makes up for the premium.

Review
Helmut Kobler
CalDigit
The New CalDigit HDPro2 Product Review

The New CalDigit HDPro2 Product Review

Buckle your seatbelts. In this review COW leader David Roth Weiss, an L.A. based director, editor, and colorist takes you inside CalDigit's brand new HDPro2; an advanced 8-drive video storage solution that will absolutely rock you with blistering performance and sustained video throughput unlike anything you've ever experienced before.

Review
David Roth Weiss
CalDigit
New VR Mini from CalDigit

New VR Mini from CalDigit

CreativeCOW leader Jerry Hofmann takes a look at CalDigit's new VR mini -- serious storage small enough for your pocket.

Review
Jerry Hofmann
CalDigit
CalDigit HDOneCalDigit HDOne

Los Angeles Director and Editor David Roth Weiss finally takes the plunge and tests CalDigits new HDOne, an affordable turnkey RAID-5 protected video storage system.

Review
David Roth Weiss
Recent Articles / Tutorials:
Field Production
“Before I forget: don’t wear any underwear.”

“Before I forget: don’t wear any underwear.”

Before coming to Creative COW, before his lives in product marketing and product management at Avid and Boris FX, Creative COW Editor-in-Chief Tim Wilson ran a video production company. As we also observe the 100th Anniversary of the founding of the US Parks Service, Tim recalls one one especially memorable adventure to Everglades National Park, wherein he found himself quite literally up to his armpits in alligators. He had no idea that this was going to happen when the day began. At the time, he was focused on a brand new fear: getting sliced in half by burning underwear.

Editorial, Feature, People / Interview
Tim Wilson
Art of the Edit
The Science of Editing

The Science of Editing

Sven Pape, aka @ThisGuyEdits, joins Dr. Karen Pearlman -- former President of the Australian Screen Editors Guild and a three-time nominee for Best Editing at the Australian Screen Editors Guild Annual Awards -- for a provocative look at "Editor's Thinking," a cognitive skill set that you can use to improve your screenplay before you start principal photography of your film.


Sven Pape
Panasonic Cameras
Shooting MTV's Mary + Jane with Panasonic VariCam 35

Shooting MTV's Mary + Jane with Panasonic VariCam 35

To shoot the ½ hour scripted comedy series for MTV, Mary + Jane, the producers at Television 360 enlisted cinematographer Charles Papert (Crazy Ex-Girlfriend, Key and Peele), who found that Panasonic VariCam is a great fit for moving fast and getting great images when time and resources are scarce.


COW News
Adobe After Effects
Imagineer mocha Pro 5 Plug-In for Adobe: An In Depth Review

Imagineer mocha Pro 5 Plug-In for Adobe: An In Depth Review

Imagineer mocha Pro 5 Plug-in for Adobe brings all the amazing features of the professional version of the mocha Planar Tracker directly into After Effects and Premiere Pro in the form of a plugin. In this in-depth review, After Effects tutorial guru Tobias Gleissenberger of Surfaced Studio will show you what you can do with this new plug-in, and discuss what he likes and doesn't like about the new update.

Tutorial
Tobias Gleissenberger
Apple Final Cut Pro X
Hawaiki Keyer 3.0 Upgrader Tutorial

Hawaiki Keyer 3.0 Upgrader Tutorial

After 25 years as an editor, compositor, and VFX artist, frequent Creative COW poster and tutorial author Simon Ubsdell knows what he needs from a keyer -- and knew he wasn't getting good enough results from FCPX or Motion. Discussions in COW forums led him to create the highly regarded Hawaiki Keyer for Mac users using Apple Final Cut Pro X, Apple Motion, and Adobe After Effects and Premiere Pro on Mac as well. Enthusiasm expressed by COW members for its latest release led us to ask Simon for a tour of the even more advanced Hawaiki Keyer 3.0.

Tutorial
Simon Ubsdell
Cinematography
All Eyes on IBC 2016 for Cameras and Lenses Galore

All Eyes on IBC 2016 for Cameras and Lenses Galore

What’s that you say? An IBC that’s not only relevant, but downright exhilarating? This used to not be news, of course. However, in recent years, IBC has too often become simply an opportunity for European audiences to see products already announced at NAB. In 2016, however, the focus swings sharply to Amsterdam, especially when it comes to cameras and lenses. IBC 2016 is shaping up to be one of the most dramatic trade shows for cinematographers, broadcasters, and videographers in years. Join Creative COW Editor-in-Chief Tim Wilson for a speedy overview of some of the highlights.

Feature
Tim Wilson
MORE
© 2016 CreativeCOW.net All Rights Reserved
[TOP]