Ah! That makes sense now. RAID is for when you can't afford downtime with your computer, and data safety is taken care of another way, i.e., backups. Thank you.

tanstaafl.

The only salvation there is if you look at the pattern of the failures. Most failures are DOAs, and of the remainder most of those fail within the first couple of weeks. It looks like if you make it through the first couple of months you're home free.

Some of the bad reviews are also related to misunderstanding, such as confusion over which drives are TLER, and apparently some of the Western Digital drives (the Advance Format drives?) are not happy in a non-Windows environment. Especially you want Vista or 7 with Advance Format.

If you do enough research, things don't look quite as grim as they seem at first glance.

tanstaafl.

You are correct, it will take an incredible amount of time to perform the initial backup. Upload speeds really are pretty terrible all over North America, so you kind of get screwed there.

It took me about a week to back up all my data, and that was with 5Mbps up. But the whole idea is that after that, it's only a little each time you add/change files.

Also - and I don't know how/if Carbonite does this - but with Mozy you can set/schedule upload speeds and set upload times. And if the upload is ever interrupted, the program will just pick up where it left off. It's really set-it and forget-it.

But I do understand that you don't have the biggest outbound pipe, so that would be a concern.

Yes, those write-ups had me concerned at first. But eventually I figured out that these were probably all due to having UPS deliver a poorly packed and highly fragile mechanical marvell over long distances.

All brands seemed to have the same DOA complaints, as did most large online vendors.

The local shops sell tons of the same drives, and claim nary a single return on them (do I believe them?).

So, I decided to get the big drives, paying my own cash for the first time in over a decade, but made sure to buy them locally. Bulk shipments to shops generally come much better packed than single drive shipments to consumers, so one would expect the DOA rate to be near zero there.

Cheers

No. Your application, "backup drive", is exactly one scenario that the Greens were designed for. These are the same drives that WD puts into their various external USB/eSATA enclosures now, which are what most folk buy for backup drives.

Despite the Green being touted as "not high performance" everywhere, the one I have here is the fastest drive in the rather extensive collection of mechanical drives here. I think it's even faster than the 1TB Black drive here. At least for the uses I put it to.

Faster than the 750GB Seagate and Hitachi "performance" drives, too. Speaking of which, I plan to never pay my own money for 3.5" drives from either of those two makers.

Cheers

That surprises me. The last couple HDDs I got from Amazon were packaged in between two plastic pieces that kept the drive suspended and then so many packing peanuts that I couldn't even dig around with a couple fingers without the damn things going everywhere.

For Bruno:

Specifically, I got them from a local walk-in shop of canadacomputers. If you visit the link, click on "Contact us (at the top) to get a list of shops with google maps. There are several locations not far from you.

Cheers

My solution: I keep a Firewire RAID box (made by LaCie) at work that's big enough to hold all my home backups. When the image I want to push is too large, I simply carry it home, run the rsync locally, and then carry it back to work. Smaller incremental rsync ops run faster.

Even if you don't have a work office where you can get away with that, you could certainly leverage a friend's house for the same sort of thing.

That's definitely another good solution. My basic suggestion was: get your backup out of your house! Having a backup drive right next to your computer only protects from hard disk failure, and even then just against single disk failure. You'd better hope your backup drive is healthy too, if you ever want to restore.

So yes, use a friend's or relative's house

If you have an Ipod or suchlike, you could look into "rsync --only-write-batch" which is for solving exactly that problem. (And which was added to rsync following a suggestion made on their mailing-list by, uh, me.)

Peter

Mark, I buy all my drives from Canadacomputers. I have for about 12 years actually.

When I worked in Thornhill I was first visiting their Pacific Mall location in Markham. Then I started going to their Richmond Hill location as I also conveniently moved near there. A while after moving to Vaughan they opened a location there as well. Now living in Milton I drive to the Misissauga location. I've heard a rumor that they're supposed to open a location here in town, but I haven't seen anything announced yet.

The past four drives were actually the exception, I bought them from NCIX, and knock on wood, I didn't have any issues with shipping (well packed and delivered by Purolator).

Damn, Doug. I didn't know you thought that way. At least you admit that

I have a pair of USB external disks, one of which is at home, the other in my desk drawer at work. About twice a month, I use rdiff-backup to push a backup to the disk at home, then I take it to work and swap it for the one there.

I only have about 500GB that needs to be backed up, so I'm not that fussed about space (yet), and the twice a month incremental rarely takes more than an hour.

At some point, I'll see if I can persuade the in-laws to let me put a server and external disk on the end of their broadband, but it's not that urgent.

Yeah, but what I really want to know is how did all these people who send me emails every day find out that my penis was too small? D'ya think they're Blacks?

tanstaafl.

This is all great info. My NAS is running out of room (46 gig free of 3.5TB) and I'm looking into 2TB drives as well. I've learned a lot reading this thread and I thank you all for that. I've currently got 5 WD10EADS drives and was looking at the WD20EADS as replacements. However, I think I'll go with the WD20EARS based on this thread.

How do I figure out how much my usable space will increase by swapping one of my 1TB drives out for a 2TB? Basically, I'd like to calculate how large of a timespan I can spread my 5 drive upgrade over. I think it's a RAID 5 configuration.

If I'm not mistaken, your RAID volume size will stay the same as it is now if you upgrade only one of the 1TB drives with a 2TB drive. That's because the system still needs to protect against the failure of the largest drive, your new 2TB. You'll need to put in 2 of them before seeing more space.

Unless the NAS in question does something very clever (like the Drobo ones do, for instance), your usable capacity doesn't increase until you've replaced all five drives in the RAID5 set.

Peter

Or ReadyNAS, but I suppose only with their modified XRAID2. I thought QNAP and Synology also had something similar, so I guess I take it for granted that everyone is running this type of setup.

So, with a hybrid setup two drives need to be replaced and with a standard RAID5 setup, all disks need to be replaced. However, with standard RAID 5, don't you also have to create a new volume within the new space? In other words, you can't expand the original volume without rebuilding everything (which would require a backup and reformat).

Even with ReadyNAS XRAID2 (latest version) there are still caveats - the replacement drives need to be larger than the largest drives already in the system for expansion. Not so with Drobo. Then again, Drobo is like a time-bomb...

Again the answer probably varies with the cleverness of the NAS. If it includes tools of the Partition Magic or resize2fs variety, to expand whatever filesystem it uses in-place, then it needn't require the creation of a new volume. But yes, most NAS devices' firmware probably doesn't incorporate such tools.

Peter

That's correct Doug. What you actually need is not a RAID setup. Windows Home Server is perfect for your needs. I think it's one of the best products Microsoft has released in the last decade. It's VERY to work with, and does exactly what it promises to do: backup your data. But in a smart way, not just a simple backup.
It automatically takes daily backups of all your windows computers (XP, Vista or 7) and stores the data on its own drives. But the data actually are snapshots, meaning you can go back several days into your backup. Say you've changed a file by mistake, but only notice this after three days, when the data has already been backed up. With WHS, it's a cinch to the backup of three days ago, and recover the unchanged file.
WHS also uses a completely new file system, based on NTFS of course, but -for now?- only used in WHS. In WHS, there are now drive letters anymore, only storage. If the system has one HD of 2TB, you have 2TB of storage (obviously). BUT... if you add a two new 2TB's, then the system immediately adds these to the storage pool and you have 6 TB of storage! This is the difference with RAID. With RAID, three 2TB drives would only give you 4 TB of usuable space, AND adding these two drive would take a lot of time since the entire RAID array would have to be rebuilt. With WHS, it only takes seconds.
Yes, I hear you say, but it doesn't offer any redundancy like RAID does. If one of the HD's fails, you lose the data. Not really. WHS offers the possibility to duplicate folders or files that are very important to you. Say your 'photos' folder is extremely important to you. Then you can set it like so in WHS that is will backup this particularly folder TWICE, on two separate drives. If one drive should fail, the data is still there on the secondary drive.

WHS is also an excellent media server, so there's that on top of the normal server and backup facilities.

It's smart in other ways too, like it can backup up to ten different PC's, but will only backup the same data once, even if it's on different PC's. This saves a lot of disk space.

You can build your own WHS (basically it's just like building any other PC, but you run WHS as an OS, but there are also pre-build models. The one HP sells is particularly nice

Mark, that's exactly the reason why I'm interesed in these drives as well. But I'm wondering: could I use one of those '4k' drives in a RAID array, while the other disks are not 4k drives? Or is this a recipe for disaster? (also, the same question goes for my Windows Home Server, which is not actually RAID, but uses a 'storage pool')

Looks like this is the procedure I'll follow, then.

http://www.qnap.com/pro_features_RLM.asp#ap01

Good to know that I'd need to replace all 5 drives. Guess I should start looking for a buyer for the old 5.

Mark, can you expand a bit on the wdidle3 program?

Since the two WD20EARS drives I have ordered will not be boot drives, does that mean all I have to do (once they are set up and running) is get to a C:> prompt and type in wdidle3 /D ? (This assumes that I have the wdidle3.exe program located somewhere that the system can find it).

An explanation of the LCC problem and why wdidle3 is necessary would be a good thing, too.

tanstaafl.

I don't see why that would be an issue, but then I don't use MS Windows here.

Cheers

All I can say on this, is that WD likes to "unload the heads" after 8-seconds of inactivity, and then place the drives into a lower power state. They call this the "Idle 3" state.

If you are just using these for backups, then no worries with the factory setting.

The problem, is that drives are usually rated for a finite number of load/unload cycles -- somewhere on the order of 100,000 - 500,000.

When one of these drives is used as a general purpose system drive, the load/unload counts can very quickly rise into the tens of thousands over the course of a couple of weeks.

So, WD used to make available a utility, WDIDLE3.EXE, to change that value. Of course this is all vendor-proprietary nonsense, instead of simply using the existing standard SATA power management commands.

WD now make it harder to obtain their WDIDLE3 utility, but web pages from them still suggest it is available upon request.

I googled for it here, and found wdidle3-1.03.zip for my 1.5TB drive. I used this to reset the "idle 3" timeout from 8-seconds to 30-seconds on my own drive, at my own risk.

WDIDLE3 /S30

There's a help file in the .zip archive explaining the various command options.

Cheers

Didn't expect you to. But I also have a QNAP NAS, which essentially runs Linux RAID (I think?). I was wondering if this will automatically align the new 4K drive correctly and make it play happily with the other 512b drives?

Edit: partial answer found here. It seems Windows from Vista and up is ok (most of the time ), but XP can be troublesome. Windows Home Server, which is based on Windows 2003 server, the XP stack essentially, has problems. This will probably be fixed with the new Windows Home Server "Vail" which is due by the end of this year since that version is based on Windows 2008 server.

It seems the only system that flawlessly recognises these new drives and automatically aligns the partitions correctly is Linux. So no surprises there.

Looks like the concept of offsite backup is well talked about, but did want to add my additional thoughts in that it is very critical for anything important. I saw a number of disasters wipe businesses out when I was working at HP in the storage team. Some customers just sadly didn't see the importance, until after a major problem like a flooded datacenter. You have to get data far off site for it to be safe. I remember one story from a customer who had their business in the WTC towers, and their setup had their main datacenter in tower one and the backup in tower two. After the attacks, many businesses were moving the backups at least off the island. For home users, Carbonite and Mozy (among other solutions) provide that offsite backup without much setup or complexity. My personal solution is a Time Capsule at my grandparents house in Colorado. Every night, my old PowerPC Mini powers up, opens the Time Machine backup from the Mac Pro stored on my ReadyNAS, and uses rsync to send up the very important data of mine like all my photos. The initial backup did take a bit, but now that it's there, it's always completing the changes backup overnight.

As far as 4k drives and NASes, still check with your vendor. ReadyNAS Intel based systems added support for them officially with proper alignment in the recently released 4.2.12. Not sure if support for the 4k drives with proper alignment will make it into the Sparc (NV+ and a few others) based ReadyNAS units, but I do know there is at least one more firmware version planned for them.

Also wanted to chime in and say I've been pleased with the Samsung 2TB EcoGreen series. I have 3 of them running in my ReadyNAS Ultra 4, and have put them through quite a bit of testing during the beta period. They make a very weird spinup sound though. Performance seems to be around ~65 ish MB/s when attached via SATA. Not the speediest drives, but they weren't meant to be. I need to do some speed tests on them in the Ultra after a factory default, but from my earlier testing, I was getting native speed from them over the network too.

Well, Win7 and Vista both apparently Do The Right Thing by default, or so they say.

Many Linux distros still do the wrong alignment by default -- putting partition 1 at sector 63 rather than sector number 64 (65th sector). I don't know what your QNAP does, but probably the same.

Blame WD for some of that -- there's a standard way for the drive to inform the O/S of correct alignment, and the beta versions of these drives did that correctly, but somehow it got lost when they went to production.

There's a jumper on the drive to have it internally offset everything by 512 bytes, for use with appliances like QNAP.

Oh.. and scratch the variable spin speed thought -- these drives are actually fixed at 5400rpm, despite WD's marketing lingo.

It's based on Windows 2008 R2, so it's x64 only. I had a play with the preview release a few months ago, and it looked pretty good. It was even usable on the Intel Atom I threw at it. I'm not using it right now, though, because I didn't want to become dependent on a pre-release. When it's out, I'll probably pick it up again.

How is that possible? I thought the Atom was a 32 bit CPU? As you say Vail is x64 only?

edit: your Atom must be Diamondville or Pineview core then?