This past winter, I was copying various huge media files to a portable 1TB drive formatted with NTFS, basically filling up the year's entertainment quota for my inlaws.

This was with a 2.5" 1TB WD notebook drive in a Vantec USB2-only enclosure.

All went well, albeit slowly, and I unmounted and sync'd the filesystems, and shut down the computer. The next morning, none of our computers could identify a useable filesystem on the drive.

I was not impressed by NTFS, but it's the only format their computers could read from when 4GB+ file sizes are involved. So I reformatted it, and copied back about 400GB of stuff AGAIN.

All was well, and nothing heard from again until this past weekend.

Months ago, they apparently had plugged the drive in, shuffled a few files around, added more, etc.. and then retired the drive for later viewing.

When we hooked it all up to the WDTV player for viewing on Saturday past.. nothing. No files, no filesystem, same as before.

The physical WD 1TB drive is fine. And my mistrust of NTFS may still be prudent, but that also wasn't the cause.. It's pretty much got to be the USB-SATA bridge chip inside the enclosure.

Most likely a 32-bit (signed) bug of some kind: Allowing for negative numbers ("signed") means one ends up with only 31 useful bits out of 32, which just happens to be the number of bits needed for keeping track of sector addresses on a 1TB drive.

That *should* be enough, especially since "1TB" in drive-maker-speak is really less than 1TB in powers of two. So one shouldn't expect that to be the issue, but perhaps they've also "stolen" one more bit for housekeeping purposes, leaving only 30 useful address bits.. max capacity 512GB.

I told my in-laws to toss the enclosure and let me get them a new known-good model from DealExtreme.

Beware.. though I'm not sure how one might "know" any better.

Cheers

Strange. I've never heard of NTFS doing anything like that but I've also never tried it in a USB enclosure on that size drive.

Is it really resolved at this point though? You have a theory at this point...

Those enclosure can give you the strangest results. Remember this problem I had a couple of years ago? The base of the problem was also an enclosure doing strange things...

Nowadays, I try to avoid those things as I don't trust them anymore. Or I use eSATA.

The googling I did for that specific enclosure suggests it corrupts data above 512GB (some silly chip designer figured 9 bits were enough?), so I fully expect that to be the issue.

But no, not 100% certain -- I have neither the drive nor the enclosure to verify with.

But not NTFS. That part never really made sense to me, as little as I may trust MS with anything important.

exFat would be an alternative that you could use. Supported by MS systems, and it looks like there is write support for it in Linux. Works around the older FAT issues of large files and large disks.

Ok. Cool. I've found for the most part NTFS to be quite reliable over the time I used.

The only time I did have problems with it was (I think) when I had created lots (many thousands) of hard links using rsync to create multiple image backups. If the file didn't change rsync created a hard link to the last backup which might have been a hard link to the backup before that and so on. It was corrupting the Master File Table somehow and the whole thing fell over. I guess that makes it not the best thing then

But yeah exFAT is probably the best option otherwise. Seems there's a patent issue with Linux according to Wikipedia though.

However won't the underlying problem still remain? Isn't it the controller that's fundamentally broken accessing the large drive?

NTFS should be a pretty reliable filesystem it's used on thousands of servers under heavy loads all the time. As an aside anyone everyone play with NTFS Alternate File Streams? I still have this on my list of interesting things to mess around with. Could be an elegant way of storing thumbnails for images and even a sneaky way to do store licensing data and other "secret" data.

I've played with it a little. Seems to work fine.

That said, the new version of NTFS (or its successor) is said to get rid of this feature, I believe, along with some others, like hard links, again, I believe. You might not want to depend on it.

Everyone. They're used to implement zone identifiers (where Windows knows that you downloaded something with IE and refuses to open it). In other words the "Unblock" button you often see in Properties.

Also, they're likely to be deprecated. The new filesystem in Windows Server 8 (ReFS) doesn't support them.