Hey everyone,

I've been running a debian etch system for several years now with 3x1TB drives in a software RAID5 array. I bought three 3TB drives to replace the existing drives, and I think I've run into the "2.1TB barrier" based on some googling. I see this in the boot messages:



Do you guys know what I need to do to use these drives? They are recognized correctly (it seems) by my SATA controller card, but I have noticed that I can't boot the system when all drives are connected to this card (even just the 1TB drives). It seems my motherboard is not supporting boot to this controller card very well and requires a disk plugged in to the motherboard. My plan is to leave a 1TB drive as a boot/operating system device connected to the motherboard and would like to have the 3TB drives each with a single RAID partition in a 3-disk array. Then I can just copy the boot disk to the other two old 1TB drives for physical backup rather than dicking around with RAID1 for the OS.

Is a kernel upgrade sufficient? Some googling leads me to believe that a different partition manager is required. Would it be easier to just upgrade the entire system? Will the latest version of debian support the big drives? Is there a different distro you would recommend?

I'm using RAID1 partitions for /boot and / on the existing drives (4 partitions total, including the big storage raid5 partition and swap). If reinstalling is easiest, my plan would be to reformat the / and /boot partitions as non-RAID on an existing disk, leaving the large RAID5 partitions alone, then install. I'm assuming a new version of linux could use the existing raid5 partitions and array just fine?

I have backups of the data, but it's very slow media. I'm trying to get this done leaving the data on the RAID5 filesystem intact.

Hope all this rambling makes some sense...

Thanks in advance!

Jim

You need (1) a recent (last year or two) Linux kernel, (2) to enable CONFIG_LBDF in the kernel configuration, and (3) use a filesystem capable of addressing that amount of space (eg. ext4, xfs, jfs, ...).

Cheers

For partitioning, you can either (1) partition the drives and then construct RAIDs from the partitions, or (2) you can first RAID the (whole) drives, and then partition the RAID, or (3) not bother partitioning at all (one massive filesystem).

I think you may need to use something other than MSDOS partition tables if partitioning the raw drives.

Correct, partitions must be GPT (the "newer" disk format introduced alongside EFI back in 2000), instead of the older MBR format (introduced with the IBM PC in 1981).

So it sounds like I should just install a new system on the 3 drives and then copy over the data...

I don't think I understand how to RAID whole drives. Can I make a software RAID array using /dev/sd1, /dev/sd2, etc? I didn't realize that.

I'm fine with one big filesystem comprised of the RAID5 array, especially if I'm using another disk for the OS disk, which will be required for this motherboard, it seems.

I guess I get to learn about a new partition manager, too.

Thanks for the help, guys.

Jim

Actually, the raw drives will have names like /dev/sda, /dev/sdb, and /dev/sdc. Just drop those into your RAID configuration, and then make one massive filesystem on the resulting /dev/md0 device. Or partition /dev/md0 if you want to break it up into multiple filesystems.

Cheers

Of course. I got my nomenclature backwards - the numbers are partitions. OK, I get it.

Thanks guys!

I don't know the pros/cons of RAIDing the whole drives and then partitioning, versus partitioning first and then RAIDing the individual groups of partitions.

There may be some decent rationale for doing the latter.
One reason to partition first, is that it allows using RAID1 (bootable!) for the operating system partition, and then RAID5 for everything else.

But I prefer a separate flash/SSD drive for the boot/OS.

Yeah, about that :-)

So with the old 1TB drives, I partitioned first and then made the arrays for exactly that reason. My /boot and / partitions are RAID1.

With this new build, I want to use one of those disks as a system drive (I like your idea of switching to a SSD for the OS), and decommission the other two, using them as offline backups of the system drive.

How can I switch from a RAID1 md device, to a regular partition? Or, should I just fail the two other drives in the array and let it stay a RAID1 md device with just 1 drive in it? Can I clobber it and change the partition type of those partitions to regular filesystem partitions and NOT damage the big RAID5 partition?

Thanks again for the help!

Jim

I'm not really a sysadmin expert, nor a RAID expert.
There's probably some nifty way to just retag the partition as a non-RAID filesystem, but I've never tried/done it.

Copy off the RAID5 data to the new drives first, whatever you do.

Then your old 1TB drives will all be "available" for experimentation. I would simply partition and mkfs on a new drive (or a repurposed 1TB drive) and then use mirrordir or rsync to copy everything over to it, and re-run grub-install afterward to make the "new" filesystem bootable.

But like I said, I'm not a sysadmin, other than for my own stuff.

Cheers

You can't 'migrate' from md <-> non-md

You should remove the md tag (look at man mdadm and --zero-superblock) before running mkfs though.

The superblock can be stored at various locations on disk so simply wiping the first few bytes or remaking the partition table won't work reliably.



You could do that. A bit messy but not a real problem. Also allows you to add another device later.



Yes.

Also note the old partition table mark (0xfd) is deprecated which is rational but kinda annoying - the suggestion is to use 0xda now.