I will shortly be advising my engineer about setting up a backup system at one of the local TV stations. (Clear Channel owns the TV station in addition to the four radio stations where I work.)

The data to be backed up is currently on a striped raid-1 system, but he would feel better if it were also backed up on an independent machine somewhere else in the building.

The catch is... there is about 500--600 GB of data to back up. Most of this data does not change day to day, so an incremental system would work best.

The question is: At the most basic level, should we be looking at a tape system (ick!) or a couple of big hard drives, or ??

Most backup software I have used is overly complex and cumbersome, offering options for every scenario regardless of how unlikely. All we want is to copy the data on a bunch of hard drives over to some storage media. Xcopy does everything we need (even the incremental part with the /d parameter). The only problem with doing it that way is that once a file is copied over, it is on the destination drive forever, even if it is deleted from the source drive because it is no longer wanted. So, once in a while you have to reformat the destination drive and do the full 500 GB backup from scratch.

My inclination is to just set up a terrabyte of hard drive as a destination, and use Task Scheduler to run Xcopy once a day in the wee hours.

Am I overlooking something? Is this too simplistic to work?

tanstaafl.

Can't you just use plain regular Windows Backup? That's what I use on the server at work. It does full and incremental and you can backup to other harddrives (and it will overwrite the old backup).

Stig

Well, the major point of tapes is that they are easily removable to be put offsite in case of a catastrophic failure. If you don't want to do that, you can still use a real backup solution to write to disks instead of to tapes.

One of the problems with using something like xcopy is that it won't back up any of the special system files. If it's just data that you're comfortable simply moving to a new machine, okay. It may also have trouble with "open" files; I'm not sure about that one.

Honestly, I'd go with a real backup solution. These homegrown things always fail. I'd go with a tape library, too, but a big enough library to deal with 600GB is going to be a hell of a lot more than a remote file server of the same size. Then again, ClearChannel isn't exactly cash poor. But I bet they're cheap, huh? I'd guess that a copy of Backup Exec and an 8-tape LTO-3 library would cost you about $7500.

Look into RoboCopy, it is part of Windows 2000 resource kit. It is quiet similar to xcopy in many ways but also include a mirror command so stuff you delete on the source is deleted from the destination.

and if you have large files that are changing part of the contents their each day take a look at rsync

I've been pleased with rdiff-backup so far. I use it to back up data from my server at home to my remote shell account where my shell provider does the quote-unquote "real" backup.

rdiff-backup is nice because uses librsync to intelligently handle file differences, it does incremental backups, it lets you easily specify which directories to back up, paths/extensions to exclude from the backup, etc.

I know folks here are steering you towards tape, and with good reason. But if cost is prohibitive and you need to just backup to another system (whether it's local to your office or hosted elsewhere) I think rdiff-backup is a good choice, and it addresses some of the limitations of conventional HD backups that matthew_k mentions above.

I've never heard of anyone doing a full weekly/monthy/yearly rotation of hard drive backups.


Actually, I'm sort of doing that now for some of the radio station data. (Not the TV station data that started this thread)

You have to understand that none of the data I am backing up is mission critical stuff. We'd grumble and complain if it were lost, but life would go on, and in many (if not most) cases it would be an improvement! These are Employee Folders with work-related files in them, but these files are the precursors to the actual valuable data that is managed and backed up by someone else. In addition to the work files, there are the "other stuff" that the employees clutter up their folders with -- pictures, music, correspondence, etc. We are not the kind of company that insists that company resources be used ONLY for corporate business. Sometimes we have DOOM death matches on the network in which the General Manager participates (and gets his ass kicked!) so all in all it is a good place to work.

The employee folder backups I do are as crude and simple as you could ever imagine: I Xcopy the entire Employee directory to a 330 GB USB hard drive connected to my workstation. Each day I copy the entire thing to a new directory on the USB drive, with a naming structure of BackW1D1, BackW1D2, BackW1D3... all the way up to BackW5D7, at which point it loops back around to BackW1D1 and overwrites it. So I always have a 35-day history of complete employee folder backups. (That is an over-simplification, because, for instance, only four months of the year will there be five Sundays, except once every 28 years (has to do with Leap years) there will be five months with five Sundays.) Let's just say that I always have a month's worth of employee folders saved. The nice part about this crude system is that it runs completely unattended -- I never have to do anything, it just happens. 35 Windows Task Schedules call up 35 different batch files, crude as hell, but it works!

The backups we want to set for the TV station are of a similar nature -- just much larger files (.AVIs for commercial production and programming, that sort of thing, along with employee folders.) Again, useful stuff but not mission critical. What we want is to be able to avoid total disaster in case of a complete server meltdown. As in the radio system, accounting/billing etc. are not on the server in question. This is useful but not critical data that would be good to have a copy of somewhere other than on the server. To that end, I think the solution would be a couple of large external USB hard drives that can be left turned off most of the time, just fired up once a week or so to update the backups, and a couple of times a year reformatted and re-copied from scratch.

tanstaafl.

Sounds like you know what you need. There isn't anything cheaper or better than hard drives these days. I'd be tempted to build a backup server and use rsync instead of xcopy, as someone else sugested.

Matthew

Unless you simply can't afford to spend $7500, it sounds like you definitely want a real backup system. You're talking about catastrophic failure. A good portion of catastrophic failures are environment related: fires, flooding, hurricanes, etc. And if your backup is sitting right next to your server, it does you no good.

In addition, even if you want to replace the tapes with large USB drives, I still think you'd be better served by real backup software.

But to each his own. Sounds like you've already made up your mind about what you're going to do. I feel for the next person who deals with the system, though, and can't figure out what you've done.

and can't figure out what you've done.


What's to figure? A yellow sticky note on the server saying "There is a copy of the stuff in this computer in Room #123" would be enough to do it!

tanstaafl.

Sure, as long as no one ever has to maintain your 35 different scripts and their 35 different startup configurations.

Your right Andy, rsync would do a better job here, I had robocopy on the brain as we are using it on our new NLB web cluster.