Hey, I thought I would start a new thread for this since the other one I had going didn't have a very helpful title. For those that missed it I managed to get ext3 working on the empeg. Ext3 is a journaling filesystem which has the advantage of not needing to do a long annoying file system check if the empeg crashes during a sync or you forget to mount the drives read only after modifying something. This thread is meant to be a writeup of how to do it in case anybody else is interested. First a few warnings though...

1. This is a very old version of ext3 and there are bugs in it
2. I had to upgrade the empeg kernel to get this to work. Many of the patches did not apply cleanly so I had to hack in any code that didn't get patched by hand. It is possible I made mistakes.
3. I don't know if this version of the linux kernel will cause stability or other problems with the empeg hardware.
4. If something goes wrong this could eat all the files on your empeg
5. This will break IR support (I am hoping somebody here will figure out why)
6. I take no responsibility for any bad things that happen because of this, if you don't understand the directions don't try this.

First of all the bits and pieces. All these files are gzipped so you will need to uncompress them before using them...

1. I had to upgrade the linux kernel from version 2.2.14 to version 2.2.17. A patch to upgrade the empeg kernel tree to that version is at http://macgeek.dyndns.org/empeg/ext3/empeg_v211-linux-2.2.17.patch.gz
2. Next patch is for ext3 support, a patch for that is at http://macgeek.dyndns.org/empeg/ext3/ext3-0.0.3b.patch.gz
3. I also tweaked hijack a little bit to patch on top of the above. A patch for that is at http://macgeek.dyndns.org/empeg/ext3/hijack281+voladj+rds.patch.gz This is optional.
4. A kernel built with all of the above for a mark2 is at http://macgeek.dyndns.org/empeg/ext3/zImage-2.2.17-empeg-ext3-hijack.gz
5. I don't know if the version of fsck that ships with the empeg will be happy with ext3. A newer version of e2fsck that understands ext3 is at http://macgeek.dyndns.org/empeg/ext3/e2fsck.gz You may not actually need this, but it can't hurt.

Ok, so to get this to work first install the kernel and then follow the steps below.

1. Put the new version of fsck on your empeg. I put mine in /bin. Don't forget to make it executable. You should also make the other fsck files links to this to make sure the right ones get called like this...
empeg:/# mv /bin/fsck /bin/fsck.bak
empeg:/# mv /bin/fsck.ext2 /bin/fsck.ext2.bak
empeg:/# ln -s /bin/e2fsck /bin/fsck
empeg:/# ln -s /bin/e2fsck /bin/fsck.ext2

2. You nead to create a journal file on the target drive. There is a utility called tunefs that is supposed to do this, but of course it did not work for me. Here is the hard way to do it...

# First turn on swap, I got a panic using dd without it
empeg:/# swapon /swapfile
Adding Swap: 16596k swap-space (priority -1)

# Make the drive read-write
empeg:/# rwm

# Use dd to make a file for the journal, this creates a 10meg journal file.
# There may a more optimum size
empeg:/# dd if=/dev/zero of=/drive0/.journal bs=1k count=10000
10000+0 records in
10000+0 records out

# Figure out the inode number of the new file. It is that first number.
empeg:/# ls -i /drive0/.journal
12 /drive0/.journal

# Unmount the drive
empeg:/# umount /drive0

# Remount the drive as ext3. The first time you mount the drive
# you need to specify the inode number from above with the journal
# option.
empeg:/# mount -t ext3 /dev/hda4 /drive0 -o journal=12
EXT3-fs: recovery complete.

3. You probably are going to want to update your /etc/fstab file to mount the drives as ext3 instead of ext2.

4. This is the tricky part, we need a way to get the player to use ext3 instead of ext2 when it remounts the drives during a sync.

The first way to do that is to edit the player executable and change the string that says ext2 to ext3. For me this causes a stack trace when the player remounts the drive. It sounds like a good idea, but doesn't work at least for me.

The second way would be to write a kernel patch to intercept the mount call from the player and fix it to do what we want. I am hoping Mark Lord can help out here since he was talking about how to do this in another thread. Mark, you around these days?

The third way would be for the empeg guys to help us out and make the player a little smarter about how it mounts the drives.

5. You may need to replace the empegs init with a script that mounts the drives as ext3 instead of ext2. I already was using a custom script so I didn't worry about this.

Let me know if you have questions or suggestions on the IR driver problem. Those files are only on a DSL line so downloads will be kind of slow.

-Mike

One correction. IR does work fine with this. One of the remotes I was trying to use must have had a dead battery and the other required hijack which I hadn't patched in yet. D'oh! I was pleasantly surprised when I got in the car this morning and my normal RIO remote worked just fine.

-Mike

OK, I managed to figure out how to change the mount type within the kernel. This intercepts any calls to mount hda4 or hdc4 and changes the mount type from ext2 to ext3. The patch is at http://macgeek.dyndns.org/empeg/ext3/mountHack.patch.gz and a new kernel with this change is at http://macgeek.dyndns.org/empeg/ext3/zImage-2.2.17-empeg-ext3-hijack.gz

So... that should be everything needed for anybody interested to get ext3 working on their player. I have been running it for a couple of days now and have done at least a dozen syncs with no problems. It rocks!

-Mike

p.s. to mlord. You can ignore my private message, as you can see I managed to figure it out.

So is there really nobody interested in this? My server shows 0 downloads after 3 days. It seems like there where a few people interested in the other thread I had going. Everybody likes 30-90 minute drive checks? Weird.

Honestly, the only times I've ever had to fsck were times that I was logged into the empeg and forgot to ro it before rebooting. That's my own damn fault and I should be penalized.

I meant to ask earlier what sort of ext3-specific options the filesystems were being mounted with. (Not that I'm sure that it makes a load of difference in this case, since it's just the journalling type.)

Oh, I'm very interested. I just haven't had time to really digest the implications of installing it wrt to any hijack work I want to do. I don't have the luxury of a spare for testing

I'm also not convinced that you needed to change the kernel to trick the player into mounting the ext3 parition during sync. Doesn't the player use /bin/rwm and /bin/rom to achieve this? Couldn't you have edited etc/fstab to also achieve this? I'm asking because this trick is pretty absolute. It means always running a custom kernel to use ext3, and always running ext3 if you are using this kernel, and I was wondering if we could persuade Mark to consider including your work in hijack.

The fstab file is an unused stub, so it won't do anything useful. Not sure about rwm and rom.

However, ext3 is a superset of ext2 and you can switch which way you mount the filesystem back and forth between them with no ill effects at all.

I meant to ask earlier what sort of ext3-specific options the filesystems were being mounted with.

None. Honestly I never bothered to see what other options I may have. I am passing through whatever the empeg guys are using except for changing the type to ext3. Is there anything else useful that I should investigate?

-Mike

I'm also not convinced that you needed to change the kernel to trick the player into mounting the ext3 parition during sync. Doesn't the player use /bin/rwm and /bin/rom to achieve this? Couldn't you have edited etc/fstab to also achieve this? I'm asking because this trick is pretty absolute. It means always running a custom kernel to use ext3, and always running ext3 if you are using this kernel, and I was wondering if we could persuade Mark to consider including your work in hijack.

I couldn't come up with any other way to modify the mount type. I even tried using a custom /bin/mount, but that isn't getting used either. The player does not use the rom/rwm scripts, it calls the kernel function directly and never even looks at fstab. As someone else mentioned it is trivial to move back and forth between ext2 and ext3 so that shouldn't be an issue. I was hoping Mark would step up and make the ext3 mount configurable within hijack so that it can be turned on and off (kernel programming isn't really my thing). I was also hoping he would consider including all this within hijack so I don't have to maintain it separately. But, there are a lot of changes there so it could definitely use some testing first and I am currently the only person using it so that is why I was hoping to get some more interest going here.

-Mike

So is there really nobody interested in this?

I am, but I have been way too busy to even get a serial cable near my empeg. I still need to get d.net working for PaulWay, and some other things done.

Here, you take over the BBS/Riocar.org for a month or two, plus work one of my days at work and I might get to it soon :-)

Seriously though, glad to see this up and running after my failed attempt at ReiserFS a while back. Once I do start hacking the empeg again, ext3 mounts after a crash will be a welcome sight.

You do need to change the kernel. The player uses the mount syscall.

I'm kinda slow tuning in here. Nice job, by the way.

The part that interests me the most is the 2.2.17 kernel upgrade. I'm always interested in newer kernels for some strange reason.

The ext3 is a nice idea, and could be very worthwhile for many of us (PaulG ?). Personally, I'll pass for the moment (just too busy with other $tuff), but if I did convert everything over to ext3, then I'd also nuke ext2 completely from the kernel if possible, to save RAM.

The only glitch I can think of with that is the software upgrades (future) will expect ext2 rather than ext3, and may choke on the superblock since they use an unmodified kernel.

Cheers

I agree, this is very cool. Since I got my Mac, however... I have no serial port, and I don't have time to check this out. The reason that this mainly is cool, as opposed to XFS, reiser, or JFS, is that nothing has to be done to convert the fs (well, other than tunefs, I mean you don't have to reformat) and that an ext2 kernel can mount it if need be.

...but you can't tunafish.

--
The Ghost of man-pages Past.

Ok, I downloaded it today and was just now about to enter the installation process when I saw a new beta of the player ... So being that I want to cause the most hassle and most flakiness, what's the process by which I put ext3 on the new player software?

Greg

Ok, I downloaded it today and was just now about to enter the installation process when I saw a new beta of the player ... So being that I want to cause the most hassle and most flakiness, what's the process by which I put ext3 on the new player software?

:-) Keeping in mind that I haven't actually tried b13 yet, the install process shouldn't have changed from what I posted in the original couple of messages in this thread. You might want to wait until I get a new kernel built (probably in the next hour or so), but it doesn't sound like there were many kernel changes in b13.

-Mike

OK, latest and greatest files are now available for 2b13 and hijack 283. MountHack, ext3, and (obviously) e2fsck did not change. The rest didn't change much. I probably won't be able to test the new version until tomorrow myself, but if somebody wants to beat me to it the files are at.. http://macgeek.dyndns.org/empeg/ext3/ext3-0.0.3b.patch.gz http://macgeek.dyndns.org/empeg/ext3/empeg2b13-2.2.17.diff.gz http://macgeek.dyndns.org/empeg/ext3/mountHack.patch.gz http://macgeek.dyndns.org/empeg/ext3/v283.hijack.v200b13-2.2.17+ext3.patch.gz http://macgeek.dyndns.org/empeg/ext3/zImage-mk2-linux2.2.17-hj283-ext3-mh.gz http://macgeek.dyndns.org/empeg/ext3/e2fsck.gz It is a lot of work to maintain that many different patches. If you aren't compiling stuff yourself all you need is the last two and the instructions earlier in this thread. -Mike

All I have to say is WOAH NELLY! There's got to be an easier way for you to have all those archives ... Between you and Mr. Hijack, I bet you guys spend half your time gzipping and tarring files.

I appreciate you making it so easy for us to update this stuff. You *could* have just given us a diff and told us to compile our own stuff. Thanks a bunch!

Greg

Hmm, for some reason those didn't turn into links. I miss regular HTML. Weird, here they are again with explicit tags...

ext3 Patch
kernel patch to 2.2.17
Mount Hack
Updated Hijack Patch
Compiled Kernel
e2fsck

-Mike

All I have to say is WOAH NELLY! There's got to be an easier way for you to have all those archives ... Between you and Mr. Hijack, I bet you guys spend half your time gzipping and tarring files.

Nah, I spend half my time cross-compiling on a 400Mhz celeron. Now that is painful. Actually I have an ulterior motive in that I hope Mark will start including the 2.2.17 patch and the hijack changes as part of hijack so I don't have to maintain as much stuff. Then I would just be maintaining ext3 and the mount changes which would be a lot simpler.

I appreciate you making it so easy for us to update this stuff.

No problem, I had considered making an installer script for all of this, but I actually wanted to make it a little bit harder to keep it to just people who knew what they are doing until it is a little more proven.

-Mike

OK, I am successfully running b13 with the patched kernel linked above and ext3'd drives. There was only one minor glitch in the upgrade. I assumed that the stock empeg kernel would simply mount the drives as ext2. Instead it complained about "couldn't mount because of unsupported optional features" and did not mount the ext3 drives at all. After upgrading the kernel everything works fine.

So if anybody does change to ext3 and then wants to go back to ext2 you will need to delete the journal file and do an fsck to get the bits cleared so that the stock empeg kernel will mount your drives again.

-Mike

You should be able to mount it as ext2 though. So long as there is nothing that needs to be replayed from the journal. Very odd

- Trevor

Wow this sounds like a pretty easy upgrade... Are there any potential problems with using ext3? Maybe speed wise or something? It sounds like something that everyone should do, and maybe even something that should be considered for future inclusion in the Empeg standard distribution, no?

In reply to:

---

Are there any potential problems with using ext3?

---

I haven't hit any so far. It doesn't seem noticably slower to me. The biggest potential problem that I see is that this is a pretty out of date version of ext3 and I don't really know what kind of bugs may be lurking. I've got one bug that I just discovered. This does not work quite right on a dual drive empeg. When the player remounts the drives after a sync it throws a stack dump and exits. It only seems to happen on my dual drive player and I don't think it happens every time. I may just need a larger journal file or something since the second drive in my dual drive player is quite a bit larger (30 Gig vs 10) than the other drives I am working with. I am not up for fighting with it tonight though. -Mike

Run into a snag Mike.

I updated the empeg with the factory b13 update.

I updated the empeg with your precompiled zImage with hijack and whotnot.

I put e2fsck in the bin, did the moves of the old fsck and linked the names to the new e2fsck.

Now, the next part requires that I have mounted /drive0 and /drive1 ... which on any other occasion this would be true. But evidently upon boot up of the empeg, these did not mount. When I try to mount them with the -t ext2 ... the mount doesn't know what to do and tells me how I should be using the command line (usage) ... when I omit the -t ext2, it simply says there is no journal found on the drive.

What now boss?

Greg

My guess is that it got mounted as ext2 at some point during your upgrade and the info about the journal file got excised. Try remounting it with that ``journal='' syntax from above.

But I haven't even gotten to the point where I can build a journal file yet. In order to build it (at least from what I understand) I have to have it mounted first. And in order to mount it, it wants it to already have a journal file. Like I'm in some infinite loop of my own ignorance here. It's almost like dealing with the driver's license people. They want to see your ID before they'll allow you to get a replacement ID for the one you've lost, but you lost it, so you can't show it, so they won't give you a replacement. (Not quite, but the sentiment is there).

It seems like either I'm missing a step, or as soon as I update the kernel (with the required automatic restart), it thinks I should already have journal files. Is there a way to update the kernel by hand, without rebooting, so that I may build the journal files at that point before it unmounts them for good?

Greg

In reply to:

---

Run into a snag Mike.

---

You are a little out of order. You should create the journal files before updating the kernel. The kernel I provided will always change '-t ext2' into '-t ext3' so you wouldn't be able to mount as ext2 with that kernel. Grab the stock kernel off mlord's site so you can get the drives mounted as ext2 and then do the journal step before going back to my kernel. Once you have the journal files created figure out what inode they are at with 'ls -i' and then go back to my kernel and mount the drives with the 'journal=<inode>' option.

-Mike

In reply to:

---

It seems like either I'm missing a step

---

Actually I just reread my original post and I missed that step. I didn't specify when you should flash the kernel. Like I said in that other post you need to flash it after creating the journal files but before mounting as ext3. Sorry for the confusion.

The next time I release patches I will modify the mountHack patch so that you can mount the drives as ext2 on an alternate mount point with my kernel if necessary (I'll check the mount point instead of the dev name to determine if the type needs to be changed to ext3).

-Mike

Sorry. I assumed that you'd already done it once and were having trouble getting it back after the upgrade to b13