As discussed in a recent thread here, I've proven that weighted shuffles using manual song ratings are possible with a little bit of hacking the dynamic data structure. The problem is, writing a song's numerical rating to disk conflicts with the player's efforts to write dynamic data to disk. In order to work around this, I'm asking for the following two features in Hijack:

To write the song's rating to the skips_count field...
1. A kernel call takes the FID and the numerical song rating.
2. It writes the song rating to the "skips_count" field at offset 0x14 within the dyndata structure.
3. It calculates the CRC-16 of the new dyndata structure with the modified skips_count, and stores the new CRC-16 at offset 0x2.

It would look something like this. Here's the data for FID 0x14 on my player before writing the song rating:



And here is what it would look like afterwards, with the new skips count and new CRC-16:



When the player writes dynamic data to hda3...
1. See if the offset is in the "per-FID dynamic data" range (which begins at 0x200000)
2. For example, let's say player is writing per-FID dynamic data for FID 0x14
FID data is at (0x200000 + 14*0x200) = 0x202800
3. Read the existing skips_count (the 4 bytes starting at 0x202814) from the disk
4. Merge this plays count into the buffer the player is writing, replacing the skips count
5. Calculate CRC-16 of the new buffer (length is an unsigned short in the first two bytes of the buffer)
6. Store the CRC-16 in the proper position in the buffer (offset 0x2)
7. Write the modified buffer to disk.

It would look something like this:

Before:

After:


So, Mark, you think this is doable? Any interest in giving this a go? I have the userland portion finished, FWIW.

pic #1

pic #2

pic #3

pic #4

Mmm.. nice research!

I can do this, but I need you to do some more up-front investigation first. See if you can use strace to determine how the player requests the write to the partition -- does it do a write("/dev/hdaX", ...), or does it use mmap(), or what.. ??

If strace doesn't work, you can try Hijack's trace_fs=1, but I don't think it currently catches random raw disk writes -- try it and see if there's any observable behaviour. If not, then add some additional instrumentation to the sys_write() routine in linux/fs/read_write.c

Cheers

Cool, glad you think this one's doable. I'll check out the strace stuff tonight after dinner.

I can do this, but I need you to do some more up-front investigation first. See if you can use strace to determine how the player requests the write to the partition -- does it do a write("/dev/hdaX", ...), or does it use mmap(), or what.. ??

We open() /dev/hda3 and then read() and write() it. I'm pretty sure we only do so on sector boundaries.

Peter

The single /dev/hda3 holds data for fids0 and fids1 ?

EDIT: Nevermind, of course it does!

Thanks

Hope so, 'cause the FAQ only tells you to back up that one partition if you wanna save your dynamic data.

Still need the strace-ing done, Mark?

Wouldn't hurt. I won't be getting to doing anything on this for a week or so.

Cheers

Got the strace output...

[pid    32] open("/dev/hda3", O_WRONLY) = 14

[pid    32] write(14, "<\0\321b\0\0\0\0\0\0\17\0!WN@K\275I\30\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 512) = 512

.
Will that do?

Cool.

Does it always open it immediately before the write, and then close again?

Cheers

Always opens immediately before write, yeah. I didn't have my strace looking for close() calls but one would expect those would be there as well.

Okay, good. And how about the lseek/llseek that must happen just in front of the write? Is there a read() there as well?

Thanks

Here's strace with open, lseek, write, and close:

[pid    32] open("/dev/hda3", O_WRONLY) = 56

[pid    32] lseek(56, 2097152, SEEK_SET) = 2097152

[pid    32] lseek(56, 0, SEEK_CUR)      = 2097152

[pid    32] lseek(56, 2128896, SEEK_SET) = 2128896

[pid    32] lseek(56, 0, SEEK_CUR)      = 2128896

[pid    32] lseek(56, 0, SEEK_CUR)      = 2128896

[pid    32] write(56, "<\0Y\3\0\0\0\0\0\0\16\0^\227O@\241\337\254\37", 512) = 512

[pid    32] close(56)                   = 0

.
Excessive nulls deleted from the write() this time around for brevity.

Go here?

Here?

Oops, no. Over here.

Here?

Are you sure?

Oh, yeah. That's the stuff.

Hehehehe... I was thinking pretty much the same thing.

To be fair, you weren't looking at reads, were you? Actually, might a read have repositioned the file pointer?

Argh, I can't remember if I was. I probably should have, Mark kinda asked for reads too.

I'll do that tonight.

On second look, I was strace-ing for reads, too. The command line I used was:

strace -p28 -p30 -p31 -p32 -p33 -p34 -p35 -e open,read,wrrite,lseek,close -s 1024

I just picked a few of the player processes until I found the one that does the hda3 writes. No reads involved, apparently, so those unnecessary lseeks do look kinda funny.

Just a little thread bump to see if this is still on the short-term wish list. I'm probably going to want to release in the next week or so and I'd like to know if this will be ready anytime soon.

Looks like a busy week here, so don't count on it.