Hi. My players sitting kind of idle right now, and is only using 16GB out of the 50GB total that's installed. I have a couple of huge files on my computer that I'd like to get off of there, and temporarily store on the empeg since so much space is there.

Since one of the files is over 20gb, I've got to make sure all of the free space is on one drive so that it will fit. I thought it would be no problem to just go through drive1 doing commands like "mv /empeg/fids1/_00012/* /empeg/fids0/", and then re-fidsift it again when I was done to get all of the files in the right subdirectories in /empeg/fids0.

After doing a few directories worth, I wanted to see how many miscellaneous fids I had sitting in /empeg/fids0, so I did "ls /empeg/fids0". To my surprise, I saw there were none, just the directories. I remembered the changes to hijack WRT the fids subdirectories, and I thought "well, hijack must be transparently redirecting them to get them in the right subdirectory". But, I checked the subdirectories, and nothing new is in there. So, I made a directory called '_00aaa' in /empeg/fids1/ and /empeg/fids0, and made a file called 'aaa' in /empeg/fids1. Then, I ran "mv /empeg/fids1/* /empeg/fids0/", and the 'aaa' file is totally gone. It's not in /empeg/fids0, not in /empeg/fids0/_00aaa, not in /empeg/fids1, not in /empeg/fids1/_00aaa, not anywhere.

So, my questions:

1. Is there any chance that hijack is trying to automatically redirect the file moving and failing, thus eating the file somehow?

2. If it's not hijack, am I just doing something colossally stupid and not seeing it?

3. If it's not hijack and it's not me, then what could it be?

4. If I were to start all over again, is there any command that I could have done to get it done faster rather than that big sequence of mv commands?

The faq does give excellent instructions on how to move files safely.


1. Is there any chance that hijack is trying to automatically redirect the file moving and failing, thus eating the file somehow?

Nope - Hijack does not interfere with ls at all, so if ls doesn't show files in a certain directory then they just ain't there.

2. If it's not hijack, am I just doing something colossally stupid and not seeing it?

Possibly - check through the faq to ensure correct syntax for commands, and to make sure the drive is correctly mounted (mind you, if you can create dirs, then I guess it is rw)

3. If it's not hijack and it's not me, then what could it be?

B.E.M.'s?

4. If I were to start all over again, is there any command that I could have done to get it done faster rather than that big sequence of mv commands?

Very simple commands in the faq, and other related useful info here

Nope - Hijack does not interfere with ls at all, so if ls doesn't show files in a certain directory then they just ain't there

I know hijack has nothing to do with ls. That's not what I was talking about. What I was talking about was the fact that hijack intercepts file system access to /empeg/fids directories, as referred to here. Since that's what I was doing, and I know hijack is in the middle of that, that's why I'm suspicious that it might have somehow inadvertantly caused the disappearance.

It's not that hijack's somehow hiding the files. They're really gone, and I'm trying to figure out why.

Possibly - check through the faq to ensure correct syntax for commands, and to make sure the drive is correctly mounted (mind you, if you can create dirs, then I guess it is rw)

If I can somehow cause half my files to be obliterated, then I'm pretty sure it was rw.

B.E.M.'s?

Maybe it's late, but I'm not catching this. I'm totally guessing here: Big Evil Monsters?


By the way, that faq entry is just about ftping stuff back and forth. I was hoping for a good recursive shell command that would automatically go into each directory, and move the fids to the corresponding directory on the other drive. Something that will take into account the fact that mv won't move a directory onto another with the same name, but also makes mv move the stuff that's inside, also creating the directories on the target if they don't already exist.

Sure it's moot now, since half my files are gone, but just for my own edification I'd like to see how someone more familiar with Linux would do it.

You blew it. The correct command for low level moving of stuff is not this:

mv /empeg/fids1/_00012/* /empeg/fids0/

Instead, you should have done this:

mkdir -p /drive0/fids/_00012 && mv /drive1/fids/_00012/* /drive0/fids/_00012/ EDIT: corrected, ugh

The files inside fids subdirs do not have unique names, so moving them all to a common directory simply overwrites them with each other.

Hijack didn't do this; a human did.

-ml

That's the problem with computers. They do exactly what you tell them to.

Here's an even better (and much simpler) way to merge the fids directory from drive1 into that of drive0, regardless of fids subdirs or not:

cp -auvfx /drive1/fids/* /drive0/fids
## and then, optional step to nuke the old files:
rm -r /drive1/fids/*


Nice and simple. Just run fidsift.sh again aterwards.

Cheers

Mark, what about my example here:

So, I made a directory called '_00aaa' in /empeg/fids1/ and /empeg/fids0, and made a file called 'aaa' in /empeg/fids1. Then, I ran "mv /empeg/fids1/* /empeg/fids0/", and the 'aaa' file is totally gone. It's not in /empeg/fids0, not in /empeg/fids0/_00aaa, not in /empeg/fids1, not in /empeg/fids1/_00aaa, not anywhere.

In this example, there's not a collision with any other name at all, at least not where it's supposed to be going. Also, it's not like I only saw the most recent fids to be copied to the destination directory. I didn't see any.

Here's another example I just did:

empeg:/# mkdir /empeg/fids1/_90120

empeg:/# mkdir /empeg/fids0/_90120

empeg:/# cp /proc/empeg_notify /empeg/fids1/_90120/666

empeg:/# ls /empeg/fids1/_90120/

666

empeg:/# mv /empeg/fids1/_90120/* /empeg/fids0/

empeg:/# ls /empeg/fids1/_90120/

empeg:/# ls /empeg/fids1/       

_00000	_00005	_0000a	_0000f	_00014	_00019	_0001e	_00028	_00035

_00001	_00006	_0000b	_00010	_00015	_0001a	_00023	_00029	_90120

_00002	_00007	_0000c	_00011	_00016	_0001b	_00025	_0002a

_00003	_00008	_0000d	_00012	_00017	_0001c	_00026	_00033

_00004	_00009	_0000e	_00013	_00018	_0001d	_00027	_00034

empeg:/# ls /empeg/fids0/

_00000	_00006	_0000c	_00012	_00018	_0001e	_00024	_0002a	_00032

_00001	_00007	_0000d	_00013	_00019	_0001f	_00025	_0002b	_00033

_00002	_00008	_0000e	_00014	_0001a	_00020	_00026	_0002c	_00034

_00003	_00009	_0000f	_00015	_0001b	_00021	_00027	_0002d	_00035

_00004	_0000a	_00010	_00016	_0001c	_00022	_00028	_0002e	_90120

_00005	_0000b	_00011	_00017	_0001d	_00023	_00029	_0002f

empeg:/# ls /empeg/fids0/_90120/

empeg:/# 

So where did it go? I sincerely apologize if I'm still having a brain fart here, but shouldn't the '666' file be in /empeg/fids0/? If not why? Thanks in advance for everyone's patience, but it still looks fishy to me.



edit: fixed formatting

That's the problem with computers. They do exactly what you tell them to.

But in this case, was I not telling the computer to move some files from one directory to another? I wasn't using any extra options to mv that I didn't understand. I was saying "move from here to here", not "move from here to a printer on the campus of Eastern Montana Tech"

Okay, I tried the above example through telnet instead of through serial just to see if that would make any difference. It didn't. So, I did the following:

sh-2.03# mkdir /empeg/fids1/test

sh-2.03# mkdir /empeg/fids0/test

sh-2.03# cp /proc/empeg_notify /empeg/fids1/test/poo

sh-2.03# ls /empeg/fids1/test/

poo

sh-2.03# mv /empeg/fids1/test/* /empeg/fids0/

sh-2.03# ls /empeg/fids1/test/

sh-2.03# ls /empeg/fids1/

_00000  _00005  _0000a  _0000f  _00014  _00019  _0001e  _00028  _00035

_00001  _00006  _0000b  _00010  _00015  _0001a  _00023  _00029  _90120

_00002  _00007  _0000c  _00011  _00016  _0001b  _00025  _0002a  test

_00003  _00008  _0000d  _00012  _00017  _0001c  _00026  _00033

_00004  _00009  _0000e  _00013  _00018  _0001d  _00027  _00034

sh-2.03# ls /empeg/fids0/test/

sh-2.03# ls /empeg/fids0/     

_00000  _00006  _0000c  _00012  _00018  _0001e  _00024  _0002a  _00032  test

_00001  _00007  _0000d  _00013  _00019  _0001f  _00025  _0002b  _00033

_00002  _00008  _0000e  _00014  _0001a  _00020  _00026  _0002c  _00034

_00003  _00009  _0000f  _00015  _0001b  _00021  _00027  _0002d  _00035

_00004  _0000a  _00010  _00016  _0001c  _00022  _00028  _0002e  _90120

_00005  _0000b  _00011  _00017  _0001d  _00023  _00029  _0002f  poo

sh-2.03# 

As you can see from the last ls, that example worked just like I expected. But, the previous example where it looked like an actual fid didn't. This furthers my belief that it's somehow hijack that's getting in the way here.

If it is indeed hijack causing my grief, is it just the price we pay of having the automatic fid mangling, or is it something that should work but is currently broken?

Again, I apologize for the trouble. I know that there are better ways to do what I was trying to do. But, I still can't understand why what I was trying to do didn't work.

Did it end up in /drive0/fids/_00000 by any chance?

("ls -logt | head" to show the timestamp on the most recent files)

So where did it go? I sincerely apologize if I'm still having a brain fart here, but shouldn't the '666' file be in /empeg/fids0/? If not why?

What version of Hijack? There's a bug, or at least missing feature, in some versions. Those versions only support players with all new-style (in subdirectories with "_") fids, or all old-style fids. If you have a mixture, it will conceal the old-style ones. If you reinstall 2.0-final, or install a brand-new Hijack, you might find that your /empeg/fids0 directory contains 256 of your fids (000..FF0).

It won't contain any more than that, because you got the original copy commands wrong and copied the contents of each of the /empeg/fids1/_nnnnn directores, in turn, into the /empeg/fids0 directory itself -- not into /empeg/fids0/_nnnnn directories, as the commands Mark suggests would have done. Hijack didn't eat any files; you copied each directory's contents over the previous one's, and Hijack merely conceals from you the final set of files you copied.

If you're issuing an "mv" command you aren't sure about, you can use "mv -i" and it'll ask you for confirmation of overwriting an existing file at the destination. (Mind you, that wouldn't help if Hijack were also hiding the files from mv.)

Peter

It's v380, so that shouldn't be the problem

Did it end up in /drive0/fids/_00000 by any chance?

Ooh, good guess! That is indeed where it ended up. (head isn't on the empeg, though, apparently). My 666 and my aaa file are indeed in there, along with about a dozen other fids with yesterday timestamps. I'm surprised not to see more, considering I moved a ton of files yesterday. Is it possible that rather than overwriting files with duplicate names, mv just ignored files with duplicate names? Or is it possible that it did overwrite them but they kept the old timestamps?

Obviously my logic was flawed from the beginning, because if they did go into the /fids0 directory, there would have been tons of duplicate names and very few files. But, is there anything hijack could be doing better here, not to save retards like me, but to make the file system appear 'normal' for lack of a better word? From hijack's perspective, it probably only seeing mv say to the kernel "write a file called 'aaa' to /empeg/fids0" and not having any idea where it came from. So, hijack is then saying to itself "that's obviously fid 00000aaa, and since there are fid subdirectories present, i'll put that in _00000/aaa". Is it even worth it to think about making hijack only do this if the player's running, or if it's not coming from the shell, or any other hacky things like that?

I think you should be using /drive0/fids rather than /empeg/fids0 if you don't want to be bitten by Hijack's magic...

Well yeah, I know that now...

Is it even worth it to think about making hijack only do this if the player's running, or if it's not coming from the shell, or any other hacky things like that?

With all due respect, I'd personally put this in the "Doc, it hurts when I do this" category. The solution is "don't do that." Now that you know the nature of the problem, you won't do it again. It seems to me any attempts to enable/disable the fid-mangling logic based on who's looking or who's copying or what's running at the time would be hackish at best, and dangerous to normal player/synchronization operations at worst. Your situation of wanting to manually move FIDs around to make space to do file taxiing seems isolated, so I don't know if it's worth creating a special case for in a kernel that we all use.

I don't say this to torpedo your request, but, frankly, I know of a lot of other things on the Hijack wish list that I'd rather see, and have a much larger potential benefit.

You should be using the /drive0/fids and /drive1/fids paths to access those files. Only the player software and (j)Emplode should ever use /empeg/fids* symlinks.

Again, to do what you wanted CORRECTLY:

cp -auvfx /drive1/fids/* /drive0/fids/


Cheers

Tony,

I agree with your logic 100%. The only reason I would even argue the other way would be to deflect attention from my own stupidity from stumbling into this. But, one last thing before I go:

Mark,

I hadn't been paying attention to the empeg world for a while, and didn't fully understand what the /empeg/fids links were all about when I started monkeying with them.

Only the player software and (j)Emplode should ever use /empeg/fids* symlinks.

If the way to avoid landing in the middle of the fid mangling logic is to stay out of those links entirely, is there any benefit to having Hijack just hide /empeg/fids0 and /empeg/fids1 from the shell or other programs to protect people like me from themselves?



And to anyone else entering this thread late who hasn't gotten the message from previous posts: Hijack did not do anything wrong. Hijack did not eat my files, nor did it hide them from me. Hijack did not steal my dog, put sugar in my gas tank, or take my girlfriend to the senior prom. I was working on something late at night, and should have thought it through better. I was really colossally stupid. I should be forced to wear a tall pointy hat and sit in the corner. The hat would say "Doesn't think the process through before doing things that could make everything disappear (but at least has backups)". The only thing that Hijack did was make me not see the results I was expecting (which would have been poor results anyway). Beyond that, Hijack has done wonderful things for my player, and is in fact, the greatest thing ever to happen to me. I've lost 22 pounds on Hijack, and my hair is thicker and fuller than ever. Hijack got my wife and me together, and will be the godfather of our first child.

There. I hope that's clear.



Edit: cleared it up a little more

And stop slobbering on my shoes!!

LMAO.

-Zeke