Mp3TOFID and Rsync Woes
Posted by: kendrick
Mp3TOFID and Rsync Woes - 12/02/2005 17:29
Does anyone have a good guide on how to use rsync and mp3tofid the one on the mp3tofid isn't very complete.
I followed the instructions there and i get this error when i try to run rsync on my empeg
+ ./rsync --times --delete --verbose --recursive --copy-unsafe-links --exclude=config.ini --exclude=lost+found 192.168.0.180::empegfids/drive0 /
receiving file list ... rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229)
Any ideas?
On my Music server i created the file /etc/rsyncd.conf
inside of that i put
[empegfids]
path = /mnt/empegfids
read only = yes
hosts allow = 192.168.0.178
use chroot = no
Then i ran rsync --daemon
I transferred the file rsync-2.5.5-vfdhack-1.tar.gz to my empeg
and then unzipped it onto my empeg.
On my empeg i go into the rsync-2.5.5-vfdhack-1 directory where the rsync file resides and i run the command
+ ./rsync -auv --delete --dry-run
rsync://192.168.0.180/empegfids .
receiving file list ...
And it just stops there.
Also when i run the command
./rsync --times --delete --verbose --recursive --copy-unsafe-links --exclude=config.ini --exclude=lost+found 192.168.0.180::empegfids/drive0 /
It does the same thing.
Any ideas?
Posted by: tfabris
Re: Mp3TOFID and Rsync Woes - 12/02/2005 17:49
Is that error copied-and-pasted, or did you re-type it by hand?
The IP address of the player is faulty in that error message, it's missing a period (I'm guessing it should be 1.80 instead of 180). Maybe that's the problem?
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 12/02/2005 17:58
Sup I did it again with teh correct ip 192.168.0.180
Same thing still happens.
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 13/02/2005 06:22
Hrmm Mp3TOFID and Rsync only seem to work for me if i first connect to the empeg via the hyperterminal with the serial cable connected.
After i do that, then i can connect through the microsoft telnet program without the serial cable connected and run the rsync.
Is that normal?
The only difference I can think of is that in Hypterminal i send the Q command to enter into developer mode.
Anyway you can enter into developer mode from the Microsoft Terminal Window?
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 13/02/2005 16:28
Any ideas?
No one uses rsync and MP3toFID here?
I guess my main question at this point is there anyway to go into Development mode from Microsoft Telnet Program because that seems to be the only way i can get Rsync and Mp3toFID to work.
Posted by: mlord
Re: Mp3TOFID and Rsync Woes - 13/02/2005 17:09
Perhaps it's simply the old networking bug. The empeg hardware/firmware have a problem with some networking setups, whereby they fail DHCP and fail to send packets out initially after each reboot. I usually just ping the player until it comes to life, and then everything else works until the next reboot. All of my various empeg scripts I use here now do that as a matter of course.
cheers
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 13/02/2005 22:46
Ok, if i ping the player before the rsync command then it works.
However the next problem im having is. I do my initial mp3tofid and that takes about 45 minutes. Then i added a song to the directory and did the mp3tofid command again and it still takes the same amount of time?
I thought mp3tofid was supposed to run in less than a minute or somethin?
Posted by: mlord
Re: Mp3TOFID and Rsync Woes - 14/02/2005 01:35
Dunno.. I've never used that s/w here. But the first thing to try would be to ensure that the date, time, and timezone on the player match that on the PC that's being used for the rsync. Shouldn't matter, but..
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 14/02/2005 10:35
Quote:
I thought mp3tofid was supposed to run in less than a minute or somethin?
Yes, it should only rescan what is changed since the last run.
Are you using Linux, some other *ix, or cygwin?
On what kind of filesystem are your tunes stored?
Pim
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 14/02/2005 10:42
Quote:
Shouldn't matter, but..
Yes, it helps, but it doesn't matter. rsync will handle that.
But kendricks problem is not rsync inefficiency, but mp3tofid
inefficiency. It shouldn't scan all tunes every run, it should
rely on previously generated fids, if the corresponding tune
has not changed.
There are a couple of checks like mtime, size, inode number
and major/minor numbers of the filesystem that contains the
tunes.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 14/02/2005 13:26
My Tunes are stored on NTFS file system on a windows machine.
The tunes are mounted to my linux box over the network.
Then I am using my linux box to perform the mp3tofid
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 14/02/2005 15:23
Quote:
My Tunes are stored on NTFS file system on a windows machine.
The tunes are mounted to my linux box over the network.
I'm afraid that's the problem. mp3tofid uses a fid number allocation
strategy based on the tune's inode number. The idea is that the inode
number of a tune does not change when you rename or move it,
or edit it. This is important, as you do not want the fid numbers of
your tunes to change, otherwise rsync will be copying the files
over and over.
Unfortunately, inode numbers are an alien concept in NTFS.
cygwin knows how to emulate inode numbers in a consistent,
persistent way, but smbfs does not.
You can check this for yourself. Do "ls -li mytune.mp3".
Unmount the filesystem, remount it and repeat the ls command.
You'll see different inode numbers.
Your best bet is to turn things round. Store your tunes on your
linux box and use samba to share them to your windows clients.
This is what I do.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 14/02/2005 15:35
Alright, i'll put my music on a linux filesystem
What rsync command do you use by the way, so that FIDS dont have to get deleted and replaced everytime?
Also is there anyway to make a script so that my empeg is being pinged while the rsync command is running thats the only way my box seems to do it without the serial cable hooked up?
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 14/02/2005 16:04
Quote:
What rsync command do you use by the way, so that FIDS dont have to get deleted and replaced everytime?
Moving your tunes to the linux box will cause yet another full rsync transfer,
unless you're in for some hacking. But after that, fid numbers will remain
persistent and rsync will be efficient.
For full transfers, it will be much faster to just delete the current fids
on the player, as having rsync compare them all will slow things down
seriously.
Quote:
Also is there anyway to make a script so that my empeg is being pinged while the rsync command is running thats the only way my box seems to do it without the serial cable hooked up?
I use telnetd without having to ping. I do use "ifconfig lo up" in my rsync script
on the player. Maybe this helps too.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 14/02/2005 17:39
rsync \
$@ \
--times \
--delete \
--verbose \
--recursive \
--copy-unsafe-links \
--exclude=config.ini \
--exclude=lost+found \
${rsyncserver}::${modulename}/drive0 /
Hey, PIM this is the rsync command that I am currnetly using?
Is this similar to the one you use? If not could you post or attach the script you use?
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 14/02/2005 20:03
I had to change my script to use one rsync run per drive. otherwise
my player would crash, even when using swap. Now that have had
a memory upgrade, it isn't necessary anymore, but I left it the way it
is. It's probably a tad faster as well.
Code:
#!/bin/sh
rsyncserver=192.168.101.1
modulename=empegfids
set -x
killall -INT player
# only remount if drives are mounted read-only
if grep -q "/dev/hd.4.* ro " /proc/mounts
then
remount=true
else
remount=false
fi
sleep 5
test $remount = true && sh -x rwm
ifconfig lo 127.0.0.1 up
for i in 0 1
do
rsync \
$@ \
--times \
--stats \
--delete \
--verbose \
--progress \
--recursive \
--empeg-progress \
--copy-unsafe-links \
--block-size=100000 \
--exclude=rsync \
${rsyncserver}::${modulename}/drive${i} /
done
test $remount = true && sh -x rom
killall is not included on the player. I got it from debian.
rsync itself is excluded in my script; that's because I sync
all my tools from empegfids::drive0/local/bin
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 16/02/2005 05:16
PIM
when i do ifconfig lo 127.0.0.1 up
It says ifconfig command not found.
Also when i try do do ping
i get command not found
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 16/02/2005 07:06
Right, my ping and ifconfig are also from debian, so it seems.
I'm having trouble finding the link to the debian tarball
I used, but I did find
this on Roger's website.
Hope this helps.
Pim
Posted by: Roger
Re: Mp3TOFID and Rsync Woes - 16/02/2005 07:08
You want
http://www.differentpla.net/~roger/empeg/car/files/base.tar.gzTransfer it to your /drive0 folder, then, in the root:
# tar xvfz drive0/base.tar.gz
You'll need to create /etc/resolv.conf if you want name resolution to work.
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 16/02/2005 07:10
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 16/02/2005 07:52
Cool, Thanks.
I'll include all of this in the instructions im trying to make also.
Hey PIM does that tarball you linked include the killall as well? All the extras needed to run your script?
Also I notieced your script has a {i} does that make it run the script for /drive0 then /drive1
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 16/02/2005 13:23
Hrmm, I haven't tried the ifconfig part of yoru script yet.
But for some odd reason when i run my script via Microsoft Telnet it runs extremely slow, to the point where it'll eventually just stop.
However if i run the script through Hyperterminal with the Serial Port connected it runs extremely fast.
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 16/02/2005 14:50
If you use hyperterminal, you know for sure the player process is stopped.
If you telnet, you don't. You need to make sure the player process is stopped.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 16/02/2005 17:27
Could I just install busybox to include all these commands that i need like ifconfig, killall, etc.
I have the busybox source, is there a special way you have to install it on the empeg or can I just do a regular ./configure, make , make install?
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 16/02/2005 21:23
Quote:
Hey PIM does that tarball you linked include the killall as well? All the extras needed to run your script?
yes
Quote:
Also I notieced your script has a {i} does that make it run the script for /drive0 then /drive1
Yes, ${i} is a shell variable that will take values 0 and 1.
Pim
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 16/02/2005 21:29
Quote:
Could I just install busybox to include all these commands that i need like ifconfig, killall, etc.
I suppose so, but it's a lot easier to just copy what you need from the tarballs.
Quote:
I have the busybox source, is there a special way you have to install it on the empeg or can I just do a regular ./configure, make , make install?
You cannot develop on the player itself. You need to set up a crosscompiler.
Download the arm linux toolchain from the hijack site and prepare some
time for getting it to work.
Some configure scripts make wrong decisions when running in a crosscompiler
environment. I noticed that when building rsync.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 16/02/2005 21:34
I see, you know where I can find a killall tarball?
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 16/02/2005 22:21
Right, killall isn't in either tarball mantioned in this thread.
I probably downloaded it from debian directly.
For your convenience,
here is my collection of arm tools.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 16/02/2005 22:34
Awesome, thats very nice of you I appreciate it.
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 16/02/2005 23:07
To use your set of tools, do i need to use teh ARM cross compiler or can i just extract the tar.gz that you attached to /bin or something?
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 16/02/2005 23:16
Common practice is to install non-standard binaries in /usr/local/bin.
This directory is in the PATH, so you can use the tools right away.
No need to set up a cross compiler.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 17/02/2005 07:38
Hrmm, Well I guess my next problem is. I saw you speaking on thsi earlier not sure if its the same thing though.
I have my Music stored on an ext3 filesystem on my linux box.
My windows machine accesses the Music via samba share .
I add music to the Linux box from windows via the windows samba share.
Would this be the reason that mp3tofid seems to do a full run everytime I run it? Would I have to do everything from the linux box
in order for mp3tofid to work correctly?
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 17/02/2005 11:59
Provided you did the following:
- move your music to the linux ext3 filesystem
- run mp3tofid
- rsync to your player
- add or change this through samba on a windows client
- run mp3tofid again
- run rsync again
then you should see mp3tofid only rescans changed and
new tunes, unless you force it using a commandline switch.
If mp3tofid did not rescan unchanged tunes on the second
run, then the second rsync should be fast too.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 23/02/2005 05:46
PIM thanks for all your help. i am currently putting together a guide to help those who were like me and dont know where to start.
I have rsync and mp3tofid working great thanks to you.
However my next question is. Is it possible to have a script on your local mp3 machine that runs mp3tofid and then sends a command that executes the rsync script on your empeg.
This way you wont have to run mp3tofid on your local mp3 machine. Then log into your empeg and run the rsync script.
It would be nice to have one script to run on your local mp3 machine.
I know that you would need rsh on the empeg and the local mp3 machine. Im not exactly sure where to get rsh for the empeg from.
Or what kind of script i would run. Do you have any experience setting this up or could give me assistance on doing so.
If thats not possible is it possible to run a cronjob on the empeg.
So i can have it periodically rsync.
Posted by: mtempsch
Re: Mp3TOFID and Rsync Woes - 23/02/2005 07:26
Quote:
know that you would need rsh on the empeg and the local mp3 machine. Im not exactly sure where to get rsh for the empeg from.
I believe rsh for arm is included in the Debian for Arm tar-ball Pim linked in this post
Posted by: pim
Re: Mp3TOFID and Rsync Woes - 23/02/2005 20:59
Yes, there's both rsh and in.rshd in this tarball.
A daemon might be more difficult set up though.
You might also consider putting your script in a hijack menu.
Pim
Posted by: kendrick
Re: Mp3TOFID and Rsync Woes - 25/02/2005 16:38
So how do I install the debian package without messing up my empeg install?
Posted by: pgrzelak
Re: Mp3TOFID and Rsync Woes - 25/02/2005 16:57
Crossposted. See reply in troubleshooting thread. Thanks.
Posted by: ajayrockrock
Re: Mp3TOFID and Rsync Woes - 06/03/2005 06:02
Quote:
Would this be the reason that mp3tofid seems to do a full run everytime I run it? Would I have to do everything from the linux box
in order for mp3tofid to work correctly?
I don't know if you found the answer to this already but when I was setting up mp3tofid I kept running it with the '-n' switch which is described as "force creation of new inode database". It was a dumb mistake but it's something that I overlooked for a while. Now without the -n switch, mp3tofid still takes a minute or so to run but it's much faster.
--Ajay