Has anyone with a mk2 beta 12 had problems with tracks skipping a little? it won't do it reliably but it has happened a few times. it will be playing fine, then does a small jump (0.5 sec) ahead.

Murray 06000047
____________________

Try to figure out if it's always the same song and the same position of the song where it jumps.

TeeMcBee

No, I've had it happen to a few different songs, but if I rewind it's OK. it is a _very _ intermittent error. it just leaves you thinking "what was that?"
It doesn't happen every song, more like every 20 songs or more.

I haven't noticed it in the day that I've had 12a loaded.

Murray 06000047
____________________

Just had the same ...

Nils

Disclaimer: I'm no kernel expert, so what I say could be completely wrong.

But, it seems to me there's a potential race condition in the empeg_audio2.c driver. I'm not sure if the kernel will let multiple threads, or multiple processes enter audio_write at the same time. Equally, I'm not sure if the player would even try to do this. I do know that when I stuck some code in the middle of the dev->head twiddling stuff, I had a similar problem (but only at the end of tracks). When I moved all the dev->head twiddling stuff together, and put my stuff after it, the problem appeared to go away, or at least reduce in frequency.

So, my question is, is it possible for multiple player threads to call audio_write at the same time?

Or alternatively, have I completely misunderstood, and am spouting a load of nonsense?

Richard.

had it happen to a few different songs, but if I rewind it's OK

If I remember correctly I noticed this first in release 10A. When on a few longer trips it appeared that the error happened at the very same spot in a lengthy playlist same track; same place) if the playlist was left alone running from the very beginning. Playing with the sequence of the playlist prior to the track in error seemed to prevent (postpone??) the error. Also there as nothing wrong with the track itself: playing it by itslef, in another (shorter) playlist made it run smoothly.

I bet it is in the software. A buffer underrun; some clean-up action; or something similar.

Haven't had a chance to play with it since: sold Emma shortly thereafter and don't have sufficient tracks on Mark2 yet to play with the assumption again.


Henno
mk2 nr 6

There shouldn't be a problem - there's only one thread which does audio decoding, and so only one thread which calls write.

The head twiddling isn't protected as that is the only place the head pointer is twiddled, and with only one writer should be safe (famous last words!)

Hugo

Murray 06000047

ClemsonJeep 060000046 ;)

I've been noticing this in the last few days. On my Mk1 (beta 12) it doesn't seem to skip ahead as much as pause momentarily. The duration of the pause seems to vary, anything up to 0.5s.

It doesn't seem to follow any pattern and repeating the track won't replicate it, so I assume it isn't a problem with the encoding. (Just in case, all my .mp3s are encoded using VBR, most with AC2.1 but I have been trying LAME. The skip seems to appear in tracks created using both encoders.)

I have noticed it in the Mk2 as well, but since it's not in the car yet, I'm still listening to the Mk1 for longer periods.

Geoff
---- -------
Got one of the first Mark 2 empegs...

Yeah, I've been seeing the same thing for the last several versions, up to and including beta 12b. A brief pause sometimes during playback, rare, intermittent, and not easily reproducible.

I'd attribute it to threading problems in the custom kernel I'm running, but it's also been reported by many folks who I know aren't running custom kernels.

Guys@empeg... have you seen this happen in the laboratory, and do you have any idea as to its cause?

To others on the BBS, here's an idea: turn on your disk-indicator icons (an option in the latest Emplode). If this bug happens to you, look and see if the disk indicator is up when it pauses like that. Report what you find. I had my disk indicator activated the last time it happened to me, but I didn't think to look at it.

___________
Tony Fabris

This is a random problem for me too - using Mark 2. It is definitely pausing and not skipping, and it seems to affect the first song in a playlist.

and it seems to affect the first song in a playlist.

For the record, I've never had it happen on the first song in a playlist. It usually doesn't happen to me unless the unit's been playing a large playlist for a long time.

___________
Tony Fabris

It usually doesn't happen to me unless the unit's been playing a large playlist for a long time.

Agree. I had the same experience back when my Mk1 was still running 10A. Back then I kept the same playlist on a few long drives and when leaving playback alone, the interupts happened after some 50 minutes. Also, replaying the very same track by itself was flawless.

Haven't had it with Mk2, until today, but it may be unrelated. Somehow I had changed visuals and was trying to find back Funnelweb, while driving and missed it a few times. While flicking through the visuals, playback was interupted a few times for tiny fractions of a second, each time on the exact change of a visual.

Funny enough I've not been able to purposely reproduce this behaviour: flicking through the visuals thereafter had no effect ?!?!
(though it was during the same ride). I'll keep watch

Henno
mk2 nr 6

I just had it happen on 12c. Same thing as always: a non-reproducible pause in the middle of a song. The hard disk icon was up at the time. It was near the end of a song, and the hard disk had been reading the beginning of the next song into the cache.

If this helps, the little "light" on the hard disk had just gone from white to gray when it happened.

Another thing... This might be my imagination, since I can't reproduce it. But here goes:

When I checked to make sure that it wasn't the MP3 file's problem by rewinding past the skip, it made a funny noise during the rewind. The song was a quiet piece with clean instruments, but during the rewind, there was a brief burst of screeching static, as if the player was trying to send a block of garbage data to the output DAC.

I think (I'm not certain), that the static bit corresponded to the skipped spot in the song. But when I stopped rewinding, the skip was gone and I couldn't reproduce either the skip or the static by rewinding again.

Just in case that helps...

___________
Tony Fabris

Whoa! Now HERE's a strange thing... It may be related, it may not, but here goes...

I was just getting ready to play with the append/shuffle thing, and I turned shuffle off and on again, then cycled through the info modes on my way to "now and next". Guess what I saw?

The wrong track name being displayed, that's what I saw.

It was playing "Deeper and Deeper" by The Fixx, but it was displaying Nine Inch Nails' "The Frail" as its current track. The "Info:Track" and "Info:Line" screens had "The Frail", but the "Now and Next" screen was correct.

Now get this. You're gonna love this one.

The track I'd encountered the skip on, just a few minutes before? Guess which one it was? "The Frail" by Nine Inch Nails. I could even hit << to get back to it (it was about three songs back in the list).

Hmmm!

___________
Tony Fabris

Hmm. Now I just got the "skip" bug when it was shuffling the whole player and it was on the song "Ray of light". I rewound it a bit to make sure the track was OK, then I left it alone. When it hit the end of the song, it said "End of playlist" and sat there. I couldn't FF or REW, or change tracks, or anything. Switching the info modes revealed a completely blank playlist with no songs in it at all, and the pause indicator on. Selecting a playlist again from the menu worked, of course.

And no, I hadn't done a search and selected only that song. It had been playing a shuffled playlist of my whole player (over 1000 songs).

I wonder if that was related to the skip, too?

Oh, and while we're at it, all during the first song of the new playlist after this happened, the "Now and Next" screen thought the player was paused (showed the pause indicator), when it was actually playing Kate Bush's "Running up that hill" with no problem (a little joke there, if you know the song lyrics). When it went to the next song in the playlist, the pause indicator disappeared.

___________
Tony Fabris

OK, I've spent today investigating these reports and I've concluded the following:

1. If the disk is spun down and you skip track to a relatively high bit rate file (the "relatively" bit depends on how many disks you have and how quickly they spin up) there is a chance that the player won't have cached enough of the next track to avoid a wait when it runs out of data before it can get it from the disk. This won't affect the majority of players since the disk spins up quickly enough. It is exacerbated by files with large ID3v2 tags that were downloaded to the player by a version of emplode prior to beta12c since the ID3v2 data is also cached.

2. When caching occurs small gaps occur very rarely in the music. I believe this is due to contention on some of the locks used by both the audio and the cache. I believe that this is slightly more likely to cause breaks in playback when selecting a new playlist due to the audio buffers not already being full.

I have addressed both issues and inserted some debugging code. If anyone feels that they are in a position to try and reproduce this problem over the weekend I'd like to send out a version containing both the logging and the fixes in the hope that this will provide more information.

If you're interested then please email me (mac@empeg.com) directly.

--
Mike Crowe
I may not be speaking on behalf of empeg above :-)

I'm in the dubiously enviable position of having a mk1 & mk2 (someone make me an offer)
I played exactly the same track on both. it is identical because it came from the same process. Both players have B12c installed.

I play them back simultaneously, one in the Left one in the Right speaker.
it seems that when the Mk2 does a disk operation, sometimes it interrupts the thread of the audio. this results in a stutter and they lose sync playback.

I am yet to have it happen on the Mk1.

I am doing more testing and will let you know.


Murray 06000047
____________________

Has anyone had this happen on a release image, or does it only affect developer images?

Developer images currently contain a debugging build of the player, which has extra overhead (checking) on some operations -- possibly enough to cause a skip.

Peter

I currently have beta12b consumer installed on my Mk1, and have noticed the problem intermittently. I'm hoping to get 12c installed this evening when I get home from work.

I install consumer releases on the Mk1 & developer on the Mk2, but haven't had time to play with the Mk2 as much as I would like yet.

Geoff
---- -------
Got one of the first Mark 2 empegs...

Has anyone had this happen on a release image, or does it only affect developer images?

All of my reports have been based on the consumer release, not the developer release. Good question, though, thanks.

___________
Tony Fabris

OK, we've implemented various things which should address this problem and are looking for guinea pigs. We have a test version that contains some on-screen debugging information that should be useful should gaps occur during playback.

If you wish to run this version then download the appropriate image as described below. If skipping does not occur after prolonged use then please report this. If it does then try to quote the information displayed in the top right of the screen including the state of the caching indicator (which is forced on in this release).

Mark 1 http://www.empeg.com/downloads/car1-consumer-beta12d-gaptest1.upgrade
Mark 2 http://www.empeg.com/downloads/car2-consumer-beta12d-gaptest1.upgrade

TIA

--
Mike Crowe
I may not be speaking on behalf of empeg above :-)

Thanks, Mike, I just applied it to my Mark1. The skips seem to have been happening more often with 12c, so if it's there, it'll probably happen to me today as I have it playing at my desk.

In fact, I think I just got one (not completely sure- will keep checking to see if it happens again) and it had the disk icon up as well as A:14 N:232 X:232. What do the numbers mean?

___________
Tony Fabris

Hmm, this is very interesting. This confirms that what I suspected is causing the problem.

The numbers mean Average, Notable (i.e. quite a bit larger than expected), maXimum time in milliseconds that the audio_write kernel call is taking. 232 is bad. :-(

The maximum is never reset so if it happens you might like to reboot in order to do so.

--
Mike Crowe
I may not be speaking on behalf of empeg above :-)

The numbers mean Average, Notable (i.e. quite a bit larger than expected), maXimum time in milliseconds that the audio_write kernel call is taking. 232 is bad.

Yeah, usually the numbers are hovering at a:14 n:0 x:[no greater than 40 or so]. Then the one time that I saw it skip (only that first time so far), the numbers shot up at the same time the disk-read icon appeared. I have no idea what came first: The icon or the skip, because I wasn't looking at it until about half a second after the skip was done.

I was unsure about whether the skip was an actual skip because it was so brief. They are often longer. But if the number is in milliseconds, then that sounds about right. The skip was about 2/10 of a second.

The maximum is never reset so if it happens you might like to reboot in order to do so.

I noticed that right away, and did so, thanks. It hasn't happened since the first time and I'll keep half an eye on it as it plays at my desk.

___________
Tony Fabris

Oooo. 308 this time! Do I hear 400? Going once... Going twice...

(Mike... I'm going to put 12c back on if it's alright with you. You don't need any more information from me with this particular test build, do you?)

___________
Tony Fabris

308 this time! Do I hear 400? Going once... Going twice...

Tony, you did add an extra disk, didn't you?
What type did you add? Is it fast enough / cached?

Henno
mk2 nr 6

The number is more related to the scheduling of the multithreaded player than the type of HDD - any HDD is more than fast enough to supply the data for even 320kbit MP3 playing.

It's something we're trying to get to the bottom of; now we seem to be on the right track hopefully we can squash the bug.

Hugo

(Mike... I'm going to put 12c back on if it's alright with you. You don't need any more information from me with this particular test build, do you?)

This has given us exactly the information we needed. Thanks.

However, I'm a little suspicious that there maybe some hardware reason you are getting such long delays. I've left a prototype Mark II unit running for around four hours this evening (and it was running for most of today too) and I had no audible gaps and a maximum value of 55 which is a bit higher than I'd like but liveable with. I'll leave it running overnight if I can face the display away from my trying to sleep eyes! :-)

Could you leave your unit (running beta12c is fine) connected to a serial port and if you get a skip see if anything is output please?

--
Mike Crowe
I may not be speaking on behalf of empeg above :-)

However, I'm a little suspicious that there maybe some hardware reason you are getting such long delays.

Yeah, hardware trouble is a possibility, to be sure. But if so, then I'm not the only one with hardware trouble. Others have reported the exact same problem.

The 200/300ms numbers I'm getting... are those in a routine that involves the hard disks? Is it possible that hard disk errors could cause that sort of a problem? Or is the measurement of the time in that routine independent of other operations?

For the record, when I've had the pauses happen here on my desk, the unit was running "cool", i.e., it wasn't warm to the touch when I put my hand on it (except for the front display area which is always a little warm due to the heater filaments). So thermal failure of the hard disks is unlikely.

I've left a prototype Mark II unit running for around four hours this evening...

Yeah, the intermittent bugs are a bitch, aren't they? Have you guys experienced any skips at all with any of your players, or are you going totally on our reports?

Could you leave your unit (running beta12c is fine) connected to a serial port and if you get a skip see if anything is output please?

I'll do my best. Can't do it today, tomorrow I'll re-install the developer 12d-debug and leave it running with HyperTerminal connected. Then if I see a big number in the debug box, I'll look and see if the serial port reported anything.

___________
Tony Fabris

Just tried 12d-debug tonight on the Mk1 and got A:14 N:211 X:211 in the 4th or 5th track. Just as I was typing the first couple of words of this message, it skipped again, A:14 N:345 X:345 in the 13th track into the playlist... I tried to turn around to see if the disk icon was up, but I couldn't be sure - if it was, it had gone by the time I turned round!

I'll try connecting it to the serial and leave it running overnight...

Haven't tried it with the Mk2 yet, hopefully tomorrow...

Geoff
---- -------
Got one of the first Mark 2 empegs...

left a prototype Mark II unit running for around four hours this evening and I had no audible gaps and a maximum value of 55

For what's this worth:
Left my mk2 running overnight, playing on a 2386 tracks shuffled playlist. No info display / no visuals running, just the values showing. After 8 hours and 101 tracks they showed A19, N51, X51.

Henno
mk2 nr 6

Could you leave your unit (running beta12c is fine) connected to a serial port and if you get a skip see if anything is output please?

I've only had one "Notable" so far today, and all that the serial port said is "audio_write call took 142ms" (I assume part of the same debugging code that puts the numbers on the screen), but no other errors. Unfortunately, when it happened, I had the volume down so I couldn't hear whether it was an audible skip.

Interestingly, the Empeg screen says n:144 x:144, even though the serial port said 142.

___________
Tony Fabris

I put 12d on the Mk2 and started it off before heading to work this morning, and almost 12 hours later it reads A:15 N:55 X:55 so the problem seems to be mainly with the Mk1.

Geoff
---- -------
Got one of the first Mark 2 empegs...

Darn! Got a 347 today while sitting at a drive-thru window. Needless to say, I didn't have a computer with me to read what was coming off the serial port. Sigh...

___________
Tony Fabris

Hugo and I have spent the last two days getting our hands dirty on the workings of scheduling in the Linux kernel and think we may have found the problem. If you tried the beta12d-gaptest1 version then it would be very useful if you could reinstall that and then apply the appropriate one of the following upgrades. To keep the download size down it only applies a new kernel so applying it to beta12c may help but you won't get the useful statistics on the screen.

The same request applies - we're interested in the debug output from the top right of the screen.

Mark 1 http://www.empeg.com/downloads/car1-kernel-beta12d-gaptest2.upgrade
Mark 2 http://www.empeg.com/downloads/car2-kernel-beta12d-gaptest2.upgrade

TIA

--
Mike Crowe
I may not be speaking on behalf of empeg above :-)

I've only seen the "blank playlist" thing after booting (sometimes it takes 2 or 3 reboot attempts to get back to normal)

One of the few remaining Mk1 owners... #00015

The same request applies - we're interested in the debug output from the top right of the screen.

Sure thing. I'll even keep monitoring the serial output.

Question: Did you
a) fix the suspected bug and hope this test never generates any "N"s?
b) locate the suspected bug and put debugging code around it to get more information when we reproduce it?

By the way: Since this is a consumer release instead of a developer release, is it even useful for me to monitor the serial output? Would the consumer release put stuff out on the serial port if it encountered a hardware problem?

___________
Tony Fabris

We think we've fixed the bug (which wasn't at all empeg specific - we're waiting on other kernel gurus we've emailed to confirm this). It just depends whether the fix is good enough. We're hoping it should be good enough to avoid gaps. It may still give values for N though but they shouldn't be too high (say higher than 100).

Monitoring the serial port with a release build isn't worth it unless you or we suspect hardware problems. This version outputs absolutely nothing to the serial port about the bug.


--
Mike Crowe
I may not be speaking on behalf of empeg above :-)

I installed the gaptest2 update on my Mk1 (which was giving me the most problems) when I got in from work. It has been running for 1.5 hours so far, and the values on the screen are A:19 N:0 X:36 - a vast improvement over the same time with gaptest1 - I think I had X:345 within half an hour.

I'll try it in the Mk2 as well, but it only got X as high as 55, so I'm sure it'll be fine.

Nice piece of work Mike & Hugo, congratulations
(as soon as I hit 'Send' the Mk1 will probably skip, knowing my luck )

Geoff
---- -------
Got one of the first Mark 2 empegs...

Yeah, I've been running Gaptest2 for a few hours and haven't had a problem, no "n"s have happened yet. On the other hand, I ran it all day yesterday and only got two "n"s, so since it was an intermittent problem, we should continue to test this build for a little longer before declaring the bug swatted.

I have a question, though:

You gave us two separate upgrade files, one for the Mk1 and one for the Mk2. But this upgrade was kernel-only.

So I'm assuming, because of this, that the kernel is different between the Mk1 and the Mk2. If that's true I want to make sure to get that into my kernel-uploading FAQ. Is that true?

___________
Tony Fabris

Yes, the kernels are different for Mark I and Mark II. The source is identical but the configurations are different to accomodate the different USB chips and the networking available on the Mark II for example.

We could probably produce a kernel that worked on both but having different kernels keeps the download size down - especially for Mark I owners.

--
Mike Crowe
I may not be speaking on behalf of empeg above :-)

That's what I figured, thanks.

And I've had the gaptest2 file running nonstop all day, there were no "n"s, and "x" never went above 43. I'd say: Stick a fork in it, it's done.

___________
Tony Fabris

Yep, gaptest2 on my Mk1 has been running for about 6 hours with A:18 N:0 X:50, and for about 2 hours on my Mk2, giving A:15 N:0 X:44
I'm off to bed now, but I'll leave them running overnight and let you know what the values are again then - I'm not expecting them to be any different

Geoff
---- -------
Got one of the first Mark 2 empegs...

Very quickly (I'm running late as usual )

After running overnight...
Mk1: A:16 N:0 X:50
Mk2: A:15 N:0 X:48



Geoff
---- -------
Got one of the first Mark 2 empegs...