linux only hard drive upgrade?
Posted by: ajayrockrock
linux only hard drive upgrade? - 30/01/2009 05:23
Is there a way to use the drive builder image subsequently install the software using only linux? I don't have a windows machine at the house anymore.
I do have virtualbox and tried to setup the serial port but that's not working either. Then again, I'm trying to get *any* output from /dev/ttyS0 and it's not showing anything.
Before I go diving into my serial port issues, I was just wondering if it's even possible to upgrade the box using only linux.
Thanks,
Ajay
Posted by: LittleBlueThing
Re: linux only hard drive upgrade? - 30/01/2009 07:33
I'm sure it is - I have to admit to having a large disk here and just not getting around to upgrading my unit.
You've been here:
http://www.riocar.org/modules.php?op=modload&name=Drive_Upgrade&file=indexand found Mark's upgrader:
http://rtr.ca/bigdisk/yes?
You may want to add a page here:
http://empegwiki.org/index.php?title=Main_Pageand follow up to this thread - I may do the same if the muse takes me
Posted by: mlord
Re: linux only hard drive upgrade? - 30/01/2009 11:36
The problem with my upgrader.c, is that it requires functioning ethernet on the empeg. Normally, the player software configures the networking after boot.
But if there is no player software yet (eg. building new disks), then the empeg cannot communicate with upgrader.c.
I really should fix that someday, using kernel ip autoconfig.
-ml
Posted by: tfabris
Re: linux only hard drive upgrade? - 30/01/2009 15:24
Is there a way to use the drive builder image subsequently install the software using only linux? I don't have a windows machine at the house anymore.
Currently, the first install of a new piece of ".upgrade" software on a fresh drive on the Empeg truly does require Windows.
I do have virtualbox and tried to setup the serial port but that's not working either. Then again, I'm trying to get *any* output from /dev/ttyS0 and it's not showing anything.
I think others have successfully gotten the ugprader working through a virtual machine like that, so you're on the right track.
Your other option is to find a friend with Windows and a serial port.
Posted by: ajayrockrock
Re: linux only hard drive upgrade? - 30/01/2009 17:06
@LittleBlueThing, yeah, I've read the hard drive upgrade guide, it's just applying the .upgrade files that I have issue with.
@tfafris, I think I'll try to mess with my virtualbox setup a little more to get it to work.
I have a question, if I plug in my empeg to the serial port and do, 'cat /dev/ttyS0', then power up the empeg, I should see some output right? I'm trying to confirm if my serial cable is still good. There's only one serial port on my machine and I confirmed that it's been setup in the dmesg output plus 'setserial -g /dev/ttyS0'
Right now I'm not seeing anything, I expected to see some garbage or something (y'know, like when you do cat /dev/mouse and move the mouse around it prints all this crap to the screen).
Posted by: tfabris
Re: linux only hard drive upgrade? - 30/01/2009 18:41
I don't know if those are the correct Linux commands, but assuming they are, then yes, you should see something on the screen immediately if your serial port/cable is working properly and you boot the empeg. The empeg spends about 10 seconds spewing stuff to the serial port on every boot. Less if it doesn't have the player software installed yet, but you should see at least something on the serial cable even if the player doesn't have any disk drives attached or has just had the builder image installed.
Things I've seen that can cause the serial port to not work. Check each of these:
1. Serial port not correctly configured in the computer's BIOS.
2. Serial port not correctly configured in the computers operating system control panel.
3. A different software process is grabbing the serial port (such as PalmSync).
4. Bad/frayed cable.
5. Incorrect type of cable (see below).
6. COM port speed and settings on the computer are not set correctly to match the COM port speed and settings of the car player (see below).
Cable type and speed needed for the empeg:
1. If plugging into the empeg directly, you need a crossover serial cable, also called a NULL MODEM cable. This is a standard serial cable but with the TX and RX wires reversed. The computer's com port speed should be set to 115200, 8N1, No Flow Control.
2. If plugging into an empeg car docking sled, you need a "straight-through" serial cable (RX/TX are already swapped on the sled). Com port speed should be set to 4800, 8N1, No Flow Control. Also, if plugging into the sled, make sure the serial connector has a ground wire attached to the correct points; some older MK2 models were missing a ground wire on the sled serial connector.
Posted by: LittleBlueThing
Re: linux only hard drive upgrade? - 31/01/2009 08:36
instead of cat, install picocom and run:
picocom -b 115200 /dev/ttyS0
Mark - would it make sense to put something (udhcpd) in your bigdisk upgrader image to configure the network?
What is in the bigdisk images and can they be rebuilt from source? (and maybe bblobs)
Tony - won't 'download' work from linux (a tiny .c program that installs a kernel image via serial at the "penguins" bootloader prompt?)
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 11:01
Mark - would it make sense to put something (udhcpd) in your bigdisk upgrader image to configure the network?
Maybe. But that still doesn't solve the problem of how to get the bigdisk .upgrade loaded on to a blank empeg. A better solution would be to enable/use the kernel's *built-in* IP-autoconfig feature.
What is in the bigdisk images and can they be rebuilt from source? (and maybe bblobs)
I've got them quite automated here now. Hang on for a spell, and I'll post a tree with scripts.
Tony - won't 'download' work from linux (a tiny .c program that installs a kernel image via serial at the "penguins" bootloader prompt?)
That program is only for kernels and other flash blobs, not for .upgrade files. And it still fails on some, but not all, USB serial ports as of the Linux 2.6.2x series of kernels (on the host side) -- it used to work with older kernels.
Cheers
Posted by: peter
Re: linux only hard drive upgrade? - 31/01/2009 11:28
That program is only for kernels and other flash blobs, not for .upgrade files.
There was a Linux-based upgrader, Upgclient; it was released in the binary (i386) releases of Emptool. For some reason, most likely an oversight, source releases of Emptool didn't contain the source of it. And then the binary releases must have have gone missing when the support pages were rejigged.
However, Upgclient itself was only a thin wrapper around class Upgrader from inside the Emptool source distribution. Probably not much more, in fact, than (untested):
SerialConnection conn("/dev/ttyS0", 115200);
Upgrader upg;
STATUS rc = upg.CheckUpgrade("foo.upgrade");
if (SUCCEEDED(rc)) rc = upg.DoUpgrade(&conn);
if (FAILED(rc)) printf("Upgrade failed: %x\n", rc);
And it still fails on some, but not all, USB serial ports as of the Linux 2.6.2x series of kernels (on the host side) -- it used to work with older kernels.
Upgclient shared enough code with download.c that it too failed on most USB serial ports -- or, at least, on the one that we tried. In fact, I wasn't aware that there were any kernel versions on which USB serial upgrades worked more reliably than others... so far as we could tell, the problem wasn't in the kernel, it was to do with the regrettably low latency required by the player side of the upgrade protocol, latency which was exceeded by the USB overhead in some USB-to-serial implementations.
Peter
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 11:55
[quote]What is in the bigdisk images and can they be rebuilt from source?
My upgrader.c program can be used to break up .upgrade files into their component blobs, each of which can then be identified by the Linux "file" command for further breakdown.
I've updated my bigdisk builder page to now include snapshots of the build directories/tools I use to respin the .upgrade files.
Things are pretty simple. There is an "hda5" subdirectory, which contains everything you want to put into the root filesystem. There is also a "_proc_empeg_kernel" symlink, which should point at whatever kernel zImage file you want.
Then just run the included "./respin_upgrade.sh" to update things.
Cheers
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 11:57
There was a Linux-based upgrader, Upgclient; it was released in the binary (i386) releases of Emptool. For some reason, most likely an oversight, source releases of Emptool didn't contain the source of it. And then the binary releases must have have gone missing when the support pages were rejigged.
I have the upgclient binaries here, from v2beta12 and v2rc1. But they never really worked for me here, and I never bothered to try and understand why not.
Cheers
Posted by: peter
Re: linux only hard drive upgrade? - 31/01/2009 12:06
There was a Linux-based upgrader, Upgclient; it was released in the binary (i386) releases of Emptool. For some reason, most likely an oversight, source releases of Emptool didn't contain the source of it. And then the binary releases must have have gone missing when the support pages were rejigged.
Aha:
http://empeg.com/v3alpha/GPL-LGPL-compliance/empegcar-v3.00-alpha1.i386-linux.tar.gzYes, it's the release from v3 alpha 1, but none of that stuff had changed for ages. Unpack it and look at upgclient (or upgclient.static) and the README.
Peter
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 12:25
Mmm.. When I run that here, it just loops and says "ERROR: 5 empeg unit not found". Probably because I only have
one empeg unit connected rather than
5 of them?
With strace, I see it opening/closing the serial port lockfile
(and the .upgrade file, too) over and over and over and .. but NEVER opening the serial port itself. That might explain the failure.
-ml
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 12:30
Yup, if I had to guess what the non-existant source code is doing, I'd say it has a bug where it tries to open the .upgrade file as a serial port, rather than the actual serial port.
-ml
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 12:37
Mmm.. the .static version works a little better, trying to open the serial port and getting "EPROTO" back. Now that is curious..
Posted by: peter
Re: linux only hard drive upgrade? - 31/01/2009 12:39
Yup, if I had to guess what the non-existant source code is doing, I'd say it has a bug where it tries to open the .upgrade file as a serial port, rather than the actual serial port.
What's your command-line? I don't have an attached Empeg here, but it's certainly opening and ioctl'ing the correct device when I strace it.
Peter
Posted by: LittleBlueThing
Re: linux only hard drive upgrade? - 31/01/2009 12:39
Peter, I think you meant:
http://empeg.com/v3alpha/emptool-v3.00-alpha1.src.tar.gzThanks - that helps, I'll take a look.
I still use normal serial ports so it's not an issue yet (he says, remembering that the empeg port is now being used to monitor the satellite box... hmmm)
Posted by: peter
Re: linux only hard drive upgrade? - 31/01/2009 12:41
That's the source of Emptool but not Upgclient, though it does contain the bits you'd need to write a new Upgclient if the old one can't be made to work on modern kernels. The thing I linked to included binaries of both Emptool and Upgclient.
Peter
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 12:45
Ahh.. I get it. upgclient.static works only with the default "/dev/ttyS0", not when using the "-p" parameter to supply a different path. I can create symlinks in /dev/ to get around that, though.
Like download.c, it fails on the wired USB serial ports I have here. But it does seem to be doing *something* over the bluetooth AIR-cable serial port I have, which normally does work okay with (my modified version of) download.c
But it's very slow right now, whatever it's doing..
Posted by: peter
Re: linux only hard drive upgrade? - 31/01/2009 12:49
Ahh.. I get it. upgclient.static works only with the default "/dev/ttyS0", not when using the "-p" parameter to supply a different path. I can create symlinks in /dev/ to get around that, though.
Sorry, can't reproduce that either, though in a rather non-Unixy move it does
require the space between "-p" and "/dev/ttyS1".
Peter
Posted by: LittleBlueThing
Re: linux only hard drive upgrade? - 31/01/2009 12:51
truly does require Windows.
Think carefully before saying things like that Tony - you know what they're like
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 12:54
rm /var/lock/LCK..tty* ; ./upgclient.static -r car2_v3a11_hijack.upgrade
empeg-car Upgrade client.
empegupgrade: Using upgrade file 'car2_v3a11_hijack.upgrade' to device '/dev/ttyS0' (115200 baud)
Checking upgrade file integrity [100%]
Pumping player software
Finding unit [ 3%]
Upgrading flash (bootloader)
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...
0x8a414b5: 0d 0a 30 30.39 61 32 33|35 61 0d 0a.33 61 37 39 ..009a235a..3a79
0x8a414c5: 39 36 35 62.0d 0a 38 33|62 31 31 33.35 63 2d 62 965b..83b1135c-b
0x8a414d5: 62 37 63 38.64 62 38 2d|33 32 30 37.33 30 31 31 b7c8db8-32073011
0x8a414e5: 2d 64 35 36.64 36 34 63|37 0d 0a 30.30 0d 0a 30 -d56d64c7..00..0
0x8a414f5: 30 0d 0a 3f. 0..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
Upgrading flash (kernel)
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414bb: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
connection_all.cpp: 155 (18530): Connection::FlushReceiveBuffer throwing away...0x8a414b2: 0d 0a 3f ..?
Posted by: peter
Re: linux only hard drive upgrade? - 31/01/2009 12:58
Mmm.. When I run that here, it just loops and says "ERROR: 5 empeg unit not found". Probably because I only have one empeg unit connected rather than 5 of them?
FWIW, the idea was to make it easy for scripting languages to pick out error codes textually. (Someone had probably been reading
http://c2.com/cgi/wiki?AlternateHardAndSoftLayers .) The same reasoning was behind the odd verbosity of Emptool itself.
In practice I don't think anyone ever wrote any scripts that drove Emptool or Upgclient directly, which is fortunate as I also don't think we put any real effort into stability of that "API" between releases. It appears that anyone smart enough to script something like Emptool, is also smart enough to just link against lib/protocol directly -- or, indeed, rewrite the whole shooting-match in Java.
Peter
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 13:00
Ahh.. I get it. upgclient.static works only with the default "/dev/ttyS0", not when using the "-p" parameter to supply a different path. I can create symlinks in /dev/ to get around that, though.
Sorry, can't reproduce that either, though in a rather non-Unixy move it does
require the space between "-p" and "/dev/ttyS1".
I've fiddled some more, and the behaviour is not consistent. Sometimes it works with a "-p otherdevice" parameter, and sometimes it does not. But it never seems to work unless /dev/ttyS0 is a real device. But that could also be inconsistent over a longer trial period.
So.. uninitialized variables somewhere, I guess.
Cheers
Posted by: peter
Re: linux only hard drive upgrade? - 31/01/2009 13:21
No, you're quite right, that version of upgclient is hosed, even on real serial ports. I wonder how we managed that!
Fortuitously, it turns out I do still have an older version installed on a machine here. I've tested it (on /dev/ttyS1 no less) and it does work. It doesn't report a version number, but note that it should be 441,024 bytes long as opposed to the 453,828 bytes of the v3alpha1 release.
Peter
Posted by: LittleBlueThing
Re: linux only hard drive upgrade? - 31/01/2009 13:51
I have emptool src from v2.00-final here if that's of any use.
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 13:53
It doesn't report a version number, but note that it should be 441,024 bytes long...
That would be the same one that shipped with v2final, and possibly also with earlier versions.
Cheers
Posted by: LittleBlueThing
Re: linux only hard drive upgrade? - 31/01/2009 13:55
I have that 441,024 upgclient too - it has the same dl date as the 2.00-final src so I'm guessing it's the same version
Posted by: mlord
Re: linux only hard drive upgrade? - 31/01/2009 13:59
I have emptool src from v2.00-final here if that's of any use.
I've got it too. What you could try with it, is taking Peter's pseudocode wrapper (earlier in this thread) and trying to recreate upgclient from sources.
If you can achieve that, then I can fix the serial port stuff to work better over USB/bluetooth style ports.
EDIT: or maybe I'll just enable kernel IP-autoconfig (CONFIG_IP_PNP, CONFIG_IP_PNP_DHCP) in Hijack, so that my ethernet upgrader.c can always work with any unit that has a Hijack kernel in flash. Some other tweaks would also be necessary, I suppose, to let ftpd run without a rootfs or /procCheers
Posted by: tfabris
Re: linux only hard drive upgrade? - 31/01/2009 17:59
truly does require Windows.
Think carefully before saying things like that Tony - you know what they're like
What's awesome is that now you're all doing a full development effort to make that statement wrong. I love you guys.
Posted by: ajayrockrock
Re: linux only hard drive upgrade? - 06/02/2009 23:09
wow, I'm glad this thread is continuing! It took a few days but I finally got my replacement serial cable. And my "cat /dev/ttyS0" stuff did work. All it did was print out some garbage which was enough to tell me that it's active.
I couldn't get it to work via virtualbox so I grabbed the upgclient and yeah, that doesn't work either:
$ ./upgclient Windows_Share/builder_bigdisk_v4.upgrade
empeg-car Upgrade client.
empegupgrade: Using upgrade file 'Windows_Share/builder_bigdisk_v4.upgrade' to device '/dev/ttyS0' (115200 baud)
Checking upgrade file integrity [100%]
Pumping player software
Finding unit [ 2%]
Upgrading flash (bootloader)
Erasing flash [ 0%]
ERROR: 8 Flash erase failed
Upgrade aborted due to errors [ 0%]
Upgrade failed :-(
Upgrade failed with error 0x80040057
Any ideas?
Posted by: Attack
Re: linux only hard drive upgrade? - 06/02/2009 23:15
I think you have to plug in the Empeg power when it says "Finding Unit", but I could be wrong.
Posted by: ajayrockrock
Re: linux only hard drive upgrade? - 06/02/2009 23:17
just for the hell of it, I grabbed the v3alpha one that was posted and got similar results:
$ ./upgclient.static ../../Windows_Share/builder_bigdisk_v4.upgrade
empeg-car Upgrade client.
empegupgrade: Using upgrade file '../../Windows_Share/builder_bigdisk_v4.upgrade' to device '/dev/ttyS0' (115200 baud)
Checking upgrade file integrity [100%]
Pumping player software
Finding unit [ 1%]
Upgrading flash (bootloader)
connection_all.cpp: 155 (2000): Connection::FlushReceiveBuffer throwing away...0x84b843d: 0d 0a 30 30.39 61 32 33|30 34 0d 0a.33 61 37 61 ..009a2304..3a7a
0x84b844d: 38 33 65 39.0d 0a 34 35|31 38 61 62.36 30 2d 37 83e9..4518ab60-7
0x84b845d: 62 35 62 b5b
connection_all.cpp: 155 (2000): Connection::FlushReceiveBuffer throwing away...0x84b8439: 65 64 35 2d.33 64 64 37|61 66 30 38.2d 64 33 36 ed5-3dd7af08-d36
0x84b8449: 65 65 34 39.32 0d 0a 30|30 0d 0a 30.30 0d 0a 3f ee492..00..00..?
0x84b8459: 75 u
ERROR: 8 Flash erase failed
Upgrade aborted due to errors [ 0%]
Upgrade failed :-(
Upgrade failed with error 0x80040057
Same error code though... Do you guys know what 0x80040057 means? Google doesn't really give me any results except for this thread:
http://empegbbs.com/ubbthreads.php?ubb=showflat&Number=46154
Posted by: ajayrockrock
Re: linux only hard drive upgrade? - 07/02/2009 07:53
I think you have to plug in the Empeg power when it says "Finding Unit", but I could be wrong.
Yup, I waited until the "Finding Unit" prompt before I plugged in the power.
--Ajay
Posted by: mlord
Re: linux only hard drive upgrade? - 07/02/2009 12:57
This all means that you'll be needing a real hardwired serial port
with that software, rather than a USB serial adapter.
Posted by: ajayrockrock
Re: linux only hard drive upgrade? - 08/02/2009 08:07
Yeah, I have a real hardware serial adapter connected to my empeg. Any other ideas?
--Ajay
PS. I've had an empeg since 2002 and I'm so excited to be in a thread with mlord.
Posted by: altman
Re: linux only hard drive upgrade? - 10/02/2009 04:27
Hmm, I mean there's nothing particularly special about the protocol (timing-wise) - the only time critical bit is interrupting the bootloader at the right moment.
ISTR there is no problem driving the erase commands (etc) interactively... which points to a problem with the upgrader code which is making it susceptible to differences? Does anyone have the code unzipped so they can post the relevant section?
Hugo
Posted by: peter
Re: linux only hard drive upgrade? - 10/02/2009 10:32
Hmm, I mean there's nothing particularly special about the protocol (timing-wise) - the only time critical bit is interrupting the bootloader at the right moment.
ISTR there is no problem driving the erase commands (etc) interactively... which points to a problem with the upgrader code which is making it susceptible to differences? Does anyone have the code unzipped so they can post the relevant section?
The main difference in Mark's "USB-compatible" version seems to be that it buffers up the outgoing data and then sends it in one system call -- the stock download.c sends it a byte at a time.
The only theory I have as to how this could possibly make a difference, is if somehow the USB-to-serial adaptor can't saturate 115200bps transmit if it's only getting one byte per USB packet. (Perhaps, when driving it interactively, the terminal is line-buffered.) But actually I can't make the numbers for that add up, even for a USB-to-serial adaptor that's Low Speed 1.5Mbps.
Peter
Posted by: peter
Re: linux only hard drive upgrade? - 10/02/2009 10:54
Do you guys know what 0x80040057 means?
*Sigh* Looking at the Emptool sources, 0x80040057 is UPG_E_SOMETHINGBAD, the associated error message being "Something bad happened".
Upgrader::ErasePage() has failed for some reason, but the exact error code (which would have allowed us to determine whether you have a connection problem, or a player problem) is thrown away and replaced by meaningless generic ones
twice before being reported to the user.
Peter
Posted by: mlord
Re: linux only hard drive upgrade? - 10/02/2009 11:58
The main difference in Mark's "USB-compatible" version seems to be that it buffers up the outgoing data and then sends it in one system call -- the stock download.c sends it a byte at a time.
Yeah, without that buffering, downloads take *hours* rather than a minute or two. The other changes were to improve tolerance in delays on recept of ack's and the like from the empeg, though this obviously still
needs work -- that's where the remaining timing dependencies still lurk.
Cheers
Posted by: peter
Re: linux only hard drive upgrade? - 10/02/2009 12:16
Yeah, without that buffering, downloads take *hours* rather than a minute or two.
If that's the case, did your investigation ever reveal why? I'd love to know, however inefficient userland is, how come it's not saturating 115200bps over the serial link. On a modern PC, that's tens of thousands of CPU cycles per bit...
Peter
Posted by: mlord
Re: linux only hard drive upgrade? - 10/02/2009 14:28
It was years ago, and I didn't dig too deeply.
But my impression was, that sending one or two bytes at a time,
through a packetized transport (USB), and waiting for it to complete
before sending the next one or two bytes.. kinda made sense that
throughput would be slow.
I don't think the USB serial stack uses windowed transport
like TCP does.
EDIT: Mmm.. some memory is returning now..
I think it might have been the bluetooth
serial dongles here that were the slowest before those mods.
Cheers
Posted by: ajayrockrock
Re: linux only hard drive upgrade? - 12/02/2009 16:40
Update, I have since installed Windows on a spare box and upgraded my empeg (now have 300gigs across 2 drives).
I still would like a linux-only way to upgrade my empeg in the future. So if anyone has any tips on getting this to work I'd be more then willing to test anything.
--Ajay, shopping for a new empeg-friendly car...
Posted by: altman
Re: linux only hard drive upgrade? - 16/02/2009 04:40
Hmm, I mean there's nothing particularly special about the protocol (timing-wise) - the only time critical bit is interrupting the bootloader at the right moment.
ISTR there is no problem driving the erase commands (etc) interactively... which points to a problem with the upgrader code which is making it susceptible to differences? Does anyone have the code unzipped so they can post the relevant section?
The main difference in Mark's "USB-compatible" version seems to be that it buffers up the outgoing data and then sends it in one system call -- the stock download.c sends it a byte at a time.
The only theory I have as to how this could possibly make a difference, is if somehow the USB-to-serial adaptor can't saturate 115200bps transmit if it's only getting one byte per USB packet. (Perhaps, when driving it interactively, the terminal is line-buffered.) But actually I can't make the numbers for that add up, even for a USB-to-serial adaptor that's Low Speed 1.5Mbps.
It's definitely not a line buffered type thing; the bootloader will wait forever for the next byte (no timeouts). My guess is that my possibly overzealous flushing of input buffers in the upgrader so as I don't have to parse the echo from the bootloader is falling over. Nobody ever does flush the way I like it (which is: zero the head/tail pointer at that instant) - which tends to get replaced by "read with zero timeout until you see empty" which I can see getting seriously upset by USB.
With that in mind, can anyone nearer the code look? If not I will fire up gcc and compile it on my macbook, find a usb-serial adaptor, and see what happens
Hugo
Posted by: altman
Re: linux only hard drive upgrade? - 16/02/2009 04:42
But my impression was, that sending one or two bytes at a time,
through a packetized transport (USB), and waiting for it to complete
before sending the next one or two bytes.. kinda made sense that
throughput would be slow.
Depending on how janky the stack implementation is, the worst case could well be 1 byte per frame (1 millsecond). In reality, it should definitely get aggregated before that point, so even if the first byte takes a while millisecond to go out, the subsequent ones would at least fill a USB1 64-byte out packet.
Still, you can't argue with what actually happens in real life
Hugo