Since the temperatures dropped in fall I started getting random sigkill errors on boot. I have to reinsert the empeg several times until it boots fully. I thought it was a bad solder joint at the HDD connector and finally took it apart just 20 minutes ago and re-soldered the connector (i.e. liquified all solder points). Plugged it in and updated Hijack to v444. Pressed "reboot" just for the hell of it and there it was again, the dreaded sigkill eror. I have two HDDs, one fairly new, less than two years (one of the original ones had died). Please see the attached pics and video (made with my treo - requires Real player). Note the grabled graphics (but the connector is fine and once it boots, everything is ok).
Any ideas?

Have you installed any hardware in the box?

No, there's no additional hardware (except additional RAM, but that was installed long before the errors came).
I forgot to mention that sometimes the player settings get lost when the error occurs.
It also seems to be somewhat temperature-dependent in that it appears to be less likely to occur when I bring the empeg to the car from inside, compared to when it's been locked in the cold trunk. But it's a fairly weak correlation.
It also seems to happen less often if I start the car and the empeg has been running before, e.g. when it's been sitting in the garage with the empeg in its sled as opposed to when I put the empeg in the sled from outside. But that's also not 100%.

I'd still check the RAM installation.

Ok, I hooked the empeg up to record the bootlog. It seems to only start when 16M of RAM are detected, so you were right on. This is a bootlog when it booted normally:

Code:

---

CCC¡V

     •�µ�…É�‰½½ÑÍÑÉ…Á�ÙŹÁÉ�ÉÁÁÁÅÅÁÙ�¡¡Õ�½•µÁ•�¹�½µ¥5)ÿIf there is anyone present who wants to upgrade the flash, let th

em speak now,

or forever hold their peace...it seems not. Let fly the Penguins of Linux!



e000 v1.04

Copying kernel...

Calling linux kernel...

Uncompressing Linux..................................... done, booting the kernel.

Linux version 2.2.17-rmk5-np17-empeg52-hijack-v444 (hijack@rtr.ca) (gcc version 2.95.3 20010315 (release)) #2 Fri Dec 2

14:42:18 EST 2005

Processor: Intel StrongARM-1100 revision 11

Checking for extra DRAM:

c10000f4: wrote 00ff00ff, read 00ff00ff

NetWinder Floating Point Emulator V0.94.1 (c) 1998 Corel Computer Corp.

empeg-car player (hardware revision 9, serial number 10101626) 16MB DRAM

Command line: mem=16m

Calibrating delay loop... 207.67 BogoMIPS

Memory: 15008k/16M available (984k code, 20k reserved, 368k data, 4k init)

Dentry hash table entries: 2048 (order 2, 16k)

Buffer cache hash table entries: 16384 (order 4, 64k)

Page cache hash table entries: 4096 (order 2, 16k)

POSIX conformance testing by UNIFIX

Linux NET4.0 for Linux 2.2

Based upon Swansea University Computer Society NET3.039

NET4: Linux TCP/IP 1.0 for NET4.0

IP Protocols: ICMP, UDP, TCP

TCP: Hash tables configured (ehash 16384 bhash 16384)

IrDA (tm) Protocols for Linux-2.2 (Dag Brattli)

Starting kswapd v 1.5

SA1100 serial driver version 4.27 with no serial options enabled

ttyS00 at 0xf8010000 (irq = 15) is a SA1100 UART

ttyS01 at 0xf8050000 (irq = 17) is a SA1100 UART

ttyS02 at 0xf8030000 (irq = 16) is a SA1100 UART

Signature is 636f6972 'rioc'

Tuner: loopback=0, ID=-1

Scheduling custom logo.

empeg display initialised.

empeg dsp audio initialised

empeg dsp mixer initialised

empeg dsp initialised

empeg audio-in initialised, CS4231A revision a0

empeg remote control/panel button initialised.

empeg usb initialised, PDIUSBD12 id 1012

empeg state support initialised 0089/88c1 (save to d0005480).

empeg RDS driver initialised

empeg power-pic driver initialised (first boot)

RAM disk driver initialized:  16 RAM disks of 4096K size

empeg single channel IDE

Probing primary interface...

hda: IC25N040ATMR04-0, ATA DISK drive

hdb: IC25N080ATMR04-0, ATA DISK drive

ide0 at 0x000-0x007,0x038 on irq 6

hda: IC25N040ATMR04-0, 38154MB w/1740kB Cache, CHS=4864/255/63

hdb: IC25N080ATMR04-0, 76319MB w/7884kB Cache, CHS=9729/255/63

empeg-flash driver initialized

smc chip id/revision 0x3349

smc9194.c:v0.12 03/06/96 by Erik Stahlman (erik@vt.edu)



SMC9194: SMC91C94(r:9) at 0x4008000 IRQ:7 INTF:TP MEM:6144b MAC 00:02:d7:22:06:5a

Partition check:

 hda: hda1 < hda5 hda6 > hda2 hda3 hda4

 hdb: hdb1 < hdb5 hdb6 > hdb2 hdb3 hdb4

RAMDISK: ext2 filesystem found at block 0

RAMDISK: Loading 320 blocks [1 disk] into ram disk... done.

EXT2-fs warning: checktime reached, running e2fsck is recommended

VFS: Mounted root (ext2 filesystem).

empeg-pump v0.03 (19980601)

Press Ctrl-A to enter pump...•Ô•j½Õ¹Ñ•‘�ɽ½Ñ�¡•áÑÉ�™¥±•ÍåÍÑ•µ¥�É•…‘½¹±å¹)5ÿchange_root: old root has d_count=1

Trying to unmount old root ... okay

Freeing unused kernel memory: 4k initempeg init 0.8

I see this is a developer image!

Mounting proc

Mounting first music partition

Tried to mount /dev/hda4 as reiserfs but got error 19

Mounting second music partition

Remounting first music partition read-only

Remounting second music partition read-only

Press 'q' now to go into development mode. You Have Zero Seconds To Comply...

Starting player

àlayer redirected to /proc/ttyH

U««K½¹•é�ѱ…¹Ñ¥�½M½ÕÑ¡}•½É�¥…)5ÿ*K+kkKJ¹Ñ•É�•ÁÑ¥¹���½¹™¥�¹¥¹¥)5ÿ¡øEmpire: Version 0.40 starting...

hijack: removed menu entry: "Hard Disk Detection"

khttpd: listening on port 80

kftpd: listening on port 21

IrDA: Registered device irda0

---

And this is the bootlog with the sigkill-error:

Code:

---

 empeg-car bootstrap v1.02 20001106 (hugo@empeg.com)

If there is anyone present who wants to upgrade the flash, let them speak now,

or forever hold their peace...it seems not. Let fly the Penguins of Linux!



e000 v1.04

Copying kernel...

Calling linux kernel...

Uncompressing Linux..................................... done, booting the kernel.

Linux version 2.2.17-rmk5-np17-empeg52-hijack-v444 (hijack@rtr.ca) (gcc version 2.95.3 20010315 (release)) #2 Fri Dec 2

14:42:18 EST 2005

Processor: Intel StrongARM-1100 revision 11

Checking for extra DRAM:

c1000000: passed.

c1100000: passed.

c1200000: passed.

c1300000: passed.

c1400000: passed.

c1500000: passed.

c1600000: passed.

c1700000: passed.

c1800000: passed.

c1900000: passed.

c1a00000: passed.

c1b00000: passed.

c1c00000: passed.

c1d00000: passed.

c1e00000: passed.

c1f00000: passed.

c2000000: wrote ffffffff, read 00000000

NetWinder Floating Point Emulator V0.94.1 (c) 1998 Corel Computer Corp.

empeg-car player (hardware revision 9, serial number 10101626) 32MB DRAM

Command line: mem=16m

Calibrating delay loop... 207.67 BogoMIPS

Memory: 31232k/32M available (984k code, 20k reserved, 528k data, 4k init)

Dentry hash table entries: 4096 (order 3, 32k)

Buffer cache hash table entries: 32768 (order 5, 128k)

Page cache hash table entries: 8192 (order 3, 32k)

POSIX conformance testing by UNIFIX

Linux NET4.0 for Linux 2.2

Based upon Swansea University Computer Society NET3.039

NET4: Linux TCP/IP 1.0 for NET4.0

IP Protocols: ICMP, UDP, TCP

TCP: Hash tables configured (ehash 32768 bhash 32768)

IrDA (tm) Protocols for Linux-2.2 (Dag Brattli)

Starting kswapd v 1.5

SA1100 serial driver version 4.27 with no serial options enabled

ttyS00 at 0xf8010000 (irq = 15) is a SA1100 UART

ttyS01 at 0xf8050000 (irq = 17) is a SA1100 UART

ttyS02 at 0xf8030000 (irq = 16) is a SA1100 UART

Signature is 636f6972 'rioc'

Tuner: loopback=0, ID=-1

Scheduling custom logo.

empeg display initialised.

kmem_alloc: Bad poison (name=size-65536)

empeg dsp audio initialised

empeg dsp mixer initialised

empeg dsp initialised

empeg audio-in initialised, CS4231A revision a0

empeg remote control/panel button initialised.

empeg usb initialised, PDIUSBD12 id 1012

empeg state support initialised 0089/88c1 (save to d0005480).

empeg RDS driver initialised

empeg power-pic driver initialised (first boot)

RAM disk driver initialized:  16 RAM disks of 4096K size

empeg single channel IDE

Probing primary interface...

hda: IC25N040ATMR04-0, ATA DISK drive

hdb: IC25N080ATMR04-0, ATA DISK drive

ide0 at 0x000-0x007,0x038 on irq 6

kmem_alloc: Bad poison (name=size-128)

kmem_alloc: Bad poison (name=size-128)

kmem_alloc: Bad poison (name=size-128)

kmem_alloc: Bad poison (name=size-128)

Unable to handle kernel paging request at virtual address 0000547c

memmap = C0004000, pgd = c0004000

*pgd = c0131801, *pmd = c0131801, *pte = 00000000, *ppte = 00000000

Internal error: Oops: 2

CPU: 0

pc : [<c002b078>]    lr : [<0000547c>]

sp : c0187e78  ip : c1fa5fc0  fp : c0187e94

r10: c011a394  r9 : c0126438  r8 : c01f9b80

r7 : 00000015  r6 : 00000000  r5 : 00008124  r4 : c0183140

r3 : 00000007  r2 : a5c32f2b  r1 : 00000015  r0 : 0000004b

Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  Segment kernel

Control: C000517D  Table: C000517D  DAC: 0000001D

Process swapper (pid: 1, stackpage=c0187000)

Stack:

c0187e60:                   0000547c c002b078  60000093 ffffffff c010a82c 00008124

c0187e80: 00000006 00000000 c0187ebc c0187e98  c0059524 c002b00c c00f60f0 c1fa5340

c0187ea0: c010a82c c1fa5340 c012d8f4 c01f9ae0  c0187ed8 c0187ec0 c00a5c14 c00594d8

c0187ec0: c012d948 c012d8f4 00000001 c0187efc  c0187edc c00a5d18 c00a5be0 c01f9b80

c0187ee0: c012d995 c012d840 c01f9ae0 00000000  c0187f20 c0187f00 c00a5dd0 c00a5cd0

c0187f00: c01f9ae0 00000000 c00f4ee8 ffffffff  c009ac3c c0187f34 c0187f24 c00a5e1c

c0187f20: c00a5d74 c011df80 c0187f44 c0187f38  c00a06e8 c00a5dfc c0187f58 c0187f48

c0187f40: c00a0bf4 c00a06dc 000000ff c0187f84  c0187f5c c009c784 c00a0be0 00000001

c0187f60: c0102768 c0123a30 c010a788 c01246e4  4401a11b c0008580 c0187f94 c0187f88

c0187f80: c00a0c18 c009c6bc c0187fb4 c0187f98  c0046d10 c00a0c10 00000001 c0102768

c0187fa0: c0123a30 c010a788 c0187fdc c0187fb8  c0009ae0 c0046d0c 00000000 c0009ba8

c0187fc0: c012d9ac c0102760 c0102764 c0102988  c0187ffc c0187fe0 c0009bb8 c00099c0

c0187fe0: c0009ba8 c012d9ac c0102760 c0102764  c0101fd8 c0188000 c000b6ec c0009bb4

Backtrace:

Function entered at [<c002b000>] from [<c0059524>]

 r7 = 00000000  r6 = 00000006  r5 = 00008124  r4 = C010A82C

Function entered at [<c00594cc>] from [<c00a5c14>]

 r7 = C01F9AE0  r6 = C012D8F4  r5 = C1FA5340  r4 = C010A82C

Function entered at [<c00a5bd4>] from [<c00a5d18>]

 r6 = 00000001  r5 = C012D8F4  r4 = C012D948

Function entered at [<c00a5cc4>] from [<c00a5dd0>]

 r8 = 00000000  r7 = C01F9AE0  r6 = C012D840  r5 = C012D995

 r4 = C01F9B80

Function entered at [<c00a5d68>] from [<c00a5e1c>]

 r8 = C009AC3C  r7 = FFFFFFFF  r6 = C00F4EE8  r5 = 00000000

 r4 = C01F9AE0

Function entered at [<c00a5df0>] from [<c00a06e8>]

 r4 = C011DF80

Function entered at [<c00a06d0>] from [<c00a0bf4>]

Function entered at [<c00a0bd4>] from [<c009c784>]

 r4 = 000000FF

Function entered at [<c009c6b0>] from [<c00a0c18>]

 r10 = C0008580  r9 = 4401A11B  r8 = C01246E4  r7 = C010A788

 r6 = C0123A30  r5 = C0102768  r4 = 00000001

Function entered at [<c00a0c04>] from [<c0046d10>]

Function entered at [<c0046d00>] from [<c0009ae0>]

 r7 = C010A788  r6 = C0123A30  r5 = C0102768  r4 = 00000001

Function entered at [<c00099b4>] from [<c0009bb8>]

 r8 = C0102988  r7 = C0102764  r6 = C0102760  r5 = C012D9AC

 r4 = C0009BA8

Function entered at [<c0009ba8>] from [<c000b6ec>]

 r7 = C0102764  r6 = C0102760  r5 = C012D9AC  r4 = C0009BA8

Function entered at [<c00096c4>] from [<c00098b0>]

Function entered at [<c00096dc>] from [<c0008080>]

 r7 = C0123F00  r6 = C0123E54  r5 = C012E3E0  r4 = C012E3E0

Code: e2833001 e58c300c (e59e2000) e3520000 e58c2000

---

Do these bootlogs tell me where exactly I need to look?

This looks like a faulty block of memory.

Yesterday, I re-soldered the main lead that goes from the memory to the strongarm chip and it recognized the RAM just fine. I put it into the car and it worked like a charm first try.
Then I parked the car, put the empeg into the trunk and two hours later the problem was back!
Any idea which of the four chips (I piggy-backed two extra RAM chips) it could be? Any suggestion as to how to fix the problem? Has anybody else with the piggy-back method had any problems?

Here's a boot-log of a successful 32MB boot sequence:

Code:

---

 empeg-car bootstrap v1.02 20001106 (hugo@empeg.com)
If there is anyone present who wants to upgrade the flash, let them speak now,
or forever hold their peace...it seems not. Let fly the Penguins of Linux!

e000 v1.04
Copying kernel...
Calling linux kernel...
Uncompressing Linux..................................... done, booting the kernel.
Linux version 2.2.17-rmk5-np17-empeg52-hijack-v444 (hijack@rtr.ca) (gcc version 2.95.3 20010315 (release)) #2 Fri Dec 2
14:42:18 EST 2005
Processor: Intel StrongARM-1100 revision 11
Checking for extra DRAM:
c1000000: passed.
c1100000: passed.
c1200000: passed.
c1300000: passed.
c1400000: passed.
c1500000: passed.
c1600000: passed.
c1700000: passed.
c1800000: passed.
c1900000: passed.
c1a00000: passed.
c1b00000: passed.
c1c00000: passed.
c1d00000: passed.
c1e00000: passed.
c1f00000: passed.
c2000000: wrote ffffffff, read 00000000
NetWinder Floating Point Emulator V0.94.1 (c) 1998 Corel Computer Corp.
empeg-car player (hardware revision 9, serial number 10101626) 32MB DRAM
Command line: mem=16m
Calibrating delay loop... 207.67 BogoMIPS
Memory: 31232k/32M available (984k code, 20k reserved, 528k data, 4k init)
Dentry hash table entries: 4096 (order 3, 32k)
Buffer cache hash table entries: 32768 (order 5, 128k)
Page cache hash table entries: 8192 (order 3, 32k)
POSIX conformance testing by UNIFIX
Linux NET4.0 for Linux 2.2
Based upon Swansea University Computer Society NET3.039
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
TCP: Hash tables configured (ehash 32768 bhash 32768)
IrDA (tm) Protocols for Linux-2.2 (Dag Brattli)
Starting kswapd v 1.5
SA1100 serial driver version 4.27 with no serial options enabled
ttyS00 at 0xf8010000 (irq = 15) is a SA1100 UART
ttyS01 at 0xf8050000 (irq = 17) is a SA1100 UART
ttyS02 at 0xf8030000 (irq = 16) is a SA1100 UART
Signature is 636f6972 'rioc'
Tuner: loopback=0, ID=-1
Scheduling custom logo.
empeg display initialised.
empeg dsp audio initialised
empeg dsp mixer initialised
empeg dsp initialised
empeg audio-in initialised, CS4231A revision a0
empeg remote control/panel button initialised.
empeg usb initialised, PDIUSBD12 id 1012
empeg state support initialised 0089/88c1 (save to d0005b80).
empeg RDS driver initialised
empeg power-pic driver initialised
RAM disk driver initialized:  16 RAM disks of 4096K size
empeg single channel IDE
Probing primary interface...
hda: IC25N040ATMR04-0, ATA DISK drive
hdb: IC25N080ATMR04-0, ATA DISK drive
ide0 at 0x000-0x007,0x038 on irq 6
hda: IC25N040ATMR04-0, 38154MB w/1740kB Cache, CHS=4864/255/63
hdb: IC25N080ATMR04-0, 76319MB w/7884kB Cache, CHS=9729/255/63
empeg-flash driver initialized
smc chip id/revision 0x3349
smc9194.c:v0.12 03/06/96 by Erik Stahlman (erik@vt.edu)

SMC9194: SMC91C94(r:9) at 0x4008000 IRQ:7 INTF:TP MEM:6144b MAC 00:02:d7:22:06:5a
Partition check:
 hda: hda1 < hda5 hda6 > hda2 hda3 hda4
 hdb: hdb1 < hdb5 hdb6 > hdb2 hdb3 hdb4
RAMDISK: ext2 filesystem found at block 0
RAMDISK: Loading 320 blocks [1 disk] into ram disk... done.
EXT2-fs warning: checktime reached, running e2fsck is recommended
VFS: Mounted root (ext2 filesystem).
empeg-pump v0.03 (19980601)
Press Ctrl-A to enter pump...•Ô•j½Õ¹Ñ•‘�ɽ½Ñ�¡•áÑÉ�™¥±•ÍåÍÑ•µ¥�É•…‘½¹±å¹)5ÿchange_root: old root has d_count=1
Trying to unmount old root ... okay
Freeing unused kernel memory: 4k initempeg init 0.8
I see this is a developer image!
Mounting proc
Mounting first music partition
Tried to mount /dev/hda4 as reiserfs but got error 19
Mounting second music partition
Remounting first music partition read-only
Remounting second music partition read-only
Press 'q' now to go into development mode. You Have Zero Seconds To Comply...
Starting pla
player redirected to /proc/ttyH
U««K½¹•é�ÕɽÁ•½ •É±¥¹)5ÿ*K+kkKJ¹Ñ•É�•ÁÑ¥¹���½¹™¥�¹¥¹¥)5ÿ
Empire: Version 0.40 starting...
hijack: removed menu entry: "Hard Disk Detection"
khttpd: listening on port 80
kftpd: listening on port 21
IrDA: Registered device irda0 

---

Do the three logs I posted help me in any way to determine how to fix the problem?

The problem got worse and worse. Just last week I had to insert and pull out the empeg probably 30-40 times until it would boot.
In a desperate attempt this weekend, I pulled out the old soldering iron and re-heated and/or resoldered every single contact on the memory chips - and so far it's been booting like new!
So, just for the record/archive: the empeg worked fine for months after the piggy-back memory upgrade. Then random sigkill-errors appeared during boot. The cure: re-solder the memory pins. All of them.

I just hope the problem doesn't come back.

Thanks for all your help, Rob!