Linux system hiccups

Posted by: wfaulk

Linux system hiccups - 06/08/2010 14:57

I have a Linux (RHEL5) machine here at work that has started "hiccupping".

Basically, it works fine for about 28-29 seconds, then … pauses for about 4 seconds, then repeats. When I say "pause", I mean that the machine does not produce any output or seemingly accept any input. I know that other times I've seen these sorts of symptoms, input during the paused time would seem to be played back after the pause, but that does not seem to be the case here. (I could be wrong, though.)

There is nothing in any logs or in dmesg. I have a vague suspicion that it might be related to the nvidia kernel module, but I don't really have any good basis for that suspicion.

Since the pause seemingly affects every function of the machine, using top, or any other real-time monitor, proves ineffective, since I'm only interested in the output when it's not able to generate any.

I'm kind of out of ideas as to where to look. Anyone else have any?
Posted by: canuckInOR

Re: Linux system hiccups - 06/08/2010 15:40

Well, if you have a vague suspicion that it may be the nvidia drivers, try updating.

Otherwise, the only thing I can think of at the moment is to try disabling any swap space. Perhaps the machine is hanging up on swap i/o?
Posted by: peter

Re: Linux system hiccups - 06/08/2010 16:28

If it's a swap storm "vmstat 1" would soon show that. But if it were, the input during pause would be played back on resume. On the other hand, if it's really frozen solid for those 4 seconds, the kernel should moan in dmesg. My guess is hardware: open the machine and make sure everything (CPU, memory, PCI) is seated properly.

You can strike the nvidia module "off of" your list by booting to text-mode ("sudo telinit 3" or whatever) and seeing if it still pauses top.

Peter
Posted by: wfaulk

Re: Linux system hiccups - 06/08/2010 16:45

It's not a swap issue. I meant to mention that.
Posted by: mlord

Re: Linux system hiccups - 06/08/2010 18:06

Nothing in "dmesg" output ?
Posted by: wfaulk

Re: Linux system hiccups - 06/08/2010 18:21

Pretty much nothing since boot. Certainly nothing every 30 seconds.

A reboot "resolved" the problem.
Posted by: tman

Re: Linux system hiccups - 06/08/2010 18:23

Overheating?
Posted by: mlord

Re: Linux system hiccups - 06/08/2010 18:35

Western digital "Green" drive?

Or other drive with a short Standby/Spindown timer setting?
Posted by: wfaulk

Re: Linux system hiccups - 06/08/2010 19:25

Nope. Ancient drive. And not apparently any significant iowait anyway, not that I can feel certain of that.

Also, forcing standby mode ("hdparm -y") doesn't seem to cause the same symptoms.
Posted by: wfaulk

Re: Linux system hiccups - 13/08/2010 15:59

Okay, it's back.

Nvidia driver is only 0.0.02 versions back. I'll try going into text mode and unloading the module to see if that helps.

The only additional info I've been able to collect so far is that the reported number of interrupts and context switches drops off dramatically during the pause. Still doubtful that it's anything more than just symptomatic, though.

There is absolutely zero swap activity, so that's not it.
Posted by: BartDG

Re: Linux system hiccups - 13/08/2010 16:28

My guess is overheating as well. Maybe the CPU fan has stopped? (or isn't seated properly on top of the CPU anymore?)