High Load But Low Cpu And Ram Useage

[Crushdepth]

I'm having problems with my virtual private server. It keeps locking up, creating havoc with my work website. Apache keeps restarting as well. Using the 'top' utility, the load average is quite high (spiking to > 4) during periods of lock up. But CPU and memory useage are very low, no processes are obviously consuming much CPU or RAM.

Support are unable to fix it or explain what the problem is. They've rebooted the VPS, recompiled Apache, and I upgraded my plan to 1 gig of SLM RAM. It hasn't helped. I tried turning off Apache and MySQL services (thereby shutting down my website) and the VPS *still* overloads. I'm stumped.

How can you have a high load average when your CPU and RAM are largely idle?

Edited March 11, 2009 by Crushdepth

[dave_boo]

Do you have beagled running? Could be indexing your drive(s). Otherwise don't have much more of an idea; perhaps you could drop us an idea of running processes? A

#ps aux | less

will give us the pertinent information.

[Deksan]

I'm having problems with my virtual private server. It keeps locking up, creating havoc with my work website. Apache keeps restarting as well. Using the 'top' utility, the load average is quite high (spiking to > 4) during periods of lock up. But CPU and memory useage are very low, no processes are obviously consuming much CPU or RAM.

Support are unable to fix it or explain what the problem is. They've rebooted the VPS, recompiled Apache, and I upgraded my plan to 1 gig of SLM RAM. It hasn't helped. I tried turning off Apache and MySQL services (thereby shutting down my website) and the VPS *still* overloads. I'm stumped.

How can you have a high load average when your CPU and RAM are largely idle?

Hi,

Just a side question where did you order a VPS ? Is it in thailand ? How expensive was it ? It seems to me that there is a configuration issue or the machine where the VPS is hosted is overloaded.

Thanks.

[Crushdepth]

The VPS is in the US with Futurehosting.com. In the past their facilities and service have been outstanding. Now I am beginning to wonder if they are chronically overloading their servers - they have offered me a replacement VPS, but the load average on that the new one seems a bit high considering that I haven't put anything on it yet!

I'll post the ps results on my original vps as soon as I managed to get a shell open.

[Crushdepth]

Ok I've attached the output of ps as a text file as it goes over a couple of screens. Appreciate your feedback.

psoutput.txt

[Deksan]

Ok I've attached the output of ps as a text file as it goes over a couple of screens. Appreciate your feedback.

Looks like you have a lot of /usr/local/bin/sim process. I dont know what that is but it seems weird tohave that many ?? Could you try a vmstat 1 ? You will get something like that :

procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------

r b swpd free buff cache si so bi bo in cs us sy id wa st

0 1 415096 28144 24152 2298912 0 0 425 50 0 1 3 2 91 4 0

0 1 415096 25384 24104 2302044 0 0 50844 96 695 1092 0 1 87 11 0

0 0 415096 26340 24080 2301040 0 0 6984 48 1213 1393 1 0 96 2 0

0 1 415096 26340 24112 2301008 0 0 32 208 350 524 2 2 92 4 0

0 0 415096 26248 24536 2301612 0 0 0 720 301 571 0 0 98 2 0

1 0 415096 26248 24540 2301608 0 0 4 64 296 356 0 2 97 1 0

0 0 415096 26248 24544 2301604 0 0 4 228 319 500 0 0 97 3 0

0 0 415096 26248 24544 2301604 0 0 0 144 270 355 0 0 99 1 0

[Phil Conners]

With the current prices VPS are offered for nobody should be surprised that they are hugely overloaded. You can only run so many VPS servers on any given physical server. Try doing the math with the cost of a hysical server (appropriately configured, which will serve several VPS servers, i.e. typical multi-cpu, multi-core, 12 GB RAM, Ultra fast SCSI drives etc...). Such a server will cost in the area of 800 USD/month. How many 20 USD/m VPS servers (with all the expected features, i.e. unlimited this and unlimited that) do you need to sell just to break even.

Edited March 12, 2009 by Phil Conners

[TopDogger]

Switch to a host that uses Xen virtualization..

[Phil Conners]

Xen can't create resources that aren't there ...

[dave_boo]

Deksan; sim seems to be a process used by servers to monitor services. It's similar to CPanel...but I agree, there's a lot of instances of it running, especially with those piping to the /dev/null (but that could be standard practice-I wouldn't know because I don't use sim). Also of concern is how many instances of Apache are running. Just from the user 'nobody' there's 6 instances of it, and theres another from 'root'. Granted, per the state column, the root instance is in Interruptible Sleep, Low Priority, and session leader. The majority of the instances started by 'nobody' are Interruptible Sleep, Low Priority, and Multi-threaded. However, all together they consume ~20% of the system's memory! Seems really high for httpd that's been running for only 1 day.

Having pointed that out, I'd like to side with Phil. IF your hosting company allows it, I'd try and install a different distro (SuSE has a wonderful net install) and go from there. Yes it's a lot of work, but wouldn't that be better than the current situation?

[Crushdepth]

sim is the 'system integrity monitory', it monitors various services and if they go down it emails a warning and tries to restart them. It does seem odd to have multiple instances (can't one monitor the lot?).

Apache bothers me a bit. I don't know anything about turning Apache, but each instance seems to be using a lot of memory (~35MB), at least a few articles I read yesterdays said 12-15MB is 'good'.

Changing OS is not an option. The support staff have thrown in the towel and offered to move me to a clean VPS, hopefully that will sort things out. If not, I'll be looking for a new provider. A shame because this company really have been great up till now.

[TopDogger]

Xen can't create resources that aren't there ...

Nobody said it could, I suggest you do a little reseach into the benefits of Xen..

Edited March 13, 2009 by TopDogger

[Phil Conners]

Xen can't create resources that aren't there ...

Nobody said it could, I suggest you do a little reseach into the benefits of Xen..

You're preaching to the choir. I'm well aware of both the advantages (and disadvantages) of Xen.

Edited March 13, 2009 by Phil Conners

[Veazer]

...Apache bothers me a bit. I don't know anything about turning Apache, but each instance seems to be using a lot of memory (~35MB), at least a few articles I read yesterdays said 12-15MB is 'good'....

Is Lighttpd an option for you? i've seen a few guides for setting up lighttpd on VPS going around.

[TopDogger]

Xen can't create resources that aren't there ...

Nobody said it could, I suggest you do a little reseach into the benefits of Xen..

You're preaching to the choir. I'm well aware of both the advantages (and disadvantages) of Xen.

You'll be well aware that its much more difficult to overload servers with Xen then...

[Nagatus]

Ok give me a break, I know this if first post.

AFAIK the OP said the the load was high despite the VPS not running the services (apache...)

That leaves the system processes that are installed by default by the service provider.

From the process list I saw, there seem to be nothing unusual apart from the number of sleep 3 processes, but I assume they are related to sim.

_IF_ the OP has not installed anything else on the server, there is no reason why the load shoulg be high.

Typically you see high loads on virtual servers due to the host OS having problems with memory usage and needing to swap a lot. This causes a lot of I/O waits which increase the load significantly without the CPU being used.

HTH

Edited March 13, 2009 by Nagatus

[Crushdepth]

The move to the new VPS is just about finished, load on this one now seems much lower than previously. Looks quite good actually. A big relief to have it sorted out. httpd is using 12-21 MB per instance in the new set up (as opposed to ~35MB each in the old one).

Edited March 14, 2009 by Crushdepth