Error Too Many Server Requests

[britmaveric]

Probably stating the obvious, but was unable to get the forum for a bit...

"error too many server requests"

[george]

We had to do a security patch, and it failed the first time, so we had to request another re-start of the server.

Fixed now. Solly, solly!

[britmaveric]

We had to do a security patch, and it failed the first time, so we had to request another re-start of the server.

Fixed now. Solly, solly!

<{POST_SNAPBACK}>

No worries - I had confidence you lot would sort it out!

[daleyboy]

Cool same problem i was having

[RDN]

Just for the record:

---

---

---

---

---

A worrying 60 minutes...

Edited November 9, 2005 by RDN

[britmaveric]

RDN - I'm quite certain George is religious about the backups for the dBase.

[george]

It's a headache to backup/update/upgrade a huge MySQL database that is in heavy use with several hundred users accessing the database. Should I close the server? Should I try to have the system up and running in the meantime?

What would you do?

[daleyboy]

If i was you george i would put a note up saying server closed while maintence is carried out. If it means less downtime i am all for it.

[sriracha john]

mySQL error: User thaivcom_ibrd1 has already more than 'max_user_connections' active connections

mySQL error code:

Date: Wednesday 09th of November 2005 12:13:41 AM

Just curious, how many connections ARE "more than max user connections"??

[britmaveric]

It's a headache to backup/update/upgrade a huge MySQL database that is in heavy use with several hundred users accessing the database. Should I close the server? Should I try to have the system up and running in the meantime?

What would you do?

<{POST_SNAPBACK}>

Well mirroring would be a good solution - main server goes down and secondary kicks in without a blip.

However probably a customized maintenance default message when work is being done would be a cheaper alternative. (avoid us worrying)

[george]

Mirroring would work with a static website, but not with a very active database driven forum database.

We need to upgrade again, I reckon'....

[george]

mySQL error: User thaivcom_ibrd1 has already more than 'max_user_connections' active connections

mySQL error code:

Date: Wednesday 09th of November 2005 12:13:41 AM

Just curious, how many connections ARE "more than max user connections"??

Around 1,000, but it's not important. The error message is somehow misleading (server is down due to backup, database is locked, and the error message says something else).

[britmaveric]

Mirroring would work with a static website, but not with a very active database driven forum database.

We need to upgrade again, I reckon'....

<{POST_SNAPBACK}>

Wonder if you can use Oracle with Invision? Probably too cost prohibitive at any rate.

Edited November 8, 2005 by britmaveric

[george]

Server load....

When the load increases, it takes more and more time to server each user.

Let's say 2000 persons are visiting the forum, and each seeing a page every 10 seconds.

That means that there are in average 200 pages viewed per second.

Ok, let's say it takes 2 milliseconds to serve a page. 2 ms * 200 pages = 400 ms. That means we are already at 40% of our capacity.

If a script, service or process slows down the server, and it now takes 6 milliseconds to serve a page, then, in that case, it takes 6 * 200 = 1200 Ms to serve the pages from that second. That means that 200 / 6 = 33 users got served in more than 1 second and had to wait for the second second. 167 users got served in the first second.

Is that clear so far ? The first 167 users got served in the first second, but 33 had to wait for the second second.

The trouble is that another 200 users want to be served on that second too, causing 233 users to ask to be served, but once again, only 167 being serviced.

So, the 3rd second, when another 200 user kick in, there are already 33 + 33 users in the queue, for a total of 266.

As seconds go by, the number of user reach higher and higher, and users wait more and more. Eventually, one of 3 things occur :

1 ) The server reaches the max number of users ( as RDN reported ) and stops serving SOME of the users, but still serving 167 users per second.

2 ) The server crashes on the load ( if the max number of users is too low )

Eventually, the users get impatient and slow down, causing the number of new users per seconds to slow down and giving a chance for the forum to recuperate. So far, the volume doesn't seem to make that happend often...

#1 is better then #2, because only the excess users see the message. Remember, if the max users is at 500, then the first 500 users are served well, it is only the additional ones that are blocked.

Well. time to sleep...

[Guest endure]

There's backup software for SQL Server that dumps the database to another file and then backs that up. Isn't there anything similar for MySQL?

[george]

We can use any kind of backup software or script, but we want to have the system up in the mean time. Believe me, we have tried many solutions...

We will upgrade the server again shortly, our hardware is an important factor as well.

[daleyboy]

Just wondering, do you count the guests when you talk about the 2000 max users?

I guess one possible solution would be to have the forum members only this includes viewing it. After all the guests cant contribute to the forum anyway. It might even increase the number of people contributing to the site, stop all this lurking in the shadows.

Just an idea

[RDN]

RDN - I'm quite certain George is religious about the backups for the dBase.

<{POST_SNAPBACK}>

Normally the backups occur at 2 am Thai time - at least that's the time I notice a very slow response for about 15 minutes - 2:00 am to 2:15 am.

Tonight's 60 minute "down time" was due to applying a security patch (as George stated earlier) twice.

I used to work on a dual redundant system in the UK (2 DEC Alpha computers connected via an Ethernet link), and they dealt with about 150 messages per second throughput. But the database there was proprietary - just a flat file ("section file") held completely in memory and only took about 10 seconds to back-up (to disk). Subsequent copying of the backed-up disk files to DAT tape took a couple of hours, but didn't interfere with normal operation of the system.

We eventually linked 3 pairs of such systems in a simple network around London. Uptime was about 99.997% - an incredibly reliable system. I once updated the software on all 6 machines and noted that the last time the machines were out of service was 525 days previous. That was the last time I'd been there to update them!

[sriracha john]

mySQL error: User thaivcom_ibrd1 has already more than 'max_user_connections' active connections

mySQL error code:

Date: Wednesday 09th of November 2005 12:13:41 AM

Just curious, how many connections ARE "more than max user connections"??

<{POST_SNAPBACK}>

Around 1,000, but it's not important. The error message is somehow misleading (server is down due to backup, database is locked, and the error message says something else).

<{POST_SNAPBACK}>

Thanks for this and subsequent explanations, George.

Sounds then like it's basically a way for the system to save face. It doesn't wish to admit that the server is down for one reason, so instead it reports a different, less confrontational, reason.

That's certainly much more agreeable than what one might expect to read if it was an American system:

"Hey, dumbass! We're too fcukin' busy right now!...We'll let you back in when we're fcukin' well ready! Until then, screw off and come back later!"

----------------------

SJ comments: Best of luck to you and all of TV...and thank you for all your efforts for a job well done!

[britmaveric]

DEC Alpha(s) reliable lil bastard(s) aren't they? I worked w/few of those a number years back - loved em - naught problems ever.

[RDN]

George, I hope you have a suitable notice on your equipment:

Das machine is nicht fur gerfingerpoken und mittengrabben. Ist easy schnappen der Sprinngwerk, blowenfusen und poppencorken mit spitzensparken. Ist nicht fur gewerken by das Dummkopfen. Das rubbernecken sightseeren keepen hands in das Pockets. Relaxen und watch das blinkenlights...

[RDN]

DEC Alpha(s) reliable lil bastard(s) aren't they? I worked w/few of those a number years back - loved em - naught problems ever.

<{POST_SNAPBACK}>

Brilliant machines. We started with Alpha 3000's. Dinosaurs now, but totally reliable. And OpenVMS - based on good old RSX-11M Plus. Miss it, sometimes. Not often.

[RDN]

It's a headache to backup/update/upgrade a huge MySQL database that is in heavy use with several hundred users accessing the database. Should I close the server? Should I try to have the system up and running in the meantime?

What would you do?

<{POST_SNAPBACK}>

Seriously George, I tend to make a cup of tea at 2:00 am - backup time - because the response is so slow. If it helps to disconnect all users from the server (to get the backup done quicker) then I would do that. We would all get used to the backup time and accept that it is for the good of the system.

If you could display a "Backup in progress - back in 10 minutes" message at that time, I don't see a problem.

[george]

Sounds like a good idea to me, RDN. Will talk to the hostmaster.

[Richard W]

If you could display a "Backup in progress - back in 10 minutes" message at that time, I don't see a problem.

<{POST_SNAPBACK}>

An even better message would be, "Backup in progress - down for 10 minutes, back by 03.17 (Thai time)". UTC (= GMT, not always the same as UK time) would be better than no time.

[RDN]

If you could display a "Backup in progress - back in 10 minutes" message at that time, I don't see a problem.

<{POST_SNAPBACK}>

An even better message would be, "Backup in progress - down for 10 minutes, back by 03.17 (Thai time)". UTC (= GMT, not always the same as UK time) would be better than no time.

<{POST_SNAPBACK}>

I'm surprised you didn't also suggest an equivalent Thai message, Richard!