Caching Proxy Servers

[Tywais]

I recently had complaints from some staff members at the university that their Internet access was very slow. The speed tests from my office computer are shown below. One to the Thaivisa speed test site and the other to the ADSL Thailand speed test for local (Thailand) tests. I randomly selected 6 computers at the lab to run tests on. Three showed consistently high speeds the same as mine, three showed speeds 1/10 of that, around 400 - 500 kbps versus 6 Mbps mine showed.

Brought my EEE PC in to work to check if it was problems with our internet switch or specific PC. Pulled the internet cable from my desktop and plugged into the notebook and used the same IP. Speedtests from the notebook showed only around 700 - 800 kbps, so not the network switch. Checked my FF configuration and looked at the proxy settings. My desktop is using the university caching proxy server, the notebook was not. Changed the notebook, reran the tests and voila, full speed now.

So if you have speed issues, you might check if you are using your ISPs proxy servers. A factor of 10 difference for our facility. Your mileage may vary.

[niller74]

Very true in certain scenarios. It is still only traffic that is actually cached that make the difference. If your ISPs proxy have to get the data for you, you will experience a much slower rate in most cases. It is also VERY limited which data is actually stored on the proxy. Unless the ISP give higher priority to customers using their proxy, I strongly doubt people will see a large increase in modern webtraffic where most is dynamicly created.

Proxys are nice though.

What puzzles me is that you were actually able to access the internet from University without going through the proxy.

[Tywais]

What puzzles me is that you were actually able to access the internet from University without going through the proxy.

To be honest, this has me a bit confused also, not the requirement to use the proxy but that the speed tests should not really be effected by it since it is not caching pages but direct point to point data transfer. The university also has a transparent proxy which we are automatically routed to and do not have to specify it but makes me wonder if the caching server is doing something else with the traffic to cause that big a difference in speed. My technicians page loads were terrible until I put the proxy in his configuration and now loads nearly instantly.

One possibility is that the caching server is directly routed to the transparent proxy and optimizing routing.

[MikeWill]

Tywais,

would it be possible to investigate that issue further? Maybe bypassing the caching server and connecting directly to the optimizing routing (though am not exactly clear what is that).

[Tywais]

Tywais,

would it be possible to investigate that issue further? Maybe bypassing the caching server and connecting directly to the optimizing routing (though am not exactly clear what is that).

I run my own caching server on our firewall but disabled it several weeks ago due to an instability (need to replace the hard drive) and plan on restarting it and testing via that server. All our computer traffic goes through my firewall/Internet server so suspect it won't be any different then when not using the uni server.

I suspect the university proxy is doing more then performing as a caching server due to the fact the speed difference is 10x. Plus the speed tests are point to point data transfers and as such caching web pages wouldn't help in this case. If anyone has any ideas as to how that kind of speed difference can be had from just pointing to a proxy I would be interested, mostly because I hate mysteries.

I'm going to run some torrent tests also since a staff member mentioned his was slow now. That is a case where no proxy would be involved. Normally our torrent downloads hit from 5-10 Mbps depending on seeds/peers and location.

Our line goes to a Cisco routing farm at another department then on to the main computer center. Can't change that routing.

[monty]

Tywais,

would it be possible to investigate that issue further? Maybe bypassing the caching server and connecting directly to the optimizing routing (though am not exactly clear what is that).

I run my own caching server on our firewall but disabled it several weeks ago due to an instability (need to replace the hard drive) and plan on restarting it and testing via that server. All our computer traffic goes through my firewall/Internet server so suspect it won't be any different then when not using the uni server.

I suspect the university proxy is doing more then performing as a caching server due to the fact the speed difference is 10x. Plus the speed tests are point to point data transfers and as such caching web pages wouldn't help in this case. If anyone has any ideas as to how that kind of speed difference can be had from just pointing to a proxy I would be interested, mostly because I hate mysteries.

I'm going to run some torrent tests also since a staff member mentioned his was slow now. That is a case where no proxy would be involved. Normally our torrent downloads hit from 5-10 Mbps depending on seeds/peers and location.

Our line goes to a Cisco routing farm at another department then on to the main computer center. Can't change that routing.

Hmmm,

Is it possible that there is some QOS being done at the main internet source, with traffic priority to the the proxy server?

As you say a ten fold difference is very big, even with partly cached content.

[niller74]

As monty said, the only logical explanation is that traffic going through the caching proxy is getting a higher priority or even using another connection.

I know that University of Copenhagen were honoring people that actually used the caching proxy with a higher traffic priority.

[Tywais]

As monty said, the only logical explanation is that traffic going through the caching proxy is getting a higher priority or even using another connection.

I know that University of Copenhagen were honoring people that actually used the caching proxy with a higher traffic priority.

That's a pretty reasonable answer and may well be it. One thing I noticed is the that the two sites I ran tests on had 6Mbps (Thaivisa) and 20-22 Mbps (ADSLThailand) through the proxy but without the proxy they were not proportionally slower but equal, that is running around 700 - 800 kbps for both speedtest sites. Nearly like bandwidth shaping/throttling though torrents still run up to 5 - 6 Mbps on download.

[stumonster]

what is in the cache of your proxy ? ad's ?

I use firefox with noscript and adblock and the difference is stark when I have to do something with a browser on some one else's machine.

[phazey]

What proxy are you using? Is it Squid?

If so, check out delay_pools. also set your caching policy to (i think) LRU, and storage mode to diskd. If you've a large spool, increase the number of child directories. There are loads of other tweaks, but i think i'll waste my time if it's not Squid