How Proper Set Up Proxy Servers

[monty]

Now most people do not like ISP's to use caching proxy servers.

It creates things to go wrong, the majority of current websites are of a dynamic nature so you wouldn't want stuff to be cached etc...

However, even though I am not easily praising Thai ISP's, I must admit that TT&T's maxnet has a pretty efficient, transparent proxy server running.

It shows the best when trying to play Youtube video's. Even on the cheap Indy package, the majority of the video's come in at near line speed. Some of them don't, and need frequent buffering.

A closer look reveals, when looking at the bottom of your browser, that the fast loading video's are actually served by a Maxnet server!

I am amazed at how high the ratio is of cached video's versus non cached ones...

And you still have the option of going for the more expensive Premier package, where even the non cached video's never need buffering!

[sniffdog]

Google has dedicated bandwidth in Thailand. That's why (also my TOT connection) most videos stream quite well. Only those on certain servers stream slow.

PS I thought it was quite funny, when I saw the title that a super moderator opens a thread about proxy servers, while I thought discussing those was against the rules. I thought it would be a tutorial ...

Edited April 11, 2009 by sniffdog

[Phil Conners]

Oh, I though you were going to give us the proper configuration settings for Squid

[Tywais]

Oh, I though you were going to give us the proper configuration settings for Squid

Not enough space on a page for that.

The uni has both a transparent proxy and a caching proxy. If I set the browser to direct connection the speed tests are a fraction of when I point to the proxy. Though I have a feeling they may be penalizing those who don't use it by bandwidth caping at the routers.

[eldar1]

Unfortunately the main drawback of transparent is not the caching of dynamic content but the censoring of certain content, a very large problem here in thailand. There are ways around transparent proxies, but i will not discuss these here as this is a felony offense in thailand. Google is your friend as always but tread carefully.

Concerning the speeding up effect proxies can have, yeah that a pretty good feature of them. Thai ISP's do a decent job of maintaining them and making sure the cache is large enough to actually be usefull. Their deal with Akamai certainly helps, i get full speed downloads when watching the daily show and colbert nation and downloading extremely popular files through their content system is again blazing fast (for thailand).

[Prasert]

Oh, I though you were going to give us the proper configuration settings for Squid

http_port 3128 transparent

hierarchy_stoplist cgi-bin ?

acl QUERY urlpath_regex cgi-bin \?

cache deny QUERY

acl Google dstdomain .google.com

acl Google dstdomain .google.co.th

always_direct allow Google

acl apache rep_header Server ^Apache

access_log /home/squid/access.log

cache_log /home/squid/cache.log

cache_store_log none

cache_dir ufs /home/squid/cache 100 16 256

logfile_rotate 90

hosts_file /etc/hosts

refresh_pattern ^ftp: 1440 20% 10080

refresh_pattern ^gopher: 1440 0% 1440

refresh_pattern (cgi-bin|\?) 0 0% 0

refresh_pattern . 0 20% 4320

acl all src 0.0.0.0/0.0.0.0

acl manager proto cache_object

acl localhost src 127.0.0.1/255.255.255.255

acl localnet src 10.0.0.0/8 # RFC1918 possible internal network

acl localnet src 172.16.0.0/12 # RFC1918 possible internal network

acl localnet src 192.168.0.0/16 # RFC1918 possible internal network

acl to_localhost dst 127.0.0.0/8

acl SSL_ports port 443 563 # https, snews

acl SSL_ports port 873 # rsync

acl Safe_ports port 80 # http

acl Safe_ports port 21 # ftp

acl Safe_ports port 443 563 # https, snews

acl Safe_ports port 70 # gopher

acl Safe_ports port 210 # wais

acl Safe_ports port 1025-65535 # unregistered ports

acl Safe_ports port 280 # http-mgmt

acl Safe_ports port 488 # gss-http

acl Safe_ports port 591 # filemaker

acl Safe_ports port 777 # multiling http

acl Safe_ports port 631 # cups

acl Safe_ports port 873 # rsync

acl Safe_ports port 901 # SWAT

acl purge method PURGE

acl CONNECT method CONNECT

http_access allow manager localhost

http_access deny manager

http_access allow purge localhost

http_access deny purge

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost

acl lan src 172.25.0.0/16

http_access allow lan

http_access deny all

http_reply_access allow all

icp_access allow all

always_direct allow all

coredump_dir /var/spool/squid

redirect_program /usr/local/asqredir/asqredir /usr/local/asqredir/urls.txt

....proper configuration settings....

[surface]

I'd like to thank True for their local caching of torrents using Azureus on a few hosts. I don't know if it's just some employees there or a corporate policy to save on the limited international bandwidth, but most torrents I download usually have a few peers on True that deliver at the full speed of my connection. The IP's of these peers has not changed since I've noticed them, which has been at least 6 months, and the speeds I receive from them are 300k+ each.

You may think this is crazy BUT I just recently started a download of some rather old files which couldn't be considered popular by any stretch of the imagination and have been around for ages. A day after I started downloading them, I noticed these hosts were feeding me at full speed, yet the hosts themselves were only at 15% - 30% themselves. It appears to me that True has some script running which detects torrent files being downloaded (perhaps it's set off after x number of people have requested the file) and they begin to download the file themselves with these hosts to cache the content locally.

As much as people like to talk trash about True, try to find me a single ISP in the west that does that!

Bravo True, BRAVO!

[Phil Conners]

Oh, I though you were going to give us the proper configuration settings for Squid

http_port 3128 transparent

hierarchy_stoplist cgi-bin ?

acl QUERY urlpath_regex cgi-bin \?

cache deny QUERY

acl Google dstdomain .google.com

acl Google dstdomain .google.co.th

always_direct allow Google

acl apache rep_header Server ^Apache

access_log /home/squid/access.log

cache_log /home/squid/cache.log

cache_store_log none

cache_dir ufs /home/squid/cache 100 16 256

logfile_rotate 90

hosts_file /etc/hosts

refresh_pattern ^ftp: 1440 20% 10080

refresh_pattern ^gopher: 1440 0% 1440

refresh_pattern (cgi-bin|\?) 0 0% 0

refresh_pattern . 0 20% 4320

acl all src 0.0.0.0/0.0.0.0

acl manager proto cache_object

acl localhost src 127.0.0.1/255.255.255.255

acl localnet src 10.0.0.0/8 # RFC1918 possible internal network

acl localnet src 172.16.0.0/12 # RFC1918 possible internal network

acl localnet src 192.168.0.0/16 # RFC1918 possible internal network

acl to_localhost dst 127.0.0.0/8

acl SSL_ports port 443 563 # https, snews

acl SSL_ports port 873 # rsync

acl Safe_ports port 80 # http

acl Safe_ports port 21 # ftp

acl Safe_ports port 443 563 # https, snews

acl Safe_ports port 70 # gopher

acl Safe_ports port 210 # wais

acl Safe_ports port 1025-65535 # unregistered ports

acl Safe_ports port 280 # http-mgmt

acl Safe_ports port 488 # gss-http

acl Safe_ports port 591 # filemaker

acl Safe_ports port 777 # multiling http

acl Safe_ports port 631 # cups

acl Safe_ports port 873 # rsync

acl Safe_ports port 901 # SWAT

acl purge method PURGE

acl CONNECT method CONNECT

http_access allow manager localhost

http_access deny manager

http_access allow purge localhost

http_access deny purge

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost

acl lan src 172.25.0.0/16

http_access allow lan

http_access deny all

http_reply_access allow all

icp_access allow all

always_direct allow all

coredump_dir /var/spool/squid

redirect_program /usr/local/asqredir/asqredir /usr/local/asqredir/urls.txt

....proper configuration settings....

This configuration does not fully anonymize you, HTTP_X_FORWARDED_FOR still carries your real IP address...