Sunday, January 27, 2008

Transparent proxy with Squid 3 on Ubuntu server



This is a short guide how to set up a transparent cache proxy on Ubuntu Server. Basically, there are two reasons why you may be interested setting up a Squid proxy: speed up your web access and save your bandwidth. Let assume that you already set up a Linux gateway using iptables. It is possible to run squid also on Windows but unfortunately it is extremely hard to find a hardware on which Windows is able to survive more than one week. Other words, windows is out of scope of this article.
The first step is to install squid 3

apt-get install squid3

edit the squid 3 configuration file in your favorite editor

sudo vi /etc/squid3/squid.conf

and set the transparency and the allowed hosts

http_port 3128 transparent
acl our_networks src 192.168.0.0/24
acl localnet src 127.0.0.1/255.255.255.255
http_access allow our_networks
http_access allow localnet

where 192.168.0.0/24 is the IP range of local network. Probably you need adjust the swap size

cache_dir ufs /var/spool/squid3 7000 16 256

where the first number denotes the size of cache in megabytes. Save you changes and restart the squid proxy by

sudo /etc/init.d/squid3 restart

For more detailed configuration read the manual of Squid or check the configuration examples on Squid wiki page.
Remember, the memory and processor usage of squid is a function of swap size.
Last but not the least we need to redirect the HTTP traffic to your new shiny proxy

iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.0.1:3128
iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128

where eth1, eth0 are the LAN, WAN devices and 192.168.0.1 is the IP address of your LAN device.

After all, probably you feel the need to monitor the performance of your proxy. Unfortunately, most of the squid log parsers in the Ubuntu repository are configured for Squid 2.x. Nevertheless, squid 3 uses the same log format, so you can change the log file path in your parser config file (sarg, calamaris, etd.) or simply link the log directory of squid 3 to the correct path

ln -s /var/log/squid3 /var/log/squid

Good luck!

26 comments:

Anonymous said...

first thing this is not written by ubuntugeek admin and you can check the submitted username.I have checked the article but it is totally different in configuration samples if you think author copied from your article i have given the source if you don't want that please mail us to ubuntu geek team at admin@ubuntugeek.com we are happy to modify the article without your concern

Anonymous said...

I'm usiing Squid3 Stable 1 on Ubuntu via apt-get,

transparency it's not working for me no matter what,

but if i'm set manually in web browser, it works like magic,

have you proof it yourself?

thx for yur useful post, sorry my bad english

Zoltan Kuscsik said...

The POST is updated!
There was an mistyped IP in the proposed configuration. I also added the 127.0.0.1 network between allowed hosts.

Anonymous said...

Nice guide, but I have a question: did you try to create transparent content filtering using dansguardian ? If you could help me with that question, I'd be very thankfull.
You can read whole description on ubuntu forums: http://ubuntuforums.org/showthread.php?p=6364953#post6364953

Anonymous said...

You'll want to you use aufs under Linux instead of ufs.

Anonymous said...

Thanks a lot. Worked very well for me.

Anonymous said...

is there a link to know about using iptables to set up a linux machine as a gateway...i use simple ip-forwarding from the networking configuration, to setup the machine as a gateway...no iptables involved...will the squid not work with that?

Anonymous said...
This comment has been removed by a blog administrator.
fred-tc said...

Thanks for your information, it's usefull. A reader from Taiwan.

Anonymous said...

Thanks to your Post I now have squid3 working as a proxy server on ubuntu for my windows network. thank you

Anonymous said...

I followed this to script and it worked beautifully, however Squid3 would not start automatically. I had to add the dns ip addresses to the dns_nameservers entry in the squid.conf file then it started on startup. This was the case even though my resolv.conf config was correct. Thought it might help someone.

Devinder said...

Hi I had followed the steps and i still need to enter the proxy server IP.

I want to use an external proxy server 202.75.x.x:62001 to allow all LAN users to access to that proxy server when they browse for advertisement banner.

Anonymous said...

Thank you
# so good even I could {eventually} follow the instructions.
Pity I can’t get it to NAT all protocols and log them.

Anonymous said...

iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128

error on --to-ports

i'm using ubuntu server..pls help

25.806 said...

@ Anonymous: I used this configuration and it works, maybe you mistyped something?

@ Zoltan, thanks for this quick guide, very useful! But i have the same problem Marissa had... No transparency at all. Any hints?

Sholihin Mohamood said...

i still cant do this setup..can you give picture..

Adam said...

Hello,
I have pretty much the same problem, slightly different, I guess transparency is different from reverse proxy?

I am trying to use squid3 for the first time, I am not very familiar with it's logic.

I have three different domain names

example.org
example.com
example.net
All three point to the same IP address:

At the moment they all run on one Ubuntu server using virtual hosting, but because it getting too much for one machine to handle 3 websites I decided to use proxy server, at the moment I have a DNS/DHCP server running on another machine.

Now let's say I have three machines on my local network, and each machine handles one site.

How do I use proxy server to forward requests to each machine (Domain name)?

I have tried several example, checked several forums, and yet nothing seem to be working.

Now if the iptables can do all of that, my question is then what is the point in using a proxy server as a reverse proxy?

I am just totally confused.

I use Ubuntu Hardy on most of my machines.

Total:

One server DNS/DHCP used as a (router)right now.

3 other machines each has a website, mysql database on it for that site, each has a web-server.


Do I need to run three web-servers for the three websites?

Or should I use one webserver forwarding requests to the proxy server on port 3128 and then the proxy will forward the request to the right domain name (Site) ?

Can anyone here please clarify it for me, I would really appreciate your help.

Sorry if it's too long

Regards to all

Anonymous said...

Well your article helped me truly much in my college assignment. Hats incorrect to you enter, choice look forward in the direction of more interdependent articles soon as its united of my favourite issue to read.

Anonymous said...

Search engine Optimization
very handy, thanx a lot for this blog .. This is exactlpy hwat I was looking for.

Unknown said...

I have a standalone machine running both dansguardian(127.0.0.1:8888) and squid (127.0.0.1:3333) Trying to get transparent proxy working. Please assist if you can.

The forward rule below is the only rule i have in my ipfw settings. Will this not work because ipfw cannot forward requests originating from the local machine?

fwd 127.0.0.1,8888 uid squid tcp from any to any 80

When I request URL page, the rule doesn't get any hits ... and goes straight to the Internet bypassing completely.

Unknown said...

I have a standalone machine running both dansguardian(127.0.0.1:8888) and squid (127.0.0.1:3333) Trying to get transparent proxy working. Please assist if you can.

The forward rule below is the only rule i have in my ipfw settings. Will this not work because ipfw cannot forward requests originating from the local machine?

fwd 127.0.0.1,8888 uid squid tcp from any to any 80

When I request URL page, the rule doesn't get any hits ... and goes straight to the Internet bypassing completely.

Joe Negron NYC said...

@rig - I'm not too sure on the protocol or that particular ipfw NAT rule syntax and I've not used squid before - I am doing research for an upcoming project. however just as an observation, logically I would assume that the line fwd 127.0.0.1,8888 uid squid tcp from any to any 80 looks like it only handles one-way redirects. For true transparency, wouldn't you need two rules? one redirecting normal traffic to squid port and another reversing that process? I think you would want something like
WAN:80 --> LAN:8888
and LAN:8888 --> WAN:80

does that make sense?

If we end up going with squid, I will most likely research ipfw rules syntax more thoroughly. I assume you solved the problem or found a workaround, but please tell me - how did you overcome this challenge?

Joe Negron NYC
LogicWizards.NET

Anonymous said...

YOU FAIL

Failed to start Squid :

2011/01/04 14:22:36| ACL name 'all' not defined!
FATAL: Bungled (null) line 180: http_reply_access allow all
Squid Cache (Version 2.7.STABLE6): Terminated abnormally.

Unknown said...

Everything working fine but Squid is refreshing cache very fast. Which is making Squid useless.

I want Squid to provide images from cache and refresh cache every 30 minutes

Unknown said...

Everything worked fine.

But Squid is reading image files from Apache again and again.

I want Squid to respond to the request from Cache and refresh cache every 30 minutes.

How to do that?

Anonymous said...

this web site is my intake , rattling fantastic pattern and perfect content .
Assisted me a lot, just what I was looking for : D.