pfSense WPAD/PAC proxy configuration guide
Last revised 11 December 2017.
I don’t use a proxy on my network for ad filtering any more as I moved over to pfBlockerNG which provides greater control and flexibility however I’ve received a number of requests for an updated guide so here it is. I initially set my proxy to work transparently but this method didn’t work fully with all the recent changes to my network. To enable my filtering system to continue working I had to configure each device separately to use port 3128 which was an inconvenience. A system called WPAD/PAC (Web Proxy Auto-discovery Protocol & Proxy Auto Configure) promised an easy workaround and the ability for devices to auto configure themselves when connecting to to my network. This wasn’t as easy as I expected partly due to different devices having different ways of accessing the configuration files.
Create proxy discovery files 프록시 검색 파일 생성
First create the proxy configuration files. Three of them are required due to different browsers using different filenames to obtain proxy configuration information.
We’ll create the initial proxy.pac
file, and then create two ‘linked’ files to wpad.dat
and wpad.da
files which will make keeping them all in sync easier.
ee /usr/local/www/nginx/proxy.pac
Paste the following into the file you just created. Change my proxy address (192.168.0.254) to yours obviously.
function FindProxyForURL(url,host)
{
// If the requested website is hosted within the internal networks, send direct.
if (isPlainHostName(host) ||
shExpMatch(host, "*.local.lan") ||
isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
isInNet(dnsResolve(host), "172.16.0.0", "255.240.0.0") ||
isInNet(dnsResolve(host), "192.168.0.0", "255.255.0.0") ||
isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
return "DIRECT";
//Default rule: all other traffic use below proxy
return "PROXY 192.168.0.254:3128";
}
This PAC file checks to see if the host which is being looked up is a plain host name, i.e “intranet”, or is in the local networks, if it is, the access bypasses the proxy (“DIRECT”). If neither of the above conditions are met then the access is made via the proxy and port 3128.
Its possible to extend this PAC files functionality to include features such as black/white lists etc,
Create the other two file variations by generating links. This is preferable as it keeps all versions synced if you make any changes in future.
ln -s /usr/local/www/nginx/proxy.pac /usr/local/www/nginx/wpad.dat
ln -s /usr/local/www/nginx/proxy.pac /usr/local/www/nginx/wpad.da
Verify your folder looks something like this when done,
$ ls -l
-rw-r--r-- 1 root wheel 537 Jul 19 12:07 50x.html
-rw-r--r-- 1 root wheel 612 Jul 19 12:07 index.html
-rw-r--r-- 1 root wheel 212 Aug 6 11:43 proxy.pac
lrwxr-xr-x 1 root wheel 9 Aug 6 11:45 wpad.da -> proxy.pac
lrwxr-xr-x 1 root wheel 9 Aug 6 11:45 wpad.dat -> proxy.pac
Setup a second nginx web server 두 번째 nginx 웹 서버 설정
Best practice dictates we should only allow access to the pfSense webConfigurator via secure HTTP and on a different port to 443. This leaves us with a slight problem. The default and only system file server is dedicated to serving the webConfigurator pages on this secure port and can’t be used simultaneously to serve WPAD data on port 80 any more. To be able to serve up the WAD config files, we need to create a second file server which will handle the WPAD requests for us.
Lets start by duplicating the systems default nginx configuration file. We can use this as the basis for our second nginx instance. Don’t try and create this file in /var/etc folder as it will be deleted on reboot. We’ll use the same folder as where our config files are stored.
ee /usr/local/etc/nginx/nginx-wpad.conf
Paste the contents below into this file.
worker_processes 1;
pid /var/run/nginx-wpad.pid;
events {
worker_connections 1024;
}
http {
include wpad-mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name wpad;
location / {
root /usr/local/www/nginx;
index index.html index.htm;
}
}
}
Add additional filetypes
Lets create a specific wpad-mime.types for this instance of nginx by entering ee /usr/local/etc/nginx/wpad-mime.types
. Paste the following lines into this file.
types {
text/html html htm shtml;
text/css css;
image/jpeg jpeg jpg;
text/plain txt;
application/x-ns-proxy-autoconfig pac;
application/x-ns-proxy-autoconfig dat;
application/x-ns-proxy-autoconfig da;
}
Verify second nginx is working
Test our second nginx instance works by entering the following command and verifying you don’t see any errors.
/usr/local/sbin/nginx -c /usr/local/etc/nginx/nginx-wpad.conf
You can verify both Nginx servers are running by entering
ps ax | grep -i "nginx: master"
the result should include both nginx instances are running with their respective configuration files, for example.
$ ps ax | grep -i "nginx: master"
ps ax | grep -i "nginx: master"
52936 - Is 0:00.00 nginx: master process /usr/local/sbin/nginx -c /var/etc/nginx-webConfigurator.conf (nginx)
75222 - Is 0:00.00 nginx: master process /usr/local/sbin/nginx -c /usr/local/etc/nginx/nginx-wpad.conf (nginx)
84471 2 S+ 0:00.00 grep -i nginx: master
NOTE: If you have difficulties starting a second NGINX instance and see an error reporting the port is already in use, please verify that the WebConfigurator redirect is disabled. Navgate to System > Advanced > Admin Access and verify WebGUI redirect, Disable webConfigurator redirect
is selected.
Set nginx-wpad to autorun
We’ll configure a cron job to start the second nginx instance on boot, or restart it should it crash for any reason. Create a file to check if our nginx module is already running, and if not, start it by entering
ee /root/check_proxy_wpad.sh
Paste the following into this file you just created.
#!/bin/sh
pgrep -F /var/run/nginx-wpad.pid
if [ $? == 0 ];then
/bin/echo running
else
/usr/local/sbin/nginx -c /usr/local/etc/nginx/nginx-wpad.conf
fi
Set this script as executable
chmod +x /root/check_proxy_wpad.sh
Create a cron job to autorun
You could do this via the command line but the Cron tool available under System > Packages makes life a lot easier. Install it if you haven’t already and navigate to Services > Cron
Create a job as
- Minutes = *
- Hour = *
- Day of the Month = *
- Day of the Week = *
- User = root
- Command = /root/check_proxy_wpad.sh
Your job should look like this
Click Save
Its worth rebooting and verifying the second nginx instance starts automatically using the command shell and the command ps aux | grep -i "nginx: master"
command
Serve the WPAD files
There are two methods of serving WPAD files, DNS and DHCP. You can use either one or both together, in my personal testing I’ve found DNS to be the more reliable though I have heard other users who found DHCP to be more reliable.
It is possible to run both systems concurrently, if an address isn’t received via DHCP the system will fall back to DNS next. As of when I wrote the original guide in 2014, Firefox didn’t support WPAD via DHCP which was a known bug for quite a number of years. I’ll detail both methods but suggest you try both and see which one works best for you and the clients you want to serve.
Configure DNS
I’ve upgraded from the DNS Forwarder to DNS Resolver which this guide will focus on.
Navigate to Service > DNS Resolver
- DNS Resolver = [x]
- Listen port: 53
- Network Interfaces = highlight your intended interfaces
- Outgoing Network Interfaces = highlight the interfaces DNS resolution should occur on.
- System Domain Local Zone Type = Transparent
- DNSSEC = Preferably ticked but I’ve had a few issues so I leave it unticked.
- DNS Query Forwarding = [x]
- DHCP Registration = [x]
- Static DHCP = [x]
We will now create a wpad host override for ‘wpad’.
- Click + next to Host Overrides
- Host = wpad
- Domain = domain name you have set in the pfSense general settings,
local.lan
is mine. - IP = your LAN IP
- Description = WPAD or something descriptive
- Save
Test the WPAD host override works with nslookup
. This should report the host override IP address back to you. (192.168.20.1 is my DNS server)
$ nslookup wpad.local.lan
Server: 192.168.20.1
Address: 192.168.20.1#53
Name: wpad.local.lan
Address: 192.168.1.1
Its also worth testing to make sure you can download the wpad.dat file (and other two) from within your browser. Open your browser and navigate to http://wpad.local.lan/wpad.dat
. You can use curl
too if you prefer, here curl is returning my wpad.da file correctly.
$ curl wpad.local.lan/wpad.da
function FindProxyForURL(url,host)
{
// If the requested website is hosted within the internal networks, send direct.
if (isPlainHostName(host) ||
shExpMatch(host, "*.local.lan") ||
isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
isInNet(dnsResolve(host), "172.16.0.0", "255.240.0.0") ||
isInNet(dnsResolve(host), "192.168.0.0", "255.255.0.0") ||
isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
return "DIRECT";
//Default rule: all other traffic use below proxy
return "PROXY 192.168.1.1:3128";
}
Configure DHCP
In your DHCP server, add the following three options in the ‘Additional BOOTP/DHCP section’.
For the wpad.dat file
number: 252
type: string
value: "http://wpad.local.lan/wpad.dat"
For the wpad.da file
number: 252
type: string
value: "http://wpad.local.lan/wpad.da"
and finally for the proxy.pac file
number: 252
type: string
value: "http://wpad.local.lan/proxy.pac"
Verification
I ran Wireshark at this stage to make sure everything was working as expected. Here’s a screen grab showing
You can see top to bottom where I’ve highlighted key pieces of info, i.e
- DNS lookup of wpad.local.lan
- GET request wpad.dat file
- HTTP download reorting “200 OK” for the wpad.dat file
- Line based text data as ‘application/x-ns-proxy-autoconfig’
- Contents of wpad.dat
Client and browser configuration
Assuming you can download the WPAD configuration files without issue in your browser, this part should be easy. I’ll provide a few examples of my configurations for my Macbook and Firefox but other clients should be similar enough.
Mac Network Settings
Firefox Settings
Note its possible to tell Firefox to use system settings too but this is an alternative.
Windows / Internet Explorer
Iphone & Ipad
Auto discovery works in iOS too. Head into settings, click the info symbol besides your network connection, scroll down and select ‘Auto’. Don’t fill in any details, just leave them blank.
Firewall
Ensure you have a firewall rule that will permit port 80 traffic to allow the delivery of the wpad file from the server.
Possible next steps
Block surfing on port 80
Once you have the WPAD redirection working and all traffic is flowing via port 3128, you could create a firewall rule to prevent browsing via the usual HTTP port 80 ensuring all browsing is only done via your proxy.
NAT backup
To catch any PCs which aren’t configured with ‘auto configure’ in their settings, its possible to implement a port forward to forward any traffic for port 80 back to 3128.
References
Changelog
11 December 2017
Improved text formatting, fixed typos
17 November 2017
Added details re Disabling webConfigurator redirect.
2 September 2017
pgrep -F /var/run/nginx-wpad.pid use
updated proxy.pac file
pfsense
proxy cache hard disk 용량의 50% 지정
proxy 메오리 용량의 50% 지정
proxy guard 설정
My_BlackList
My_WhiteList
ssh 접속
pfsense
create proxy.pac
/usr/local/www/nginx/proxy.pac
shell 8번접속
ee /usr/local/www/nginx/proxy.pac
function FindProxyForURL (url, host) {
// our local URLs from the domains below example.com don’t need a proxy:
if (shExpMatch(url, “.YCis.co.kr/“)) {return “DIRECT”;}
if (shExpMatch(url, “.YCis.co.kr//*”)) {return “DIRECT”;}
// URLs within this network are accessed through
// port 3128 on proxy1.mydomain.local:
if (isInNet(myIpAddress(), “192.168.0.0′, “127.0.0.1”, “255.255.255.0”))
{return “PROXY 192.168.0.254:3128”;}
// All other requests go through port 8080 of proxy.example.com.
// should that fail to respond, go directly to the WWW:
return “PROXY 192.168.0.254:3128″; DIRECT”;}
ctrl+업키 엔터 저장
ln -s /usr/local/www/nginx/proxy.pac /usr/local/www/nginx/wpad.dat
ln -s /usr/local/www/nginx/proxy.pac /usr/local/www/nginx/wpad.da
ee /usr/local/etc/nginx/nginx-wpad.conf
worker_processes 1;
pid /var/run/nginx-wpad.pid;
events {
worker_connections 1024;
}
http {
include wpad-mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name wpad;
location / {
root /usr/local/www/nginx;
index index.html index.htm;
}
}
}
ee /usr/local/etc/nginx/wpad-mime.types
types {
text/html html htm shtml;
text/css css;
image/jpeg jpeg jpg;
text/plain txt;
application/x-ns-proxy-autoconfig pac;
application/x-ns-proxy-autoconfig dat;
application/x-ns-proxy-autoconfig da;
}
ps aux | grep -i “nginx: master”
ee /root/check_proxy_wpad.sh
! /bin/sh
/bin/ps ax | /usr/bin/grep /usr/local/etc/nginx/nginx-wpad.conf | /usr/bin/grep -v grep
if [ $? -eq 0 ] ; then
/bin/echo ok
else
/usr/local/sbin/nginx -c /usr/local/etc/nginx/nginx-wpad.conf
fi
chmod +x /root/check_proxy_wpad.sh
pfsense dhcp server 설정
252 txt http://wpad.YCis.co.kr/wpad.dat
252 txt http://wpad.YCis.co.kr/wpad.da
252 txt http://wpad.YCis.co.kr/proxy.pac
252 string “http://wpad.YCis.co.kr/wpad.dat”
252 string “http://wpad.YCis.co.kr/wpad.da”
252 string “http://wpad.YCis.co.kr/proxy.pac”
nginx: [emerg] bind() to 0.0.0.0:80 failed (48: Address already in use)
function FindProxyForURL(url,host)
{
if(isPlainHostName(host))
{
return “DIRECT”;
}
if(isInNet(host,”127.0.0.1″,”255.255.255.0″))
{
return “DIRECT”;
}
return “PROXY 192.168.0.254:3128”;
}
function FindProxyForURL(url,host)
{
// If the requested website is hosted within the internal networks, send direct.
if (isPlainHostName(host)
shExpMatch(host, “.YCis.co.kr/“)
isInNet(dnsResolve(host), “10.0.0.0”, “255.0.0.0”)
isInNet(dnsResolve(host), “172.16.0.0”, “255.240.0.0”)
isInNet(dnsResolve(host), “192.168.0.0”, “255.255.0.0”)
isInNet(dnsResolve(host), “127.0.0.0”, “255.255.255.0”))
return “DIRECT”;
if (isPlainHostName(url)
if (shExpMatch(url, “.YCis.co.kr/“)) {return “DIRECT”;}
if (shExpMatch(url, “.YCis.co.kr//*”)) {return “DIRECT”;}
if (isInNet(myIpAddress(), “192.168.0.0′, “127.0.0.1”, “255.255.255.0”))
{return “PROXY 192.168.0.254:3128”;}
//Default rule: all other traffic use below proxy
return “PROXY 192.168.0.254:3128”;
}
2번째 웹페이지 설정
cd /usr/local/www/nginx
cp /var/etc/nginx-webConfigurator.conf /usr/local/www/nginx/nginx-proxy-wpad.conf
ee /usr/local/www/nginx/nginx-proxy-wpad.conf
#
nginx-proxy-wpad-configuration file
#
configured for WPAD/PAC serving
#
set static document-root
server.document-root = “/usr/local/www/nginx/”
where to send error-messages to
server.errorlog = “/var/log/nginx-proxy-wpad.log”
mimetype mapping
mimetype.assign = (
“.dat” => “application/x-ns-proxy-autoconfig”,
“.da” => “application/x-ns-proxy-autoconfig”,
“.pac” => “application/x-ns-proxy-autoconfig”
)
bind to proxy address and port
server.bind = “192.168.0.254”
server.port = 80
unqiue pid to other lightppd instance
server.pid-file = “/var/run/nginx-proxy-wpad.pid”
테스트-2번째 페이지
/usr/local/sbin/lighttpd -f /usr/local/www/nginx/nginx-proxy-wpad.conf
nginx-webConfigurator.conf