Tuesday, November 17, 2009

Multi WAN Load Balancing under Windows with PfSense

Multi WAN Load Balancing under Windows with PfSense

I recently built a home server machine for multimedia (movies, gaming, and picture storage) and networking purposes. The multimedia part is quite simple, you can hook up the server to your TV and watch movies or play games, and use the server as a centralized storage for file archiving. This can mostly be done under *NIX systems, but when it comes to HD movie playing and gaming, WINE may not run everything smoothly and you may not get the same results with full HD (1080p) under Linux. So, I decided to use Windows Vista as my host operating system in this server configuration.

The second purpose of the server is networking. I wanted to combine several DSL and Cable Net connections into one network and use the bandwidth pool (Quick example: Think that you have 3 DSL lines, each with 6 Mbit bandwidth pools. If you combine these 3 lines and apply load balancing with a round robin algorithm, you’ll get 18 Mbit bandwidth pool. So if you use segmented downloading software like Internet Download Manager, you should get around 1.6 mb/s download rate. ) The goal here is, very basically, I want to combine my internet connections to get faster speed.

This is called Multi WAN Load Balancing (will be called “load balancing” from now on). Load balancing can be done very easily with a script under BSD and Linux systems. (Doing it under BSD is easier than Linux by the way). But, my server will be on Windows Vista, and as you can guess, it is nearly impossible to this under a Windows desktop operating system. By the way Windows Server 2008 and 2003 has NLB (Network Load Balancing), which is NOT we are doing in this article. These two concepts are different. Also, forget about Internet Connection Sharing under Windows, it is different from our goal here.

So, back to our problem; we want to do load balancing with round robin algorithm under Windows Vista, and there is no software doing it. The easiest way of doing this is using a BSD system, but we’re under Windows. At this point, a friend of mine, who is a network systems admin (Alper YALCINER) gave me the idea of using PfSense, an open source customized distribution of FreeBSD tailored for use as a firewall and router, with a live CD. My buddy told me that all I had to do was running PfSense under Windows via Vmware or VirtualPC and redirecting all outgoing traffic to the local connection that PfSense uses. I chose to use Vmware since VirtualPC does not offer as much as its competitors.

At this point, what you need is; a Windows system, more than one WAN (Internet) connections, and a local area connection. In my configuration, I am connecting to this server from a home wireless network, I have 3 WAN connections, and one virtual connection between PfSense and Windows, created by Vmware Network Editor.

To make life easier, here is an illustration of what we will be doing:

LoadBalancer Overview

Step by step overall process:

1. Connect to WAN’s from Windows Vista

2. Boot up pfSense with Vmware

3. Clear Windows Vista’s routing table and add pfSense LAN IP as default gateway

4. Configure pfSense

5. Do a “software routing” on home wireless connection and pfSense LAN so that clients can access Internet

6. Connect to Windows Vista from client computers via wireless network

1. Connect to WAN’s from Windows Vista

This part is pretty simple. Connect to your WAN’s under Windows Vista and make sure EACH one of them has a different gateway. (i.e in my case, the first one is, second one is, and third one is

2. Boot up pfSense with Vmware

Vmware needs to be configured so that it gets the correct interfaces. Here’s my connection view:

Virtual Machine Settings

Figure 1: Virtual Machine Settings under Vmware

Here, set the CD/DVD to your pfSense ISO image. If you want, you can install pfSense instead of using it on live CD, which might be better as it’ll automatically save your configuration each time you reboot pfSense. It is very straight forward and I am skipping installing pfSense under Vmware (you can find more information on www.pfsense.org)

The important part is; networking adapter settings. Set the “Network Adapter” to “Custom, VMnet1″, set the “Network Adapter 2″ to “Custom, VMnet2″, set the “Network Adapter 3″ to “Custom, VMnet3″, and set the “Network Adapter 4″ to “Custom, VMnet4″. You can add more network adapters by clicking “Add” button. Note that Network Adapter 3 is missing in my configuration, don’t let the numbers confuse you.

Run the Virtual Network Editor as Administrator (right click -> Run as Administrator). Mine looks like this:

Virtual Network Editor Window

Figure 2: Virtual Network Editor window

The interfaces that I use in my configuration are:

VMNet1 -> pfSense LAN Interface,

VMNet2 -> WAN1,

VMNet4-> WAN2,


Note that “Local Area Connection 2″ which is bridged to VMNet1 is automatically created by Vmware as a host only private network.

Now, you can boot pfSense.

The menu will come up. First, select “Assign Interfaces” option. Enter your interfaces, the interfaces are in the same order as Virtual Network Editor, so if you followed my configuration, your VMNet1 will be em0 (or another prefix) and your em3 will be VMNet5. Then, select “Set LAN IP Address” and assign a manual address to VMNet1. I gave to LAN IP under pfSense, and enter 24 for the block number. After these two steps, restart pfSense and your configuration should look like this:

pfSense under VMware

Figure 3: PfSense after initial configuration

Note my IP addresses in the above picture. Of course yours will probably be different.

NOTE: If you don’t see any IP addresses on OPT1 or OPT2 interfaces, set your LAN IP Address, go to webConfigurator (type pfSense LAN IP address on Windows Vista), login with “admin” as username and “pfsense” as password, go to Interfaces->OPT1, check “Enable OPT1″ and select DHCP in the configuration menu and click Save. Do this for OPT2 and restart pfSense by selecting “Shell” in the console and typing “reboot”. You should now see IP addresses for OPT1 and OPT2 interfaces. You can also do this for WAN if you can’t get an IP from DHCP at boot up.

3. Clear Windows Vista’s routing table and add pfSense LAN IP as default gateway

Now that we have pfSense configured, we should configure Windows Vista’s routing table so that it redirects all the output traffic to pfSense LAN IP. Go to Start->All Programs->Accessories and right click Command Prompt and select “Run as Administrator”. Now type “route delete″ This will delete all the default gateways. Now, type “route add mask″ to command prompt; where “″ is the IP of the pfSense LAN interface. Here, we tell Windows to redirect all outgoing traffic to pfSense so that it does load balancing.

4. Configure pfSense

Now that we setup the routing table, we should tell pfSense to do load balancing.

a. Type pfSense Lan IP in Internet Explorer or Firefox under Windows Vista and login with admin:pfsense

b. Go to Services->Load Balancer

c. Go to “Pools” tab, and click on the “+” symbol to add a new pool

d. Give a name and description. Mine is “loadbalancer”

e. Select type: “gateway”

f. Select behavior: “Load Balancing”

g. For each WAN interface, select it’s gateway as Monitor IP, select the interface from “Interface Name” list and click “Add to Pool”. At the end it should look like this:

Load balancer setup

Figure 4: Load Balancer setup under pfSense

Click Save.

Now, we need to add some rules to pfSense firewall.

a. Go to Firewall->Aliases

b. For each WAN, add a new alias by clicking “+” button

c. Select Type: Host(s)

d. Enter the gateway IP of the WAN, and add a description. Click Save

Now, we need to add one more alias that incluses ALL gateway IP’s/

a. Add a new alias by clicking “+” button

b. Add name: “RouterIPs”, and description: “All router IPs”

c. Select Type:Host(s)

d. Enter the gateway IP of each WAN by clicking “+”

e. Click Save.

It should look like this (I had to black out some parts as I used MAC addresses):

Firewall Aliases

Figure 5: Firewall Aliases under pfSense

I also added HTTPS ports as you can’t load balance on HTTPS (You have to go through one WAN or it won’t work). You can also do the same.

Now, we are in the final step. Go to Firewall - Rules

a. Click on “LAN” tab

b. Click “+” to add a new rule

c. Rule Details:

Action: Pass

Disabled: Unchecked

Interface: LAN

Protocol: any

Source: “not” unchecked

Source Type: LAN Subnet

Destination: “not” checked

Destination Type: “Single host or alias”

Destination Address: “RouterIPs” (write down the alias name that includes ALL WAN gateways)

Log: unchedked

Gateway: “loadbalancer” (Name of the load balancer pool configuration)

Description: “Load balancer without failover”

d. Hit Save. You’re done!

Some notes: You should add more rules to reach separate gateways (i.e. you can’t ping a separate gateway because the traffic will go through the load balancer which chooses a random gateway by weighted round robin algorithm) Please take a look at http://doc.pfsense.org/index.php/MultiWanVersion1.2#Basic_Firewall_Rules to see the basic rules remaining. This website also explains how to setup BitTorrent clients, so make sure you read the firewall rules part of that document.

Now, if you go to www.whatismyip.com, you should see a different IP address each time you refresh your web browser. This means that load balancer is working!

5. Do a “software routing” on home wireless connection and pfSense LAN so that clients can access Internet

Now that we have load balancer working, we need to setup a wireless connection for client computers in our home/office environment. Go to “Network and Sharing Center”, click on “Set up a connection or network”, select “Set up a wireless ad hoc network”, select your home wireless connection interface, and give a name and password. Configure your wireless interface under Windows Vista accordingly: give an IP address (i.e., enter for the subnet mask, and don’t enter a gateway.

The “Local Area Network 2″ connection, which is what I use for pfSense LAN, is set to, subnet mask and as gateway IP. You can use and as your DNS. Now, we need to bridge these two connections so the users coming from wireless should have access to pfSense LAN subnet. I couldn’t use Internet Connection Sharing under Windows Vista as it wants to use and Vmware virtual adapter cannot work under ICS. So I downloaded NAT32 (www.nat32.com) and ran it. Here are the details for NAT32:

a. A configuration wizard should come up. Select pfSense LAN Interface (Local Area Connection 2 in my example) as Internet and your wireless interface as Private

NAT32 Screen

Figure 6: NAT32 Screen

NAT32 Settings

Figure 7: NAT32 Interface Selection

b. It should share the IP of LAN interface in the next step. No changes needed for LAN interface

Interface Configuration

Figure 8: NAT32 Interface 1 Configuration

c. The next window sets up the wireless interface. In my example, it uses for NAT32 and for Windows. Note the IP that it uses for NAT32, and click “Apply”. It should start software routing now.

Interface Configuration

Figure 9: NAT32 Interface 2 Configuration

6. Connect to your wireless network from client computers

Make sure you configure the IP addresses correctly. Add NAT32 IP as gateway (, give an unused IP like and type as subnet. You can enter and for DNS. If everything works, you should be able to go to www.whatismyip.com from a client computer and see a different WAN IP each time you refresh your browser.

Voila, you now have a load balancer server with a huge bandwidth pool. If you download a file via segmented download software (Internet Download Manager for example), you should get full bandwidth. Enjoy the internet!

Thanks to Alper YALCINER for his help and patience on the process of configuration :)

Bora Bilgin

UPDATE 1 : PfSense version 2.0 supports Multi Wan Traffic Shaping (for greater results in downloads, traffic shaping is necessary).



No comments: