Close
0%
0%

pfSense Router/Firewall Install

*on a Dell Dimension E521, lol!

Similar projects worth following
My wife's old Dell Dimension E521 tower circa 2007 has been collecting dust under my art desk. With around $60 worth of parts I intend to resurrect it as a replacement for my absolute garbage Linksys EA6350.

The main reasons for this project are:

  1. I'd like to implement VLANs for security/isolation
  2. I'd like to be able to VPN into the system remotely

I recently upgraded my current router/AP (Linksys EA6350) to the latest firmware, but despite having newly added VLAN support, it's so minimal and contrived that it feels like more of a pain in the ass than it should be. The documentation also doesn't make clear whether or not I can implement VPN on a VLAN or if it has to be on the default VLAN. In any case, it's annoying at best and I'd like to learn how to implement these things closer to how they would be done in a production/commericial/enterprise environment.

I'm hoping I can still repurpose the Linksys to be a standalone/bridged AP, via VLAN. Fingers crossed, but who know.

As a challenge I'm setting a $60 parts limit for the final implementation. If I spend more, the parts must be immediately reusable in nature (for example, I always need/prefer USB flash drives for storing local installs).

  • 1 × 1 priceless antique/relic Dell Dimension E521 AMD Athlon 62 X2 dual core, 2.2GHz; 2x 512MB DDR2 PC-4200 (533MHz) DIMM RAM; Plenty for pfSense
  • 1 × Intel I350-T4 Quad Port 4-Port PCI-E Gigabit Ethernet Server Adapter just over $40 used with shipping and taxes. Needs a full height adapter
  • 4 × 512MB DDR2 PC-6400 (800MHz) DIMM RAM
  • 1 × 4-Pin Dupont Header to USB
  • 2 × 32GB Sansdisk Cruzer USB 2.0 media One for the installer, one for the Boot/HD; $13.68 new, Amazon; Onboard USB is USB 2.0, I don't want to mount externally.

View all 6 components

  • Lots of changes

    sarandi03/28/2020 at 04:37 0 comments

    It's been awhile, and for good reason - but I won't get into that here.

    Shortly after my last log, I received the HDD adapter mounts and installed a 250GB HDD salvaged from my first (and only) MacBook Pro also circa 2007 (system board died, R.I.P. - now tempted to resurrect this in the name of Louis Rossmann) I got that mounted, connected, etc. and it only required a little bit of finessing to get the cables to play nice. They're still a little tighter than I'd like but it'll have to do for now.

    I also looked into the USB header connections and problem was clear: the header pin sleeves were loose. I firmly reseated the sleeves and the internal USB worked fine ever since.

    At least, till tonight, when I removed it altogether and reinstalled pfSense to the HDD. As with the first install I used ZFS. I first copied my old configs by exporting the xml backups via the webGUI, loaded them to a fat32 USB with a partition table as outlined here.

    Basically, you have two options: have the config available on USB at install time, or do it subsequently during any boot. Since I struggled (for reasons still unclear to me) to find the above linked documentation until after the install, I did the latter.

    The install-time config requires the config to be located at /conf/config.xml while the post-install config requires either /config.xml or /config.xml - thinking about it now, I guess that makes sense - but it tripped me up for one boot cycle until I reread the docs.

    In any case, I got the config restore to work within a few minutes and as if that wasn't enough I upgraded to 2.4.5 (via the webGUI). All in all the above took about 20 minutes to get back up and running.

    The stupidest thing about all of this is that almost everything described above was simply so that I could reclaim that 32GB USB (to use as a Live Multiboot Utility/OS tool).

    I also picked up two 20" monitors ($30 shipped!) and a basic but NIB gaming keyboard (for $7!) for my home office, so now I can dedicate the old KVM setup to this box for local VGA login.

    The last few things to try before I swap this thing out with my current router/AP:

    1. Partition the HDD - I think there are already partitions for OS and SWAP, but it would be nice to have some isolation for logs and other files. I'll probably do this with a live USB GParted but I'm tempted to do it via SSH. 
    2. Try to get remote VPN working - though testing this will likely be tricky due to the stay-in-place stuff. I'm thinking about doing this via mobile. May have to wait.
    3. Setup and test SSH This was quick and easy.
    4. See if I can get Jails working (with zfs) to run a Unifi controller, though I cringe at the MongoDB. My concern here is typically the Unifi controller should be on the same switch as the AP - but in my case it would be up one level, same as the router itself - but I've been thinking I might be able to assign an interface to that jail and connect it to the switch... I feel like I'm getting in over my head with this one. Right now I'm running said controller on the Windows 7 laptop that replaced my aforementioned dead MBP. I would love to be powering one less device if possible and the uptime overlap would make this an obvious choice. I've alternately considered installing FreeBSD and pfSense on top, plus jails/bhyve, but I'll try the above first.

  • Wrapping up a few loose ends

    sarandi03/15/2020 at 06:40 0 comments

    Last week I got a local (for now) VPN set up between the user subnet and the admin ui subnet. I believe that I could simply change the firewall rules but I'd rather keep things as secure and simple as possible. That said, I really should test that just to make sure I'm correctly understanding the configuration. 

    I also ordered a full height adapter bracket for the i350-T4, as I had to remove the included shorty for it to seat properly in the PCIe slot. This means that any movement of the attached ethernet cables may jar the card loose - not good. It should be here within a week or so. I returned the original PT card and received a refund. 

    I also ordered a 3.5 to 2.5 hdd adapter 2-pack to have some decent storage, and I'm looking for an El Cheapo or free monitor to mount on my rack with my currently in disuse kvm (though not rack mount). I have a couple local and ebay leads almost within budget, though at this point the scope has expanded a bit. I'm leaning towards being lenient with this build by skimping on other projects. 

    Finally, I've been considering the possibility that many of the issues that I had been attributing to the USB or Dupont to USB adapter may instead have been caused by the old NIC. I'll probably power everything down and try to boot again with the USB mounted internally.

    Once the machine is physically sound, the next phase is swapping the pfSense machine with the current router/AP and putting it in bridge mode exclusively as an AP. I'd eventually like to flash it and my older DIR-655 with OpenWRT and DD-WRT but until all this quarantine business is over, I can't risk the wireless being down. Ideally that would allow for full VLAN options across both repurposed devices, but I haven't been able to confirm support as even the flashing instructions are sketchy at best. That might have to be its own hackaday project. Till next time. 

  • DHCP on Subnet for testing

    sarandi03/01/2020 at 17:38 0 comments

    Since I already have a router in place, I needed a way to test pfSense's routing abilities without disturbing existing DHCP assignments or future DHCP client requests.

    I'm fairly new at this - so anyone reading - please let me know if there's a better/easier way to do this. I'll describe my current setup and methodology below.

    My Linksys router/AP allows admin to set a custom local IP address for management. By default, you get the standard 192.168.1.1/24 but it does allow up to a /16 CIDR subnet mask. Users, DHCP or manually assigned, are restricted to the 192.168 range, plus whatever the subnet mask allows.

    The most straightforward way I can think of to isolate my pfSense machine but still have internet seems to be putting it both its WAN and LANs on their own subnets. I've enabled DHCP on the (so far) single LAN port. This avoids messy DHCP IP range assignment and will eventually give me a clean way to test all the interfaces on the card.

    I'm then able to connect to the router/AP's subnet (and admin UI). If I connect to LAN via ethernet ( to usb adapter) that lets me access both the router/AP's admin UI and the pfSense admin UI.

    All of the above is working well.

    Next is to look into firewall stuff - I'm thinking that's blocking pfSense admin UI from wifi. It would be nice to administer both via wifi.

  • Finally back up and running

    sarandi02/29/2020 at 04:10 0 comments

    At the time of my last posting, I had been hitting this really weird kernel panic every time I'd connect the ethernet cable to the WAN interface. Sometimes it would happen immediately, sometimes within several seconds, sometimes not for 20 seconds. But it would happen, always upon WAN connection.

    So, I tried:

    1. Updating the BIOS
    2. Disabling ACPI at pfSense boot - No option to do this via BIOS. Even for such an old heap, the system board's firmware is pretty garbage. It has Cool'n'Quiet. Always disable C'n'Q.
    3. Disabling all unnecessary devices via BIOS (including onboard NIC and all floppy support)
    4. Adding more memory - this finally worked, perhaps because of the BIOS update. But I think I also had a bad stick in my junk pile. Now at 2 meager GB. I'm pretty certain this thing can be maxed out around 12GB (users have reported 3x4GB sticks work, but adding a fourth doesn't)
    5. Altering settings permanently via /boot/loader.conf.local:
      1. hint.acpi.0.disabled="1" (see #2 above)
      2. reviewed USB boot issues here - nothing really seemed to match.
      3. Tuning the network
        1. kern.ipc.nmbclusters="1000000" (this sounded exactly like what was happening)
        2. hw.em.fc_setting=0 (disabled flow control - for PT/em-based card)
    6. Changing the IP address and releasing the DHCP reservation (was connecting WAN via a subnetwork through my existing router)

    The new NIC arrived today, so I thought what the hell. Sure as shit, popped it in and it works straight out of the new old box. So far.

    I still need to configure the LAN but I have web admin configuration access once again and the thing seems to be pretty stable.

  • Used i350-T4 Ordered

    sarandi02/25/2020 at 05:42 0 comments

    20200222 - Saturday

    The i350-T4 is ordered - shipping status unclear.

    I was under the impression that the PT shouldn't need drivers - that FreeBSD should just run the card as expected; I'm starting to doubt that; despite pfSense recognizing the card, it seems to hang when receiving packets on WAN.

    20200223 - Sunday

    I backed up the 2.5" drive to my laptop

    20200224 - Monday

    I backed up the 2.5" drive to the old Dell 3.5" - but it's clear that something is wrong with the latter. Once I can back up both to a different drive I'll feel more comfortable wiping the 2.5" HDD for the potential pfSense install.

    I also found some good info on troubleshooting possible boot/NIC issues. I posted those on the TODOs log.

    Follower @weekleyj has pointed out some helpful info, specifically that it might be worth installing 2.3.x and upgrading from there.

    https://www.netgate.com/blog/pfsense-2-4-0-release-now-available.html#important-information

  • Debugging TODOs

    sarandi02/21/2020 at 17:06 0 comments

    1. [x] Try reseating both ends of the cable the Dupont connector and the USB itself - no improvement
    2. [x] Try changing USB header positions - no improvement
    3. [x] Update config and test - failed
    4. [x] Collect TODOs
    5. [x] Look into BIOS/Boot Issues as described here. - 20200227
      1. [x] Possibly update BIOS - 20200226 - no noticeable differences in performance or BIOS options
    6. [x] Look into NIC related troubleshooting/tuning as described here.  - 20200227
      1. [x] Possibly install FreeBSD drivers for the PT - 20200226 - I looked into this more but the drivers are being mapped to the interfaces with the expected identifiers (em0-3)
    7. [x] Optional: Figure out what happened to the USB boot drive - is it still useable? [tested 20200223; USB drive functions and boots; I believe the error I was seeing is related to the NIC; Updating TODOs below. ]
      If so,
      1. [ ] Test continuity in Dupont header to USB Female connector
      2. [ ] If there's enough clearance, try mounting DMM clips and observe while trying to force error
      3. [ ]  If the above fail, mount externally and return the connector (see below)
      4. [x] Look into the CSM settings as described here. This is essentially the same as the todo:
        1. pfSense's boot loader is still being wonky - preferring the DVD drive as the highest priority. I want to eliminate this as a sticking point because it adds minutes to the boot sequence - 20200227 - this seems to have mostly resolved itself by disabling several BIOS boot options.
    8. [ ] Prep hard drives for 2.5" HDD install (I'm ready to give up on USB boot)
      1. [x] Move files from 2.5" to 3.5" HDD [completed 20200224; 3.5" drive is behaving erratically, I need a different backup solution.]
      2. [ ] backup both 2.5" and 3.5" HDDs
      3. [ ] partition 2.5" HDD to isolate logs/data from OS
    9. [ ] Reinstall pfSense on 2.5" HDD? after more research, I'm not pursuing this idea.

  • FreeBSD boot sequence killed the USB drive

    sarandi02/21/2020 at 16:56 2 comments

    I forgot the USB drive at home so debugging will have to wait. Last night I:

    1. Tried resetting the config. Used auto-detect to configure WAN
    2. As soon as LAN settings are saved, the system crashes
    3. Reinstalled pfSense, reconfigured as above but only WAN.
    4. Restarted, then the install wouldn't load. I got numerous errors including drive detatched and periph destroyed. Subsequent restarts just hang with the same errors.
      1. This might be
    5. So I tried loading the drive on my OSX machine and it was unrecognizable - still need to test this (below)

    I also:

    1. Tried upgrading RAM - no success. First 2x2GB, then 2x512MB. Neither worked.
    2. So I tried clearing/resetting CMOS to ensure the hardware was being recognized. No fix.

    In addition to the previous TODOs (some of which I have completed), here are some more:

    1. Test continuity in Dupont header to USB Female connector
    2. Prep hard drives for 2.5" drive install (I think I'm done with using USB to boot.)
      1. Move files to 3.5" drive and partition 2.5" to isolate logs/data from OS
      2. Look into the CSM settings as described here.
    3.  Collect TODOs into a (singular post to follow)

  • PCIe Fun, and the PT NIC has arrived

    sarandi02/19/2020 at 18:33 0 comments

    20200219 - Tuesday

    From now on I'm going to refer to the NICs by their most obvious identifier for two reasons:

    1. It's shorter
    2. I'm probably returning the original, but would like to easily be able to refer to either.

    I'll refer to the original as PT.

    I'll refer to the ideal model as i350.


    Snag 2: I mistook regular PCI slots for PCIe. Fortunately this isn't a huge issue. There are two PCIe slots - I'm assuming both are 1.0 or 1.1:

    1. PCIe x16
    2. PCIe x1

    In either case they shouldn't be a bottleneck for Gigabit, though I sure wish they were 2.0+.

    Plenty of bandwidth, I think? But things are still a bit unclear to me. Both are x4 (lane) cards, but there's a 20% loss associated with the above transfer rate, and I don't know if all four lanes are associated with that transfer rate to begin with. Assuming the card is only transferring relevant information and the loss is included, we get:
    So, you can see why I'm not too worried. It should be noted that while PCIe buses are capable of full duplex communication, each lane is unidirectional between its endpoints. Similarly full duplex Gigabit ethernet is 1Gb/s in each direction, each conductor being unidirectional.

    The x1 slot was open - but the x16 slot had a video card installed. Luckily there's another onboard VGA video card, so I removed the x16 card and installed the PT.

    Boot goes fine - the card is recognized and pfSense let me configure it -- but then the system goes into this weird loop. I'm thinking it's a config file issue, but here is what I'm thinking for workflow:

    1. I still need to go through the list I defined in a previous log - but a few more pressing things have come up
    2. Update config and test
    3. pfSense's boot loader is still being wonky - preferring the DVD drive as the highest priority. I want to eliminate this as a sticking point because it adds minutes to the boot sequence.
    4. If all above goes well, then I can see how stable the USB Flash install is.
    5. Return to previous log workflow.

    N.B. This has got me thinking -- I do have another old , larger tower with a more substantial motherboard. Now that I have re-familiarized myself with hardware of this era, I should inspect that board to see if it wouldn't be a better fit. I might be able to use either/both machines.

    Anyhow - moving on.

  • I Bungled the NIC Model

    sarandi02/17/2020 at 18:13 0 comments

    2020-02-17 - Monday

    I had a sneaking suspicion that I bought the wrong (and indeed I did - a first gen/PCIe 1.0a!) model. D'oh! At 2.5 GT/s it still is faster than Gigabit, but that's not the end of the lousy features. The good/funny thing is that it's same vintage/era as the Dell itself. But the card very well might not fit my case, as I for some stupid reason got the "full height" version instead of "low profile". So. We'll see.

    https://forums.serverbuilds.net/t/demystifying-intel-pro-1000-quad-port-nics/2401

    The feature set isn't as broad as subsequent models but it'll have to do until/unless I hit another roadblock, or case-block, as the case may be (too short). Not a huge deal, but not great. I was really looking forward to the onboard traffic-shaping and QoS features.

    Here's the official documentation page for the chipset (not the card itself):

    https://ark.intel.com/content/www/us/en/ark/products/20720/intel-82571eb-gigabit-ethernet-controller.html

    Curiously, this product line goes back to 2005Q3 - yikes.

    Card documentation:

    https://ark.intel.com/content/www/us/en/ark/products/50495/intel-pro-1000-pt-quad-port-low-profile-server-adapter.html

    Downloads and Software:

    Now we're finally getting somewhere. I'm not sure if pfSense/FreeBSD supports this model, so here's my safety net:

    https://ark.intel.com/content/www/us/en/ark/products/50495/intel-pro-1000-pt-quad-port-low-profile-server-adapter.html

    Complete (OS Independent) Driver Pack:

    https://downloadcenter.intel.com/download/22283?product=50495

    FreeBSD Driver:

    https://downloadcenter.intel.com/download/17509?product=50495


    The NIC That Should Have Been:

    https://www.ebay.com/itm/Low-Profile-Intel-I350-T4-Quad-Port-4-Port-PCI-E-Gigabit-Ethernet-Server-Adapter/324074299402


    More resources:

    1. https://www.cnet.com/products/intel-pro-1000-pt-quad-port-server-adapter-network-adapter-39y6138/
    2. Dubious and possibly not obviously helpful; Note - this document shows a quad port NIC but text lists it as a "Interfaces 2x RJ45 ": http://www.arp.com/medias/14309656.pdf

  • Power is the likely culprit in the read errors

    sarandi02/17/2020 at 17:36 0 comments

    2020-02-17 - Monday

    I did some light digging into the errors that I've been seeing. Here's a sample output:

    After a quick scan of the below two links, I'm guessing this is power-related and possibly caused by the Dupont-USB adapter.

    1. https://www.amazon.com/gp/customer-reviews/RD8IMZJEKMGSZ/ref=cm_cr_getr_d_rvw_ttl?ie=UTF8&ASIN=B000IV6S9S ( review of the exact model I bought )
    2. http://christopher-technicalmusings.blogspot.com/2012/03/freebsd-scsi-sense-errors-did-you-check.html ( power-related )


    A couple debugging/troubleshooting options:

    1. Try reseating both ends of the cable the dupont connector and the USB itself.
    2. Try changing USB header positions
    3. If there's enough clearance, try mounting DMM clips and observe while trying to force error 
    4. If the above fail, mount externally and return the POS.

View all 11 project logs

View all 8 instructions

Enjoy this project?

Share

Discussions

weekleyj wrote 02/22/2020 at 00:24 point

I just remembered something I had to do with my old Atom.  For older machines, it's frequently necessary to install pfSense 2.3x and upgrade to 2.4. 
Check the important information section of the release notes:
https://www.netgate.com/blog/pfsense-2-4-0-release-now-available.html

Older versions of pfSense can be downloaded here:

http://linorg.usp.br/pfsense/downloads/

  Are you sure? yes | no

sarandi wrote 02/25/2020 at 04:40 point

Thank you, @weekleyj , that is a great lead! I haven't had much time to work on this but have been wondering as much, having seen some unrelated but similar comments regarding FreeBSD (and derivatives) having issues with PCIe 2.0 ports on older-ish machines. I think my current issue is related to the NIC because I was able to get the web UI running, only to have it lock up after installing the PT quad NIC unit. It's either that or the BIOS/UEFI as mentioned in the link you posted.

The i350-T4 I mentioned earlier is on the way - and you're totally right about most R6/710s having that or a similar quad port NIC installed. I too have considered running this all on a VM someday, but maintaining a dedicated machine feels worth the piece of mind. I'd hate to have my home network down due to an unrelated system taking out the hardware.

  Are you sure? yes | no

weekleyj wrote 02/21/2020 at 23:47 point

Sarandi, OT, but most of the R6/710s I've seen on EBay already have a quad Broadcom NIC and come with at the very least 2-4 300 GB SAS disks installed.  Most are light on RAM, but it's cheap to max one out.  I've got one and it works really well with ESXi. 

I wonder how well it's work out if I got rid of my separate pfSense box and put it on a VM?

  Are you sure? yes | no

weekleyj wrote 02/21/2020 at 23:41 point

I've used pfSense on a 2 GB dual NIC Intel Atom board for years and that amount of RAM is overkill for home use, 1 GB is probably more than enough.   You really don't need the best of everything to use it unless you've got some very high traffic.    The Atom died, so my current machine is a cobbled together system from older parts I had lying around which consists of a Gigabyte motherboard with a dual core 3.2Ghz Intel CPU, 8GB RAM, Quad HP Gb NIC and mirrored 64 GB SSD's.   The quad NIC is the exception that I needed for a project.

  Are you sure? yes | no

RW wrote 02/21/2020 at 20:51 point

Following along. I tried looking at pfsense for a home router/firewall a few years back but ran into the "need the best of everything for everything" guys who insisted that I shouldn't try it on less than a 4 core Xeon with 32GB or something stupid. I also have a spare Dell Dementia, but probably won't go with that one, best it takes is a Dual P4. I'd probably use an A64 X2 system as I have a lower power X2 so could probably have it running under 100W total, rather than 100W for just the P4D. Don't think I'll be using gigabit, have a four hole intel/D-link jobbie for 100Mbit.

  Are you sure? yes | no

weekleyj wrote 02/20/2020 at 01:01 point

This quad port GB ethernet card works very well on pfSense
HP NC364T PCIe 4Pt Gigabit Server Adapter

I found it on Amazon for $48.

  Are you sure? yes | no

sarandi wrote 02/20/2020 at 02:18 point

Thank you, good to know! I have the PT already installed (got it for $20!) but I still have some config issues. I'm considering if I shouldn't replace with the i350-T4 as I'd like to throw it in a r6/710 someday and run a hypervisor. The PT can't handle that use case.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates