Re: PfSense + ubiquity

Top Page
This message is part of the following thread:
Mthe complete thread tree sorted by date
MMichael Butash via PLUG-discuss at <-
MMichael Butash via PLUG-discuss at ->
Attachments:
Message as email
+ (text/plain)
+ (text/html)
+ (text/plain)
Delete this message
Reply to this message
Author: Thomas Scott via PLUG-discuss
Date:  
To: Main PLUG discussion list
CC: Thomas Scott
Subject: Re: PfSense + ubiquity
Day job is for a certain ISP HQ in Atlanta that supplies internet for a lot
of the valley - I work in Network Operations first in Phoenix and now in
Atlanta, and was surprised to see so much of what I talk about everyday in
PLUG!

CLink trying to play FTTN as FTTH, nothing new there. I live in a
neighborhood outside of Atlanta that had some AT&T brownfield development
for FTTH, and I've had no regrets (300 up 300 down!) Cox is moving towards
"10G" with DOCSIS 4.0 and they are getting fiber closer to the home with
their node splits. If you find that you all off a sudden have an extra hop
in your path, that might be the seen you've been on one of those nodes that
have been lit and split. The amount of bandwidth going up and down will go
up dramatically.

@Michael - yeah I don't think the caps are going anywhere, the industry as
a whole (driven by big red) has moved that direction, but I think you'll
see speeds and caps rise as N+0 goes to full duplex DOCSIS. I do know
they've been relaxed with the COVID-19 FCC initiatives, but how long that
lasts, I'm not sure.

@Mac - the cox supplied modems are almost all going to "Panoramic Wi-Fi"
and the number of holes found in DOCSIS devices is... disturbing to say the
least. It was designed to be operated on a shared RF medium, and like other
"trusting" protocols (i.e. BGP) has a lot of issues. The more virtualized
it becomes, I think we'll see more of those go away - the smaller the
broadcast domains, and the smaller the first upstream router, the better
those will be able to be maintained and automated. Looking at the road
maps, it will be interesting what comes next.

- Thomas Scott | <>


On Wed, May 6, 2020 at 3:54 PM Michael Butash via PLUG-discuss <
> wrote:

> Oddly enough, the model number of your router stuck in my head, the
> C3000Z, and I realized I used the same thing, but for my 150mbps dsl
> modem. You sure you have actual gig fiber? They tend to misrepresent
> their actual products in sales. Ask me how I know.
>
> <tldr>
>
> I say this because I called CL before going to them, and asked if I could
> get fiber in the network. They said yes. Hmm, I knew damn well they did
> not, as no one wants to build fiber into old peoria neighborhoods such as
> mine. After some conversation and calling him out, he explained that "oh,
> it's a gigabit network", just not fiber to your house. I could get
> dual-band DSL, which means 75mbps x2, for a total of 150mbps, delivered by
> a gigabit network! I sort of facepalmed, but ordered it anyways as it was
> significantly more than I had with cox (80mbps at the time I think),
> significantly cheaper, and no bandwidth cap.
>
> If there is anything other than fiber directly in your modem, I'd call
> bullocks, but FTTH is a myth to me.
>
> Crappier service, but I'll take the (usually) cheap and fast. It is most
> certainly not gigabit fiber to my house, even though that's what they tried
> to sell me I was getting. Only new house/community builds get fiber, and
> if even that. Cox did the same to compete with Google fiber, and as soon
> as Google Fiber died, so did Cox ever mentioning fiber again. Truth is Cox
> doesn't need it, shielded coax can deliver soon 10g over it just fine with
> new modulation schemas and docsis improvements. Centurylink's 100 year old
> 2-8 wire infrastructure cannot, all they can do is build new with fiber,
> but they probably won't being decrepit.
>
> I hear friends of mine mention they have fiber, and wonder just if they
> really do. This is why Google Fiber folded, it was unrealistic unless a
> net-new community build. Google fiber retrofits were a disaster
> <https://gizmodo.com/when-google-fiber-abandons-your-city-as-a-failed-experi-1833244198>
> .
>
> Fun-fact: Oddly enough the guy that built Google Fiber, Milo Medin, is
> the same guy that started @Home Networks back in late 90's for Cable Modem
> services, and pioneered current industry standards in use today globally to
> deliver cable internet. The last-mile regional MSO providers snuffed
> him/company back then, took it over themselves, and then they snuffed him
> out again as he tried the same incursion with Google Fiber, and realized it
> just cost too damn much to compete. Cable Monopolies, flawless victory.
>
> Next I expect he'll team up with Elon or Bezos to try again via
> terrestrial.
>
> -mb
>
>
> On Wed, May 6, 2020 at 10:32 AM Michael Butash <> wrote:
>
>> I tend to find the CL network a bit wonky, having moved to DSL from Cox
>> (damn bandwidth caps). I find the general performance is worse than cox,
>> where I suspect they simply don't manage the bandwidth and are far too
>> oversubscribed as it feels like the internet buffers at times, literally.
>> Cox would occasionally get that way too, and it was easy to see in an
>> ongoing MTR when their peering in LA would get slammed and latency would
>> jump (not to mention I know the guys that manage that bandwidth, telling
>> them often got it fixed). Oddly Using MTR with CL, they filter icmp/udp
>> specifically that seems to hide responses to track well. Go figure, truth
>> hurts, so hide it.
>>
>> Having worked for service providers numerous times over the years,
>> working in and building them, routers are always an issue in a metro city
>> or even interstate networks. No two platforms are ever the same, whether
>> buying all Cisco, Juniper, Nokia, or any combo of all and more, which as
>> you said, many do. Hardest part is usually capacity planning, particularly
>> with something like covid, every isp took a kick in the groin at the same
>> time to augment their networks, suddenly by some magnitude, when everyone
>> else in the world is doing the same. Slowness in networking can often be
>> attributed to those not having enough capacity, though they'll never admit
>> it.
>>
>> I'm on the 150mbps dsl, and a speed test can provide that for sure, but
>> general usage, which I use a lot of tabs and apps, tends to bring things to
>> a crawl often. I'd even go back to cox if they got rid of the bandwidth
>> cap. CL might as well be government, and they're run by unions, so nothing
>> happens fast, including capacity augments.
>>
>> Re: mac limits, having been around Cox both as a customer and network
>> engineer working there early 2000's, the mac security was more about
>> limiting the amount of hosts behind a modem that could be allowed to a
>> single mac and IP address. Back Circa 1998 I had my first Cox modem, and
>> there were no routers, you just got yourself a phat 10baset switch from
>> computer city and connected up your family on public ip addresses, each
>> with their own mac and ip's. With no limits or filters that led to
>> security issues (hey, I see my neighbor's c drive shared!), Cox and others
>> then pushed people to then buy a router, which by then around 2002, you
>> could buy a cheap wrt54g linksys. The advent of docsis also allowed to
>> both filter and restrict the macs by default, also let them reduce to now
>> 1:1 IP to User ratio, which was good for ip management, the abuse
>> departments, and fbi warrants from legal. You used to be able to buy
>> another ip, they'd push a new docsis config with mac-alowed=2, but not
>> anymore.
>>
>> Same reasons they're just building in the router functions now, it
>> ensures they can offer some basic customer security, plus lets them run
>> whatever spyware in their embedded router os they want. Better off buying
>> your own standalone modem and router combo, one you ideally trust.
>>
>> -mb
>>
>>
>> On Tue, May 5, 2020 at 10:07 PM Donald Mac McCarthy via PLUG-discuss <
>> > wrote:
>>
>>> Putting a CL modem into a bridge mode where it only handles the PPPoE
>>> connection is simply checking a radial select button and hitting apply. If
>>> your firewall supports PPoE, even better, as you no longer need their Modem
>>> and router in the mix. But, that is just my experience, and it is limited.
>>> I have a CL fiber to the door drop, and they gave me a Zyxel C3000Z device
>>> for connection. I promptly ripped it out and allowed pfSense to maintain
>>> the PPPoE connection. I had to call support for packet loss one time, and
>>> they refused to help me. So goes it rolling your own I guess. Turns out a
>>> day later we had a several hour outage due to one of the multiplexing cards
>>> used to distribute the 40Gb/s core fiber to the GPON devices failed. Seems
>>> like that was a likely culprit for some of the packet loss the previous day.
>>>
>>> Having just gotten off a call in which the Senior Director of Security
>>> Architecture and Engineering (a friend of mine from Atlanta) for Cox was a
>>> participant, before he hung up I asked him about the typical Cox supplied
>>> modems. Very, very few of them are purely bridge devices - especially with
>>> the push to "Panoramic WiFi". A member of CentryLink who was also on the
>>> call (ISP InfoSec sharing/working group) mentioned how painful it was to
>>> support the number of company issued modems/gateway/router models there are
>>> for different infrastructure and connections - let alone ones that
>>> customers buy and bring to the party. BTW, the MAC address thing is because
>>> they do actually use a MAC locking like feature for security. Apparently it
>>> is bad for the network if you just go plug your modem in at several houses
>>> in the neighborhood due to the way DOCSIS works. I still have to dig into
>>> that and ask some more questions on that one.
>>>
>>> There was a collective groan among the engineers when another ISP spoke
>>> up about the number of critical flaws they find in their DOCIS devices each
>>> year.
>>>
>>> With the amount of consolidation which has happened in the past 20 years
>>> in the broadband market, the landscape is riddled with legacy bits and
>>> pieces of this provider and that provider somehow being coerced into
>>> working together to accomplish passing traffic. One of the ISPs mentioned
>>> they had no less than 350 different models of core switching equipment made
>>> by more than a dozen manufacturers in their network. They have a team of 40
>>> (really 5 teams of 8) that simply monitor and ensure that the OSPF
>>> functions properly among the various models and brands to make sure that
>>> the network properly heals/manages congestion.
>>>
>>> Anyway, just throwing it out so that people can see and understand the
>>> picture at a higher level. The final comment on the call was from an
>>> engineer at a midwestern rural provider and one that I am sure many of us
>>> can relate to. She said she spends all day pulling her hair out trying to
>>> keep the network functioning at the highest of levels. The first words out
>>> of her kids' mouths when she gets home are "Mom, the WiFi seems slow today."
>>>
>>> I talked with Alexander this afternoon, and it looks like he has a
>>> functioning network again. The APs were reluctant to give up their old
>>> configuration, so a factory reset and new DHCP leases seem to have done the
>>> trick.
>>>
>>> Hopefully this sheds a bit of light on something for a few people.
>>>
>>> Mac
>>>
>>>
>>> Michael Butash via PLUG-discuss wrote on 5/4/20 4:59 PM:
>>>
>>> Ideally when you plug into a cable modem, it comes up, and passes your
>>> ethernet to the cmts in a bridge, lets one mac address dhcp/arp, and things
>>> work. It learns that one ip/mac, and disallows any other mac. No
>>> security, nat, nothing, just real dumb dhcp + default routing with a public
>>> ip. Routers/firewalls try to NAT you, thus double NAT if using a router
>>> behind it.
>>>
>>> CL sells you a dsl modem/router that does your local security whether
>>> you want it or not, full router/nat/firewall, and probably spyware. Making
>>> it a modem is possible, but takes work, and your firewall has to support
>>> PPPoE (not all can/do). Last time I touched a combo Cox router/modem, I
>>> didn't see any way to do so. I told them to buy a real modem, and that
>>> worked with their belkin/cisco/linksys/netgear they had.
>>>
>>> If your "modem" mentions wifi, it's a router/firewall, not a modem. Not
>>> all are clear about this, as they dumb it down for consumers, but an
>>> important point.
>>>
>>> -mb
>>>
>>>
>>> On Mon, May 4, 2020 at 1:53 PM Stephen Partington via PLUG-discuss <
>>> > wrote:
>>>
>>>> I Owned a Nighthawk Router/Modem combo, The way that Netgear handled
>>>> that is that the modem was hard-wired to a bridge on the router side. and
>>>> technically you could see it as a separate device in the router configs if
>>>> you rooted around enough. but the modem side was just a modem.
>>>>
>>>> On Mon, May 4, 2020 at 11:03 AM Michael Butash via PLUG-discuss <
>>>> > wrote:
>>>>
>>>>> Cox modems *are* bridges first and foremost typically, unless you get
>>>>> a bundled router/modem, which is only what CenturyLink sells. If you got a
>>>>> "router/modem" combo, just buy a modem-only device for a dumb bridge and
>>>>> simple ethernet for a public ip. I recommend staying with an arris cable
>>>>> modem, originally motorola, they basically developed cable modem docsis,
>>>>> and are always the best.
>>>>>
>>>>> I moved from Cox to CL when Cox started adding a usage cap, and that
>>>>> was new to me to get my Fortinet firewall online with CL and their DSL
>>>>> doing PPPOE. I've seen the router/cable modem combo boxes later, but never
>>>>> owned one as I always have my own router/firewall.
>>>>>
>>>>> -mb
>>>>>
>>>>>
>>>>> On Mon, May 4, 2020 at 8:36 AM Donald Mac McCarthy <>
>>>>> wrote:
>>>>>
>>>>>> Will Cox allow for a bridge/virtual bridge mode? Xfinity does, which
>>>>>> allows you to put in a firewall, and use the modem only as a gateway,
>>>>>> therefore preventing a double NAT situation. Never lived in a Cox area
>>>>>> before, and currently ride CL fiber.
>>>>>>
>>>>>> Mac
>>>>>>
>>>>>> Michael Butash via PLUG-discuss wrote on 5/3/20 2:00 PM:
>>>>>>
>>>>>> Cox modems will learn and allow only 1 mac at a time (unless business
>>>>>> is set to allow more, but not on residential). If switching out firewalls,
>>>>>> I 99% of time reboot the modem first and foremost.
>>>>>>
>>>>>> -mb
>>>>>>
>>>>>> On Sun, May 3, 2020 at 12:08 PM Snyder, Alexander J via PLUG-discuss <
>>>>>> > wrote:
>>>>>>
>>>>>>> I got it working.
>>>>>>>
>>>>>>> I assigned the SFP+ port as my LAN and assigned it the 10.x.x.x/16
>>>>>>> network. Then I had to call COX and list the WAN Mac address with them.
>>>>>>> Upon doing so I was able to reach external sites, and all downstream
>>>>>>> devices started coming alive!
>>>>>>>
>>>>>>> Thanks for all the suggestions and help!
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Alexander
>>>>>>>
>>>>>>> Sent from my Galaxy S10+
>>>>>>>
>>>>>>> On Sun, May 3, 2020, 03:14 Herminio Hernandez, Jr. via PLUG-discuss <
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Can you login to the FW via the LAN interface? Can you ping the
>>>>>>>> FW LAN interface? Check the routing and NAT policy on the FW. All outbound
>>>>>>>> traffic should NAT to the FW WAN interface and there should be a default (
>>>>>>>> 0.0.0.0/0) route to the internet.
>>>>>>>>
>>>>>>>> On Sat, May 2, 2020 at 7:27 PM Seabass via PLUG-discuss <
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> I'm with Mac, I think it is not the firewall, but if you have the
>>>>>>>>> ability to plug it into a display with a keyboard, you can use that for
>>>>>>>>> configuration and modify a different device at the same time.
>>>>>>>>>
>>>>>>>>> Makes it easier to troubleshoot by giving you the ability to
>>>>>>>>> configure your pfSense ports at the same time.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Message: 2
>>>>>>>>> Date: Sat, 2 May 2020 09:04:35 -0700
>>>>>>>>> From: Donald Mac McCarthy <>
>>>>>>>>> To: "Snyder, Alexander J via PLUG-discuss"
>>>>>>>>> <>
>>>>>>>>> Subject: Re: pfSense + Ubiquity
>>>>>>>>> Message-ID: <>
>>>>>>>>> Content-Type: text/plain; charset="utf-8"
>>>>>>>>>
>>>>>>>>> I can help - but I am unavailable to do so until tomorrow.
>>>>>>>>>
>>>>>>>>> Make sure there are not any thing other than default VLANs on the
>>>>>>>>> interfaces to start with. Ubiquiti is famous for not havinght eSFP+
>>>>>>>>> ports active in the default configuration, and I believe the
>>>>>>>>> switch has
>>>>>>>>> all the ports to shutdown on default config as well.
>>>>>>>>>
>>>>>>>>> I think it is the switch not passing traffic through - no the
>>>>>>>>> firewall.
>>>>>>>>>
>>>>>>>>> Mac
>>>>>>>>> Snyder, Alexander J via PLUG-discuss wrote on 5/2/20 8:53 AM:
>>>>>>>>> > Does anyone out there have experience with pfSence and Ubiquity
>>>>>>>>> switches?
>>>>>>>>> >
>>>>>>>>> > I have zero with either but that didn't stop me from buying both
>>>>>>>>> ....
>>>>>>>>> > how hard could it be?! LOL.
>>>>>>>>> >
>>>>>>>>> > I bought a Negate XG-1537-1U. I bought a Unifi Pro 24 PoE switch.
>>>>>>>>> >
>>>>>>>>> > I can configure the FW immediately after
>>>>>>>>> > firstboot/restore-default-configs, but only if i set the LAN
>>>>>>>>> interface
>>>>>>>>> > to be the cable that goes directly to my laptop. That's great,
>>>>>>>>> but
>>>>>>>>> > that does shit for the downstream switch.
>>>>>>>>> >
>>>>>>>>> > I have a 10GB SFP+ Port that I want to configure as the
>>>>>>>>> downstream
>>>>>>>>> > port to ubiquity, but any configuration other than mentioned
>>>>>>>>> above
>>>>>>>>> > fails .... and I'm now on my 12th "Reset To Factory Defaults"
>>>>>>>>> ... any
>>>>>>>>> > help on this would be greatly appreciated!
>>>>>>>>> >
>>>>>>>>> > Thanks,
>>>>>>>>> > Alexander
>>>>>>>>> >
>>>>>>>>> > Sent from my Galaxy S10+
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > ---------------------------------------------------
>>>>>>>>> > PLUG-discuss mailing list -
>>>>>>>>> > To subscribe, unsubscribe, or to change your mail settings:
>>>>>>>>> > https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Donald "Mac" McCarthy
>>>>>>>>> Director, Field Operations
>>>>>>>>> Open Source Context
>>>>>>>>> +1.602.584.4445
>>>>>>>>>
>>>>>>>>> https://oscontext.com
>>>>>>>>> -------------- next part --------------
>>>>>>>>> An HTML attachment was scrubbed...
>>>>>>>>> URL: <
>>>>>>>>> http://lists.phxlinux.org/pipermail/plug-discuss/attachments/20200502/aeab14b4/attachment-0001.html
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>> ------------------------------
>>>>>>>>>
>>>>>>>>> Subject: Digest Footer
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> PLUG-discuss mailing list -
>>>>>>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>>>>>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------
>>>>>>>>>
>>>>>>>>> End of PLUG-discuss Digest, Vol 179, Issue 2
>>>>>>>>> ********************************************
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------
>>>>>>>>> PLUG-discuss mailing list -
>>>>>>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>>>>>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>>>>>>
>>>>>>>> ---------------------------------------------------
>>>>>>>> PLUG-discuss mailing list -
>>>>>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>>>>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>>>>>
>>>>>>> ---------------------------------------------------
>>>>>>> PLUG-discuss mailing list -
>>>>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>>>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------
>>>>>> PLUG-discuss mailing list -
>>>>>> To subscribe, unsubscribe, or to change your mail settings:https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Donald "Mac" McCarthy
>>>>>> Director, Field Operations
>>>>>> Open Source Context
>>>>>> +1.602.584.4445
>>>>>>
>>>>>> https://oscontext.com
>>>>>>
>>>>> ---------------------------------------------------
>>>>> PLUG-discuss mailing list -
>>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>>
>>>>
>>>>
>>>> --
>>>> A mouse trap, placed on top of your alarm clock, will prevent you from
>>>> rolling over and going back to sleep after you hit the snooze button.
>>>>
>>>> Stephen
>>>>
>>>> ---------------------------------------------------
>>>> PLUG-discuss mailing list -
>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>
>>>
>>>
>>> ---------------------------------------------------
>>> PLUG-discuss mailing list -
>>> To subscribe, unsubscribe, or to change your mail settings:https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>
>>>
>>> --
>>> Donald "Mac" McCarthy
>>> Director, Field Operations
>>> Open Source Context
>>> +1.602.584.4445
>>>
>>> https://oscontext.com
>>> ---------------------------------------------------
>>> PLUG-discuss mailing list -
>>> To subscribe, unsubscribe, or to change your mail settings:
>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>
>> ---------------------------------------------------
> PLUG-discuss mailing list -
> To subscribe, unsubscribe, or to change your mail settings:
> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
---------------------------------------------------
PLUG-discuss mailing list -
To subscribe, unsubscribe, or to change your mail settings:
https://lists.phxlinux.org/mailman/listinfo/plug-discuss
This message was posted to the following mailing lists:
Plug Discuss
Mailing List Info | Nearby Messages		<-Re: OT: How to make this simple html menu work?Re: PfSense + ubiquity->
Some Mailing List Archive administrated by UnconfiguredLurker (version 2.3)