Ethernet suggestions

[Michael Butash]

I remember seeing something like a 4 port switch pci card a few decades
ago, but they're simply not useful anymore when you can buy a 4 port gig
switch for 5 bucks from china now.

Any other 2-4 port card treats them as standalone nics each, and really
aren't meant to be bridged together on a workstation or server.  As Mac and
I stated, it's more for redundancy to one or more switches.

Bridging from physical to virtual interfaces for virtual machines is the
primary use case for any bridging at all in a server, and should best be
left at that outside of network appliances.

I don't see why there is any aversion to running a switch to do this vs. a
ring topology.

-mb


On Sat, Oct 5, 2019 at 1:44 PM Stephen Partington <cryptworks at gmail.com>
wrote:

> I'll have to look. But I think there are ethernet controllers with small
> switching fabric in them.
>
> That might be a level of scaling that would maybe work.
>
> On Sat, Oct 5, 2019, 9:15 AM Donald Mac McCarthy <mac at oscontext.com>
> wrote:
>
>> Kelly,
>>
>> Maybe I am missing something as to why this is a requirement. Is a ring
>> configuration using RSTP a requirement? If that is the case, I haven't an
>> answer that I think would help. I know RSTP allows for fast convergence of
>> failure, I just haven't come across a case where the benefit mattered vs
>> the complexity of scale. We tried to test RSTP when I was a cluster
>> administrator at a university, (802.1W must be better than 802.1D right?)
>> because a professor insisted that the performance of distributed operations
>> would be better. This was a 4 rack cluster of ~70 nodes. The performance
>> tanked. After a lot of trial and error, we settled on the architecture that
>> I am attaching a drawing of using STP.
>>
>> If redundancy and ease of operation is what you want - I would use
>> redundant switches and us Linux to create a bonded interface that is in an
>> active-passive state. You will have to use a non LACP bond (teaming) as
>> LACP does not work across switches. Your switch's backplane and uplinks
>> will be the only bottlenecks that would occur in the network. Most
>> enterprise switch manufactures build a backplane that can handle the
>> traffic that is possible to send through the all the ports combined at
>> theoretical max.
>>
>> 2 switches that have 2 or 4 port LACP bonds or if you use switches that
>> have proprietary stacking cables, use the stacking cable. Also have an LACP
>> to upstream switching as well.
>>
>> Hopefully the drawing attahed will help.
>>
>> I have run clusters of over 2500 nodes with a nearly identical
>> configuration. We used 4x 10Gb per node, 2 LACP bonds per node into 48 port
>> switches. Those switches had a 6x 40Gb uplinks that were split in LACP to 2
>> top of rack switches. Top of rack switches had 100Gb uplinks to core. At
>> the core were multiple internal networks as well as multiple wan
>> connections.
>>
>> My point in talking about the size and speed is not to brag (well, kinda
>> - don't we all like cool toys), but to point out that this architecture
>> will work with 1Gb switches and machines of 6 nodes all the way to
>> thousands of nodes with bigger uplinks. You can scale the switching as your
>> hardware changes and scales. The architecture remains the same.
>>
>> If you are only using 100 nodes, you have less complication. As for plug
>> and play like behavior, as long as you don't mac lock the switchports - the
>> switches wont care what you plug into them as long as the NICs are properly
>> configured.
>>
>> Hope this helps. If I have missed something - I hope someone else finds
>> this useful.
>>
>> Mac
>>
>> kelly stephenson wrote on 10/4/19 3:34 PM:
>>
>> Looking for some networking advice from the group.
>>
>> The system I have has several devices connected in a ring configuration
>> using one Ethernet port IN and one Ethernet port out.  The system uses RSTP
>> for loop free operation.  The idea is simplicity for installation, you just
>> unplug and plugin a new device in the ring plus you gain redundancy, if one
>> Ethernet cable breaks you still have another one.  This works but my client
>> has never had more then a half dozen devices on the network yet.
>> When I say devices just imagine very large machines.  The number of
>> devices could be as many as 100 in the ring or network.  Everything I've
>> researched on RSTP says over 8 devices and its not effective/efficient so
>> I'm researching other Ethernet failover/failsafe/redundant solutions.
>> So, the local network configuration needs to scale up to 100 devices,
>> have redundancy, and low latency for M2M control.  Any thoughts?
>>
>> Thanks
>> Kelly
>>
>>
>> ---------------------------------------------------
>> PLUG-discuss mailing list - PLUG-discuss at lists.phxlinux.org
>> To subscribe, unsubscribe, or to change your mail settings:https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>
>>
>> ---------------------------------------------------
>> PLUG-discuss mailing list - PLUG-discuss at lists.phxlinux.org
>> To subscribe, unsubscribe, or to change your mail settings:
>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss at lists.phxlinux.org
> To subscribe, unsubscribe, or to change your mail settings:
> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.phxlinux.org/pipermail/plug-discuss/attachments/20191005/93c94b22/attachment.html>