<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">One thing I tell clients - if you need
more than a gig, get a 10gbe interface. That comes with it's own
challenges too, see if you can get it to use all of that 10gbe...<br>
<br>
The issue you face with using a 802.3ad bond is the flow-hashing.
You're using a l2 policy, which I presume to mean "mac source/dst
flow-hashing". Basically, if all your communications is outside
the subnet, you're hashing to one mac, your default gateway, that
doesn't work well for distributing traffic. Use a L3-based
policy, including source/dst ports in tuples, that is what makes
switches effective.<br>
<br>
Sadly, you also have about the worst server switch ever created,
namely a legacy cat4k. Those if you look into the architecture,
they're 6gbe backplane, spread among 8 port groups (6x8=48 ports),
that every 1 gbe ports share 1x 1gbe asic interconnect to the
"fabric" (if you can call it that in 2015). At least get
something modern with big buffers on 1gbe ports, like an Arista
7048T, or if you must stay cisco, nexus 3k/5k and up.<br>
<br>
Those 4k's suck because for the oversubscription, make sure your
two server ports are spread between two asic groups if you want
max performance. Putting 8 hosts trying to talk a gig each will
just slam the asic into 8:1 oversubscription, they'll all get some
fraction of 1gbe shared per 8 ports (iperf this if you don't
believe me, I have). Also, same as your server, make sure you're
using "etherchannel load-balance src-dst-mixed-ip-port" for the
most entropy for distribution of flows at a L3 level using ip,
port, src/dst as tuples for distribution among your downstream
(and upstream) paths.<br>
<br>
Your network engineer should do that anyways, if not, spank him to
buy something outside a ccna-level book or look up the command.<br>
<br>
If you have one source, to one dest, or large "elephant herding"
flows like single filer connections, flow-distribution does little
to nothing to help. Goto first paragraph, get a 10gbe nic. :)<br>
<br>
-mb<br>
<br>
<br>
On 04/22/2015 02:42 PM, Stephen Partington wrote:<br>
</div>
<blockquote
cite="mid:CACS_G9z9V-Sdat7k3G=ACjMoDyVXtVdQ+dW-5NEKFCJkN=OftQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_default" style="font-family:trebuchet
ms,sans-serif">I am working on a ProxmoxVE cluster i have set
up.<br>
<br>
I am needing a bit better network performance as i am also
running CEPH for<br>
the stoage layer<br>
<br>
This is what i have for network configuration is the
following. it seems to<br>
be working. the nodes i have configured appear to be running
with better<br>
throughput.<br>
<br>
root@computername:~# cat /etc/network/interfaces<br>
# network interface settings<br>
auto lo<br>
iface lo inet loopback<br>
<br>
iface eth0 inet manual<br>
<br>
iface eth1 inet manual<br>
<br>
auto bond0<br>
iface bond0 inet static<br>
address 10.40.216.235<br>
netmask 255.255.255.0<br>
slaves eth0 eth1<br>
bond_miimon 100<br>
bond_mode 802.3ad<br>
bond_xmit_hash_policy layer2<br>
<br>
auto vmbr0<br>
iface vmbr0 inet static<br>
address 10.40.216.235<br>
netmask 255.255.255.0<br>
gateway 10.40.216.1<br>
bridge_ports bond0<br>
bridge_stp off<br>
bridge_fd 0<br>
<br>
root@computername:~#<br>
<br>
The backend network these servers are connected to are a Cisco
Catalyst<br>
4705R loaded with WS-X4524-GB-RJ45V modules.<br>
<br>
All my research says to use a network bond running LCAP for
best<br>
compatability/performance with this hardware. it all seems to
be running,<br>
but it is kind of weird that Proxmox would want me to create
the bridge for<br>
the VMs to run on kind of makes sense just feels weird to run
a bond<br>
inside bridge.<br>
<br>
If anyone who has worked with proxmox has a better suggestion
please let me<br>
know.<br>
<br>
Thanks for your time.<br>
<br>
-- <br>
A mouse trap, placed on top of your alarm clock, will prevent
you from<br>
rolling over and going back to sleep after you hit the snooze
button.<br>
<br>
Stephen<br>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">---------------------------------------------------
PLUG-discuss mailing list - <a class="moz-txt-link-abbreviated" href="mailto:PLUG-discuss@lists.phxlinux.org">PLUG-discuss@lists.phxlinux.org</a>
To subscribe, unsubscribe, or to change your mail settings:
<a class="moz-txt-link-freetext" href="http://lists.phxlinux.org/mailman/listinfo/plug-discuss">http://lists.phxlinux.org/mailman/listinfo/plug-discuss</a></pre>
</blockquote>
<br>
</body>
</html>