Since u are using tesla I would suggest first that u put 4 gpus in each cluster and make sure you have at least a quad core each. If you are going with modeling applications than go with infiniband, but if you are running hpc/parallel computing stuff than u probably need only gigabit or 10 gb ethernet the reason being is that the gpus will need hours to compute. I hope this might help. Nadim Hoque Cell: 480-518-6235 Address: 6302 West Kent Drive Chandler, Arizona 85226 Sent via BlackBerry from T-Mobile -----Original Message----- From: Eric Shubert Date: Mon, 29 Jun 2009 18:29:11 To: Subject: Re: Clustering I'm looking at clustering together a handful of hosts, each running dual nvidia tesla cards. Modeling applications of some sort. I honestly don't know much more than that. Michael Butash wrote: > You're probably talking infiniband switching, infiniband hba's, > pci-e/htx interfaces, fiber channel disk arrays, etc. Linux seems to > support infiniband hba's reasonably well, and 10g 4x infiniband hba's > tend to be cheap these days on ebay. We're talking $100 used hba's for > the nodes, and ~$1200 for a 12 port Cisco/Topspin switch. I thought > about buying some to play with, as it ends up cheaper than IP or Fiber > channel technologies, yet can replace them all to some extents. > > IB is quite versatile, emulating fiber-channel, IP network, or raw > interrupt switching to a cpu and memory via different driver socket > interface api's. Cray always used a similar means to make theirs with > proprietary north bridge and software, but IB is more of a standard now, > enabling (relatively) cheap supercomputing on the fly with commodity > hardware. Well, hardware-wise at least... > > So yeah, what apps are you talking about utilizing it? > > -mb > > > On Mon, 2009-06-29 at 13:47 -0700, Matt Graham wrote: >> From: Eric Shubert >>> Has anyone here implemented any clusters? >> I've only set one up, but I maintain the ones that my predecessors >> set up. It's not rocket science. >> >>> Is any particular distro better or worse at clustering? >> Not really. Every distro has heartbeat/DRBD/LVS available. >> >>> Any pointers regarding clustering you'd like to share? >> Define the problem you're trying to solve more rigorously than just >> "clustering" first. Do you want flailover between 2 boxes? Do you >> want a frontend box with N service-providing boxes behind it? The >> answers to that greatly affect what you will end up doing, as does >> the question "What services is this cluster going to provide?" >> >> Basically, all I can do is handwave without answers to "what services?" >> and "how many machines?". >> > -- -Eric 'shubes' --------------------------------------------------- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --------------------------------------------------- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss