Clustering

[Michael Butash]

Exactly as Nadeem said, start local, and if you need to expand, look at
infiniband.  Buying IB gear directly from the vendor, supported,
warranted, etc, will get expensive quick.  You'll probably find
programming info fairly obscure, as most of this is very proprietary, as
supercomputing is a cash cow to certain vars.

Obscurity seems to make it cheap on the secondary market, as most app
dev's won't/don't take it to that level, and not many really need it.
Tesla's change things as they provide an extensible way to increase
generic crunching local, and can scale significantly with pcie bandwith
and hardware with enough slots.

For me IB was interesting as I was investigating upping my house to
gige, fiber-channel, and other things to geek out on.  IB actually
looked fairly promising as it's device stack allows for IP/FC
emululation layer, as well as it's own native socket stack where
parallel processing plays into.  Then I found out it was cheaper than
buying decent 4g fiber channel gear off ebay, and got even more
interesting.  Might provide some ideal scalability if you need to expand
the processing outside of one box eventually.

-mb 


On Tue, 2009-06-30 at 00:35 -0700, Stephen wrote:
> Are you looking at the 1u Tesla or the x58 personal supercomputer they
> are talking about...
> 
> I would look into RH mostly for compatability fo ahrdware as it is
> tested by them, and for most packages... but the distro is really
> dependant on your app.
> 
> Man they have really changed tesla, it used to be a bunch of gpu cards
> in a box chained to a header and then 16 of those can be tied together
> to a single machine.
> 
> but now you can buy a preconfigured clsuter
> 
> http://www.penguincomputing.com/products/linux/server/web_promotion/tesla_workgroup_cluster
> 
> man i stop following for just 1 year...
> 
> 
> On Mon, Jun 29, 2009 at 6:29 PM, Eric Shubert<ejs at shubes.net> wrote:
> > I'm looking at clustering together a handful of hosts, each running dual
> > nvidia tesla cards. Modeling applications of some sort. I honestly don't
> > know much more than that.
> >
> > Michael Butash wrote:
> >> You're probably talking infiniband switching, infiniband hba's,
> >> pci-e/htx interfaces, fiber channel disk arrays, etc.  Linux seems to
> >> support infiniband hba's reasonably well, and 10g 4x infiniband hba's
> >> tend to be cheap these days on ebay.  We're talking $100 used hba's for
> >> the nodes, and ~$1200 for a 12 port Cisco/Topspin switch.  I thought
> >> about buying some to play with, as it ends up cheaper than IP or Fiber
> >> channel technologies, yet can replace them all to some extents.
> >>
> >> IB is quite versatile, emulating fiber-channel, IP network, or raw
> >> interrupt switching to a cpu and memory via different driver socket
> >> interface api's.  Cray always used a similar means to make theirs with
> >> proprietary north bridge and software, but IB is more of a standard now,
> >> enabling (relatively) cheap supercomputing on the fly with commodity
> >> hardware.  Well, hardware-wise at least...
> >>
> >> So yeah, what apps are you talking about utilizing it?
> >>
> >> -mb
> >>
> >>
> >> On Mon, 2009-06-29 at 13:47 -0700, Matt Graham wrote:
> >>> From: Eric Shubert <ejs at shubes.net>
> >>>> Has anyone here implemented any clusters?
> >>> I've only set one up, but I maintain the ones that my predecessors
> >>> set up.  It's not rocket science.
> >>>
> >>>> Is any particular distro better or worse at clustering?
> >>> Not really.  Every distro has heartbeat/DRBD/LVS available.
> >>>
> >>>> Any pointers regarding clustering you'd like to share?
> >>> Define the problem you're trying to solve more rigorously than just
> >>> "clustering" first.  Do you want flailover between 2 boxes?  Do you
> >>> want a frontend box with N service-providing boxes behind it?  The
> >>> answers to that greatly affect what you will end up doing, as does
> >>> the question "What services is this cluster going to provide?"
> >>>
> >>> Basically, all I can do is handwave without answers to "what services?"
> >>> and "how many machines?".
> >>>
> >>
> >
> >
> > --
> > -Eric 'shubes'
> >
> > ---------------------------------------------------
> > PLUG-discuss mailing list - PLUG-discuss at lists.plug.phoenix.az.us
> > To subscribe, unsubscribe, or to change your mail settings:
> > http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
> >
> 
> 
>