load balanced configuration

Wed May 19 14:07:40 MST 2010

Currently we have two servers in our main data center.  One serves our shopping cart.  The other contains quite a bit of content that is data driven (reads).  The content site is very active.  The orders on the shopping cart are spread apart by one or two minutes during the busiest part of the day.  We store a lot of data with each order so most of this is writing. The shopping cart is backed up to the server in the other data center.  Supposedly if there is a problem, a few things need to be done to the backup server in preparation to make it live, and a change to the DNS and we are off and running. 

The problem I am trying to solve is the other server (content site) is not currently backed up automatically.

Another layer of this is these are managed servers.  We have an excellent relationship with the data center owner and have 24/7 access to him and his staff.  He manages all three servers and has always done a good job.  

I am the one tasked with keeping our sites online 24/7.

I was hoping by configuring two servers, each in a different location, that, in the event of one of the data centers being completely severed from the Internet that the other server would automatically, without any human intervention, take over the full load of the other server and those visiting either of our sites would not know there had been an issue.

In a nutshell I am trying to create an automated backup that is a automated fail over solution.

I appreciate all your feedback!

------------------------

Keith Smith

--- On Wed, 5/19/10, Dan Dubovik <dandubo at gmail.com> wrote:

From: Dan Dubovik <dandubo at gmail.com>
Subject: Re: load balanced configuration
To: "Main PLUG discussion list" <plug-discuss at lists.plug.phoenix.az.us>
Date: Wednesday, May 19, 2010, 1:45 PM

The question I have, are you trying to actually load balance things? Or just have a remote location that you can fire up with live data at a moments notice?  Basically, are you wanting an active/active configuration, or active/passive?

active/active across DC's can get kind of hairy depending on what the network looks like.  active/passive won't give you any performance gains, but can simplify the configuration, while providing the HA you seem to be after.  As Kaia pointed out, what the traffic looks like (reads vs writes) is a consideration.  If it is something that users don't write to, and data doesn't have to be replicated across DCs frequently, this further simplifies things.

Ultimately, the configuration will depend on what the application and network looks like currently, and what level of redundancy you want / need.
-- Dan.

On Wed, May 19, 2010 at 1:40 PM, Matt Iavarone <matt.iavarone at gmail.com> wrote:

I think the original question was around stateless load balancing, not clustering.  Cross DC clustering is a headache, but HA web sites aren't exactly terchnical challenges these days.
On May 19, 2010 4:33 PM, "Alex Dean" <alex at crackpot.org> wrote:

On May 19, 2010, at 2:47 PM, keith smith wrote:

>
>
> Hi Plug,
>
> I am considering combining the ...
You're entering a world of pain. :)

HA is cool, but is no panacea.  If you haven't actually experienced downtime due to your server crashing or your datacenter losing connectivity, I recommend thinking long and hard about it.  Don't solve a problem you don't have.  The downtime created from unneeded failovers will likely exceed the actual/real downtime caused by either a server or datacenter being offline.  Managing the cluster itself (as distinct from the services provided by the cluster) needs to be accounted for as an expense/responsibility.

I don't want to sound overly pessimistic.  I've set up quite a few HA clusters, and actually enjoy it most of the time.  But it WILL cause you headaches in the middle of the night which you wouldn't have had if you only had a single server.

Leave yourself lots of time to set up a development/test cluster, and abuse it in many ways.  Pull out network cables, kill the switch, yank out power cables, etc.  Do this with real hardware, not VMs.

When the cluster nodes lose contact with each other, both will decide to become primary.  This is a split brain.  This can happen when the switch in-between them gets busy and starts dropping pings.  Now, you can always recover from such things.  I'm just recommending you become very familiar with these issues before going live with this setup.

http://clusterlabs.org/wiki/Main_Page

http://people.linbit.com/~florian/heartbeat-users-guide/

Let me/us know if you have specific questions once you start setting things up.  Good luck!

alex

---------------------------------------------------

PLUG-discuss mailing list - PLUG-discuss at lists.plug.phoenix.az.us

To subscribe, unsubscribe, or to change your mail settings:

http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss

---------------------------------------------------

PLUG-discuss mailing list - PLUG-discuss at lists.plug.phoenix.az.us

To subscribe, unsubscribe, or to change your mail settings:

http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss

-----Inline Attachment Follows-----

---------------------------------------------------
PLUG-discuss mailing list - PLUG-discuss at lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.PLUG.phoenix.az.us/pipermail/plug-discuss/attachments/20100519/363c8d21/attachment.htm>