One thing struck me here with your description...
³and a change to the DNS and we are off and running²
While your DNS records might be changed relatively quickly during an
incident, the change
Itself can take quite a while to trickle down to the end users/clients out
in the cloud.
Any client¹s DNS resolution that has not expired in the cache nor manually
refreshed will
still fail to properly resolve/connect. It doesn¹t usually, but I tell
clients to plan for 48 hours
Estimated time for the change to completely propagate.
I would hate for you to get blindsided with a person hovering over you
asking how much longer
It is going to take before the site is back up and operational. It is
frustrating when you have
Fixed the issue [ problem :-) ] but have to just sit and wait for it to
complete.
There are certainly strategies to mitigate this risk and I do not know if
you maintain your
Own DNS servers or do you work through a hosting provider/domain registrar.
I hope this helps a bit.
Ed
On 5/19/10 2:07 PM, "keith smith" <
klsmith2020@yahoo.com> wrote:
>
>
> Currently we have two servers in our main data center. One serves our
> shopping cart. The other contains quite a bit of content that is data driven
> (reads). The content site is very active. The orders on the shopping cart
> are spread apart by one or two minutes during the busiest part of the day. We
> store a lot of data with each order so most of this is writing. The shopping
> cart is backed up to the server in the other data center. Supposedly if there
> is a problem, a few things need to be done to the backup server in preparation
> to make it live, and a change to the DNS and we are off and running.
>
> The problem I am trying to solve is the other server (content site) is not
> currently backed up automatically.
>
> Another layer of this is these are managed servers. We have an excellent
> relationship with the data center owner and have 24/7 access to him and his
> staff. He manages all three servers and has always done a good job.
>
> I am the one tasked with keeping our sites online 24/7.
>
> I was hoping by configuring two servers, each in a different location, that,
> in the event of one of the data centers being completely severed from the
> Internet that the other server would automatically, without any human
> intervention, take over the full load of the other server and those visiting
> either of our sites would not know there had been an issue.
>
> In a nutshell I am trying to create an automated backup that is a automated
> fail over solution.
>
> I appreciate all your feedback!
>
> ------------------------
> Keith Smith
>
> --- On Wed, 5/19/10, Dan Dubovik <dandubo@gmail.com> wrote:
>>
>> From: Dan Dubovik <dandubo@gmail.com>
>> Subject: Re: load balanced configuration
>> To: "Main PLUG discussion list" <plug-discuss@lists.plug.phoenix.az.us>
>> Date: Wednesday, May 19, 2010, 1:45 PM
>>
>> The question I have, are you trying to actually load balance things? Or just
>> have a remote location that you can fire up with live data at a moments
>> notice? Basically, are you wanting an active/active configuration, or
>> active/passive?
>> active/active across DC's can get kind of hairy depending on what the network
>> looks like. active/passive won't give you any performance gains, but can
>> simplify the configuration, while providing the HA you seem to be after. As
>> Kaia pointed out, what the traffic looks like (reads vs writes) is a
>> consideration. If it is something that users don't write to, and data
>> doesn't have to be replicated across DCs frequently, this further simplifies
>> things.
>>
>> Ultimately, the configuration will depend on what the application and network
>> looks like currently, and what level of redundancy you want / need.
>>
>> -- Dan.
>>
>> On Wed, May 19, 2010 at 1:40 PM, Matt Iavarone <matt.iavarone@gmail.com
>> </mc/compose?to=matt.iavarone@gmail.com> > wrote:
>>> I think the original question was around stateless load balancing, not
>>> clustering. Cross DC clustering is a headache, but HA web sites aren't
>>> exactly terchnical challenges these days.
>>>> On May 19, 2010 4:33 PM, "Alex Dean" <alex@crackpot.org
>>>> </mc/compose?to=alex@crackpot.org> > wrote:
>>>>
>>>>
>>>> On May 19, 2010, at 2:47 PM, keith smith wrote:
>>>>
>>>>> >
>>>>> >
>>>>> > Hi Plug,
>>>>> >
>>>>> > I am considering combining the ... You're entering a world of pain. :)
>>>>
>>>> HA is cool, but is no panacea. If you haven't actually experienced
>>>> downtime due to your server crashing or your datacenter losing
>>>> connectivity, I recommend thinking long and hard about it. Don't solve a
>>>> problem you don't have. The downtime created from unneeded failovers will
>>>> likely exceed the actual/real downtime caused by either a server or
>>>> datacenter being offline. Managing the cluster itself (as distinct from
>>>> the services provided by the cluster) needs to be accounted for as an
>>>> expense/responsibility.
>>>>
>>>> I don't want to sound overly pessimistic. I've set up quite a few HA
>>>> clusters, and actually enjoy it most of the time. But it WILL cause you
>>>> headaches in the middle of the night which you wouldn't have had if you
>>>> only had a single server.
>>>>
>>>> Leave yourself lots of time to set up a development/test cluster, and abuse
>>>> it in many ways. Pull out network cables, kill the switch, yank out power
>>>> cables, etc. Do this with real hardware, not VMs.
>>>>
>>>> When the cluster nodes lose contact with each other, both will decide to
>>>> become primary. This is a split brain. This can happen when the switch
>>>> in-between them gets busy and starts dropping pings. Now, you can always
>>>> recover from such things. I'm just recommending you become very familiar
>>>> with these issues before going live with this setup.
>>>>
>>>> http://clusterlabs.org/wiki/Main_Page
>>>> http://people.linbit.com/~florian/heartbeat-users-guide/
>>>> <http://people.linbit.com/%7Eflorian/heartbeat-users-guide/>
>>>>
>>>> Let me/us know if you have specific questions once you start setting things
>>>> up. Good luck!
>>>>
>>>> alex
>>>> ---------------------------------------------------
>>>> PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
>>>> </mc/compose?to=PLUG-discuss@lists.plug.phoenix.az.us>
>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>>>
>>> ---------------------------------------------------
>>> PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
>>> </mc/compose?to=PLUG-discuss@lists.plug.phoenix.az.us>
>>> To subscribe, unsubscribe, or to change your mail settings:
>>> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>>
>>
>> -----Inline Attachment Follows-----
>>
>> ---------------------------------------------------
>> PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
>> </mc/compose?to=PLUG-discuss@lists.plug.phoenix.az.us>
>> To subscribe, unsubscribe, or to change your mail settings:
>> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>
>
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
> To subscribe, unsubscribe, or to change your mail settings:
> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
---------------------------------------------------
PLUG-discuss mailing list -
PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
(text/plain)