Large Infrastructure question

[Michael Butash]

Best bet is anycast routing.  BGP peers distributed among the world, all 
advertising the same block of addresses brings natual geographical 
proximity as long as bgp as-path hops are somewhat valid.  Done in a /24 
or larger (2+ for disparity), that becomes your "anycast" subnet, 
advertising to as many peers as you can justify/afford.

This is what dns shops do, as well hosting providers that have to absorb 
large bursts of traffic (ahem, ddos) to distribute among infrastructure 
to dissect and mitigate it.  Likewise you just setup farms of 
reverse-proxy forwarders behind each block, from there you can snat, 
x-forward-for inject the packets along anywhere you need to on back-end 
data center interconnects (or DSR load-balance if frisky).  Put the 
intelligence of the source ip/client at the reverse proxy layer.

Global-dns can only do so much, relying on recursion, but better to 
field the requests first without the dns tricks in anycast ip routing, 
as it's the only thing upstream from dns protocol itself for providers 
to give resiliency and distributing load "globally", naturally.

-mb


On 08/06/2014 11:17 PM, David Schwartz wrote:
> Here’s something interesting for the infrastructure geeks on the list ...
>
> How would you approach setting up a service that had to sink around, oh … say … 10-20 million small HTTP POST requests per minute throughout the day, from sources geographically distributed around the country?
>
> To do development and get the logic working, a small server is sufficient. But it needs to scale quickly once it’s launched.
>
> There will be a high degree of geo-locality, so servers could be set up to handle requests from different geographic areas.  HTTP requests from a given area would be routed to whatever server is dedicated for that area. I guess their IP address could be used for that purpose?
>
> (How granular is the location data for IP addresses on mobile devices? Are they reliable? We could add a location geotag to the packet headers if that would help.)
>
> Note that the servers don’t need to be physically LOCATED in the area; rather, they're dedicated to SERVING a well-defined geographic area.
>
> There’s no need for cross-talk, either. That is, there’s no need for a server serving, say, the LA area to cross-post with one in San Diego, except in a very small overlapping area which is easy to address.
>
> Can this sort of routing be done with a DNS service?  (eg., DNSMadeEasy.com is one I’m familiar with)
>
> Or is something more massive needed?
>
> Also note that this would be an automated service. It has a very steady stream of small incoming packets, peaking at various times of the day, with limited responses. No ads, no graphics, no user interactions at all.
>
> I know there are infrastructure services in place to handle this kind of thing, like what Amazon offers, and others. I’m looking for any specific pointers to services that might fit this use case profile.
>
> -David
>
>
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss at lists.phxlinux.org
> To subscribe, unsubscribe, or to change your mail settings:
> http://lists.phxlinux.org/mailman/listinfo/plug-discuss