Large Infrastructure question

[Eric Cope]

I'm not sure if its what you are looking for, but I read this on Hacker
News the other day:
http://www.scalescale.com/rolling-your-own-cdn-build-a-3-continent-cdn-for-25-in-1-hour/

Eric


On Thu, Aug 7, 2014 at 8:38 PM, Joseph Sinclair <plug-discussion at stcaz.net>
wrote:

> In reference to your final sentence, you're looking for the kind of
> services a CDN provides.
> (e.g. geographic routing, and rapid scale).  Something like one of the
> following combinations may offer what you need (using the technologies
> others have mentioned already):
>
> AWS with Amazon CloudFront (if your content is static)
> AWS or ComputeEngine with LimeLight Networks (for static content it's
> simple, but they can do dynamic, different for each request, as well for a
> higher fee).
> AWS or ComputeEngine with Akamai (same as LimeLight, simple for static or
> they can also do dynamic for higher fees).
>
> AWS or ComputeEngine without CDN, This can be very coarse-grained in that
> requests from a geographic region will (preferentially) go to the
> datacenter in that region.
> So you could differentiate Asia, Europe(EMEA, really), US-East, and
> US-West with the AWS or GCE zones.
>
> Hopefully those suggestions help; there are many other combinations of
> compute and CDN offerings, but those above represent the top two providers
> in each category.
>
> If you needed to go it yourself, you could use something like the geoip
> database (there are a few providers) to match IP to geography.  That's not
> hugely reliable, but it's about as good as you'll get on a global internet
> where people travel and sometimes use things like Tor to hide their origin.
> If you're on mobile, why not just tag the request with location from the
> mobile device?  That would be much more reliable than any of the other
> options.
>
> If you're needing very precise control, then you could use the mobile
> location information in a simple router service (something like NGinx or
> similar with a basic region-to-server mapping) to redirect the request to
> the correct locality server.
>
> If you're looking for extremely small (neighborhood or smaller) areas and
> it's a mobile app, there are also geofencing services (similar to Android's
> built-in services, see
> http://developer.android.com/training/location/geofencing.html) that
> identify fairly precise location and help serve different content based on
> that.
>
> Hopefully one of those options helps point you in the direction of what
> you need.
>
> On 08/06/2014 11:17 PM, David Schwartz wrote:
> > Here�s something interesting for the infrastructure geeks on the list ...
> >
> > How would you approach setting up a service that had to sink around, oh
> � say � 10-20 million small HTTP POST requests per minute throughout the
> day, from sources geographically distributed around the country?
> >
> > To do development and get the logic working, a small server is
> sufficient. But it needs to scale quickly once it�s launched.
> >
> > There will be a high degree of geo-locality, so servers could be set up
> to handle requests from different geographic areas.  HTTP requests from a
> given area would be routed to whatever server is dedicated for that area. I
> guess their IP address could be used for that purpose?
> >
> > (How granular is the location data for IP addresses on mobile devices?
> Are they reliable? We could add a location geotag to the packet headers if
> that would help.)
> >
> > Note that the servers don�t need to be physically LOCATED in the area;
> rather, they're dedicated to SERVING a well-defined geographic area.
> >
> > There�s no need for cross-talk, either. That is, there�s no need for a
> server serving, say, the LA area to cross-post with one in San Diego,
> except in a very small overlapping area which is easy to address.
> >
> > Can this sort of routing be done with a DNS service?  (eg.,
> DNSMadeEasy.com is one I�m familiar with)
> >
> > Or is something more massive needed?
> >
> > Also note that this would be an automated service. It has a very steady
> stream of small incoming packets, peaking at various times of the day, with
> limited responses. No ads, no graphics, no user interactions at all.
> >
> > I know there are infrastructure services in place to handle this kind of
> thing, like what Amazon offers, and others. I�m looking for any specific
> pointers to services that might fit this use case profile.
> >
> > -David
> >
> >
> >
> > ---------------------------------------------------
> > PLUG-discuss mailing list - PLUG-discuss at lists.phxlinux.org
> > To subscribe, unsubscribe, or to change your mail settings:
> > http://lists.phxlinux.org/mailman/listinfo/plug-discuss
> >
>
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss at lists.phxlinux.org
> To subscribe, unsubscribe, or to change your mail settings:
> http://lists.phxlinux.org/mailman/listinfo/plug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.phxlinux.org/pipermail/plug-discuss/attachments/20140807/558f1508/attachment.html>