lol.geek
8 stories
·
0 followers

CTRL control

1 Comment

strip-sue-le-fil-650-finalenglish
Kickstarter

Read the whole story
benyafai
182 days ago
reply
Haaaa, been there.
Birmingham UK
Share this story
Delete

Dilbert 2015-12-24

1 Share
Read the whole story
benyafai
453 days ago
reply
Birmingham UK
Share this story
Delete

When I install a software

1 Comment and 4 Shares

Read the whole story
benyafai
749 days ago
reply
Birmingham UK
Share this story
Delete
1 public comment
lardissone
763 days ago
reply
100% verdad!
Pergamino, Argentina

The Relative Cost of Bandwidth Around the World

6 Comments and 16 Shares

Over the last few months, there’s been increased attention on networks and how they interconnect. CloudFlare runs a large network that interconnects with many others around the world. From our vantage point, we have incredible visibility into global network operations. Given our unique situation, we thought it might be useful to explain how networks operate, and the relative costs of Internet connectivity in different parts of the world.

A Connected Network

The Internet is a vast network made up of a collection of smaller networks. The networks that make up the Internet are connected in two main ways. Networks can connect with each other directly, in which case they are said to be “peered”, or they can connect via an intermediary network known as a “transit provider”.

At the core of the Internet are a handful of very large transit providers that all peer with one another. This group of approximately twelve companies are known as Tier 1 network providers. Whether directly or indirectly, every ISP (Internet Service Provider) around the world connects with one of these Tier 1 providers. And, since the Tier 1 providers are all interconnected themselves, from any point on the network you should be able to reach any other point. That's what makes the Internet the Internet: it’s a huge group of networks that are all interconnected.

Paying to Connect

To be a part of the Internet, CloudFlare buys bandwidth, known as transit, from a number of different providers. The rate we pay for this bandwidth varies from region to region around the world. In some cases we buy from a Tier 1 provider. In other cases, we buy from regional transit providers that either peer with the networks we need to reach directly (bypassing any Tier 1), or interconnect themselves with other transit providers.

CloudFlare buys transit wholesale and on the basis of the capacity we use in any given month. Unlike some cloud services like Amazon Web Services (AWS) or traditional CDNs that bill for individual bits delivered across a network (called "stock"), we pay for a maximum utilization for a period of time (called "flow"). Typically, we pay based on the maximum number of megabits per second we use during a month on any given provider.

Traffic levels across CloudFlare's global network over the last 3 months. Each color represents one of our 28 data centers.

Most transit agreements bill the 95th percentile of utilization in any given month. That means you throw out approximately 36 not-necessarily-contiguous hours worth of peak utilization when calculating usage for the month. Legend has it that in its early days, Google used to take advantage of these contracts by using very little bandwidth for most of the month and then ship its indexes between data centers, a very high bandwidth operation, during one 24-hour period. A clever, if undoubtedly short-lived, strategy to avoid high bandwidth bills.

Another subtlety is that when you buy transit wholesale you typically only pay for traffic coming in (“ingress") ("ingress") or traffic going out (“egress”) ("egress") of your network, not both. Generally you pay which ever one is greater.

CloudFlare is a caching proxy so egress (out) typically exceeds ingress (in), usually by around 4-5x. Our bandwidth bill is therefore calculated on egress so we don't pay for ingress. This is part of the reason we don't charge extra when a site on our network comes under a DDoS attack. An attack increases our ingress but, unless the attack is very large, our ingress traffic will still not exceed egress, and therefore doesn’t increase our bandwidth bill.

Peering

While we pay for transit, peering directly with other providers is typically free — with some notable exceptions recently highlighted by Netflix. In CloudFlare's case, unlike Netflix, at this time, all our peering is currently "settlement free," meaning we don't pay for it. Therefore, the more we peer the less we pay for bandwidth. Peering also typically increases performance by cutting out intermediaries that may add latency. In general, peering is a good thing.

The chart above shows how CloudFlare has increased the number of networks we peer with over the last three months (both over IPv4 and IPv6). Currently, we peer around 45% of our total traffic globally (depending on the time of day), across nearly 3,000 different peering sessions. The chart below shows the split between peering and transit and how it's improved over the last three months as we’ve added more peers.

North America

We don't disclose exactly what we pay for transit, but I can give you a relative sense of regional differences. To start, let's assume as a benchmark in North America you'd pay a blended average across all the transit providers of $10/Mbps (megabit per second per month). In reality, we pay less than that, but it can serve as a benchmark, and keep the numbers round as we compare regions. If you assume that benchmark, for every 1,000Mbps (1Gbps) you'd pay $10,000/month (again, acknowledge that’s higher than reality, it’s just an illustrative benchmark and keeps the numbers round, bear with me).

While that benchmark establishes the transit price, the effective price for bandwidth in the region is the blended price of transit ($10/Mbps) and peering ($0/Mbps). Every byte delivered over peering is a would-be transit byte that doesn't need to be paid for. While North America has some of the lowest transit pricing in the world, it also has below average rates of peering. The chart below shows the split between peering and transit in the region. While it's gotten better over the last three months, North America still lags behind every other region in the world in terms of peering.. peering.

While we peer nearly 40% of traffic globally, we only peer around 20-25% in North America. Assuming the price of transit is the benchmark $10/Mbps in North America without peering, with peering it is effectively $8/Mbps. Based only on bandwidth costs, that makes it the second least expensive region in the world to provide an Internet service like CloudFlare. So what's the least expensive?

Europe

Europe's transit pricing roughly mirrors North America's so, again, assume a benchmark of $10/Mbps. While transit is priced similarly to North America, in Europe there is a significantly higher rate of peering. CloudFlare peers 50-55% of traffic in the region, making the effective bandwidth price $5/Mbps. Because of the high rate of peering and the low transit costs, Europe is the least expensive region in the world for bandwidth.

The higher rate of peering is due in part to the organization of the region's “peering exchanges”. A peering exchange is a service where networks can pay a fee to join, and then easily exchange traffic between each other without having to run individual cables between each others' routers. Networks connect to a peering exchange, run a single cable, and then can connect to many other networks. Since using a port on a router has a cost (routers cost money, have a finite number of ports, and a port used for one network cannot be used for another), and since data centers typically charge a monthly fee for running a cable between two different customers (known as a "cross connect"), connecting to one service, using one port and one cable, and then being able to connect to many networks can be very cost effective.

The value of an exchange depends on the number of networks that are a part of it. The Amsterdam Internet Exchange (AMS-IX), Frankfurt Internet Exchange (DE-CIX), and the London Internet Exchange (LINX) are three of the largest exchanges in the world. (Note: these links point to PeeringDB.com which provides information on peering between networks. You'll need to use the username/password guest/guest in order to login.)

In Europe, and most other regions outside North America, these and other exchanges are generally run as non-profit collectives set up to benefit their member networks. In North America, while there are Internet exchanges, they are typically run by for-profit companies. The largest of these for-profit exchanges in North America are run by Equinix, a data center company, which uses exchanges in its facilities to increase the value of locating equipment there. Since they are run with a profit motive, pricing to join North American exchanges is typically higher than exchanges in the rest of the world.

CloudFlare is a member of many of Equinix's exchanges, but, overall, fewer networks connect with Equinix compared with Europe's exchanges (compare, for instance, Equinix Ashburn, which is their most popular exchange with about 400 networks connected, versus 1,200 networks connected to AMS-IX). In North America the combination of relatively cheap transit, and relatively expensive exchanges lowers the value of joining an exchange. With less networks joining exchanges, there are fewer opportunities for networks to easily peer. The corollary is that in Europe transit is also cheap but peering is very easy, making the effective price of bandwidth in the region the lowest in the world.

Asia

Asia’s peering rates are similar to Europe. Like in Europe, CloudFlare peers 50-55% of traffic in Asia. However, transit pricing is significantly more expensive. Compared with the benchmark of $10/Mbps in North America and Europe, Asia's transit pricing is approximately 7x as expensive ($70/Mbps, based on the benchmark). When peering is taken into account, however, the effective price of bandwidth in the region is $32/Mbps.

There are three primary reasons transit is so much more expensive in Asia. First, there is less competition, and a greater number of large monopoly providers. Second, the market for Internet services is less mature. And finally, if you look at a map of Asia you’ll see a lot of one thing: water. Running undersea cabling is more expensive than running fiber optic cable across land so transit pricing offsets the cost of the infrastructure to move bytes.

Latin America

Latin America is CloudFlare's newest region. When we opened our first data center in Valparaíso, Chile, we delivered 100 percent of our traffic over transit, which you can see from the graph above. To peer traffic in Latin America you need to either be in a "carrier neutral" data center — which means multiple network operators come together in a single building where they can directly plug into each other's routers — or you need to be able to reach an Internet exchange. Both are in short supply in much of Latin America.

The country with the most robust peering ecosystem is Brazil, which also happens to be the largest country and largest source of traffic in the region. You can see that as we brought our São Paulo, Brazil data center online about two months ago we increased our peering in the region significantly. We've also worked out special arrangements with ISPs in Latin America to set up facilities directly in their data centers and peer with their networks, which is what we did in Medellín, Colombia.

While today our peering ratio in Latin America is the best of anywhere in the world at approximately 60 percent, the region's transit pricing is 8x ($80/Mbps) the benchmark of North America and Europe. That means the effective bandwidth pricing in the region is $32/Mbps, or approximately the same as Asia.

Australia

Australia is the most expensive region in which we operate, but for an interesting reason. We peer with virtually every ISP in the region except one: Telstra. Telstra, which controls approximately 50% of the market, and was traditionally the monopoly telecom provider, charges some of the highest transit pricing in the world — 20x the benchmark ($200/Mbps). Given that we are able to peer approximately half of our traffic, the effective bandwidth benchmark price is $100/Mbps.

To give you some sense of how out-of-whack Australia is, at CloudFlare we pay about as much every month for bandwidth to serve all of Europe as we do to for Australia. That’s in spite of the fact that approximately 33x the number of people live in Europe (750 million) versus Australia (22 million).

If Australians wonder why Internet and many other services are more expensive in their country than anywhere else in the world they need only look to Telstra. What's interesting is that Telstra maintains their high pricing even if only delivering traffic inside the country. Given that Australia is one large land mass with relatively concentrated population centers, it's difficult to justify the pricing based on anything other than Telstra's market power. In regions like North America where there is increasing consolidation of networks, Australia's experience with Telstra provides a cautionary tale.

Conclusion

The chart above shows the relative cost of bandwidth assuming a benchmark transit cost of $10/Megabits per second (Mbps) per month (which we know is higher than actual pricing, it’s just a benchmark) in North America and Europe.

While we keep our pricing at CloudFlare straight forward, charging a flat rate regardless of where traffic is delivered around the world, actual bandwidth prices vary dramatically between regions. We’ll continue to work to decrease our transit pricing, and increasing our peering in order to offer the best possible service at the lowest possible price. In the meantime, if you’re an ISP who wants to offer better connectivity to the increasing portion of the Internet behind CloudFlare’s network, we have an open policy and are always happy to peer.

Read the whole story
benyafai
940 days ago
reply
Birmingham UK
popular
940 days ago
reply
Share this story
Delete
6 public comments
graydon
935 days ago
reply
Monopolies in telecoms are Very Problematic.
tedder
940 days ago
reply
I've tried to explain "transit" and "peering" to people wrt the Netflix thing. This is great.
Uranus
futurile
940 days ago
reply
Really informative post on how the network-of-networks (Inter-net) really functions and some of the prices involved.

Peering is very hard to get for small/new providers - it's one of the ways the big boys protect themselves. Just think what happens if you can also get the sites providing the content to pay ... *cough* net neutrality.
London
silviapfeiffer
940 days ago
reply
Wow, telstra!
Sydney, Australia
BLueSS
940 days ago
reply
Dang, Australia.
acdha
941 days ago
reply
“To give you some sense of how out-of-whack Australia is, at CloudFlare we pay about as much every month for bandwidth to serve all of Europe as we do to for Australia. That’s in spite of the fact that approximately 33x the number of people live in Europe (750 million) versus Australia (22 million).”

“In regions like North America where there is increasing consolidation of networks, Australia's experience with Telstra provides a cautionary tale.”
Washington, DC

Free Speech

22 Comments and 74 Shares
I can't remember where I heard this, but someone once said that defending a position by citing free speech is sort of the ultimate concession; you're saying that the most compelling thing you can say for your position is that it's not literally illegal to express.
Read the whole story
popular
1066 days ago
reply
benyafai
1069 days ago
reply
Birmingham UK
Share this story
Delete
21 public comments
pavlov02
1059 days ago
reply
Just seems like common sense to me but that seems in short supply.
merlinblack
1063 days ago
reply
Not applicable in Canada but a good lesson I'm discourse.
ÜT: 53.542319,-113.494597
Romanikque
1064 days ago
reply
Alt text is better than the comic for this one...
Baltimore, MD
tewhalen
1070 days ago
reply
Wikipedia: "The Citizens' Councils used economic tactics against African Americans whom they considered as supportive of desegregation and voting rights, or for belonging to the NAACP; the tactics included 'calling in' their mortgages, denying loans and business credit, and boycotting black-owned businesses. In some cities, the Councils published lists of names of NAACP supporters and signers of anti-segregation petitions in local newspapers in order to encourage economic retaliation. For instance, in Yazoo City, Mississippi in 1955, the Citizens' Council arranged for the names of 53 signers of a petition for school integration to appear in a local paper. Soon afterward, the petitioners lost their jobs and had their credit cut off." -- Apparently, no free speech rights were violated.
chicago, il
grammargirl
1071 days ago
reply
Yup.
Brooklyn, NY
stefanetal
1071 days ago
reply
This strikes me as an 'argument from definition'. But the definition itself is contested. Lot of rights don't work this way, for instance there are non-retaliation laws asociated with many rights (especially in labor law -- say the right to marry includes the right, for the most part, not to get fired for getting married).
Northern Virginia
tewhalen
1070 days ago
Like, remember this comic when your supervisor shows up to your cubicle and asks you to donate to the "Conservative Victory Fund PAC" or lose your job. At least you'll have the comfort of knowing your free speech rights weren't violated.
chrisamico
1071 days ago
reply
I'd love to post this at the end of every news site's comments policy.
Boston, MA
diannemharris
1071 days ago
reply
I have to save this for future postings, everywhere
satadru
1071 days ago
reply
It's dawning on me that wikipedia needs Tl;DR links pointing to the relevant xkcd pages.
New York, NY
ChrisDL
1071 days ago
reply
You haven't lived until you've shared an XKCD online
New York
sfringer
1071 days ago
reply
Couldn't be better stated on free speech...
North Carolina USA
neilcar
1071 days ago
reply
Eventually, XKCD will be the answer for every ridiculous argument.
Charlotte, North Carolina
karmakaze
1071 days ago
reply
Alt text: I can't remember where I heard this, but someone once said that defending a position by citing free speech is sort of the ultimate concession; you're saying that the most compelling thing you can say for your position is that it's not literally illegal to express.
07974
darastar
1071 days ago
reply
THIS X1000
ktgeek
1071 days ago
reply
Yet another xkcd that will be reposted and reposted until the sun burns out.
Bartlett, IL
Michdevilish
1071 days ago
reply
Free to leave
Canada
JayM
1071 days ago
reply
.
Atlanta, GA
Askew
1071 days ago
...
wreichard
1071 days ago
reply
+50,000
Earth
aaronwe
1071 days ago
reply
There should be a "BUT FREE SPEECH!" corollary to Godwin's Law.
Denver
mindspillage
1071 days ago
reply
This was basically designed to be passive-aggressively linked to in mailing lists/forums/IRC...
Mountain View, California
stavrosg
1071 days ago
I can't count how many times this would have been useful in the past...
sulrich
1063 days ago
and for that reason it's getting an expansion snippet.
jtgrimes
1071 days ago
reply
Alt text: I can't remember where I heard this, but someone once said that defending a position by citing free speech is sort of the ultimate concession; you're saying that the most compelling thing you can say for your position is that it's not literally illegal to express.
Oakland, CA

03/19/14 PHD comic: 'Cosmic Inflation Explained'

5 Comments and 19 Shares
Piled Higher & Deeperby Jorge Cham
www.phdcomics.com
Click on the title below to read the comic
title: "Cosmic Inflation Explained" - originally published 3/19/2014

For the latest news in PHD Comics, CLICK HERE!

Read the whole story
benyafai
1096 days ago
reply
Birmingham UK
popular
1096 days ago
reply
Share this story
Delete
4 public comments
amijangos
1094 days ago
reply
Intersting "Let there be light" was first :)
Columbus, Indiana
grammargirl
1095 days ago
reply
So well done.
Brooklyn, NY
icepotato
1096 days ago
reply
PhD comics explains cosmic inflation!
Seattle WA
iaravps
1096 days ago
reply
I actually felt goosebumps by the end... "We were light once"
Rio de Janeiro, Brasil
Next Page of Stories