Comcast's Approach To Congestion

[Ignitionnet]

Little something form the largest cableco in the world:

http://downloads.comcast.net/docs/Comcast-IETF-P2Pi-20080528.pdf

http://www.comcast.net/terms/network/

I'll summarise differences between this and STM

There is no 'throttling' just priority, customers who are using heavily will have a lower priority than less heavy users.

Unless the network ports are nearing congestion there is no prioritisation at all. Unless the network actually needs the bandwidth control to avoid bandwidth issues it doesn't happen.

In addition the following being done to assist with P2P usage:

– Tracker optimizations (optimizing & localizing P2P flows)
– Caching
– P2P client optimizations

This strikes me as a better approach. The only time that any prioritisation happens is when the network is nearing congestion. This seems to me to be far closer to 'preserving the customer experience' than the VM approach of throttling you down whether your area is congested or not and whether you are affecting other customers or not.

Also I like the idea of just prioritising traffic, so that customers who are heavily using data can continue to use whatever bandwidth is left over from customers using less. To me anyway that is preserving the experience of lighter users while giving heavier ones whatever resources are left. If the bandwidth is there why not use it?

Just for the sake of inflicting pain on us subs / soon to be subs

A sub with 384Kbps upstream will go to 1Mbps
A sub with 768Kbps upstream will go to 2Mbps

Oh for 2Mbps on cable.

Downside being it takes a bit more than a couple of lines of config on a uBR, but sure if Virgin are more interested in the customer experience than saving money you'd think they'd have no problems implementing this somewhat more granular approach :)

Any thoughts?

[Andrewcrawford23]

Simple reason virign then need ot pay for the bandwidht outside there network and at the end of the day that why they stm so they reduce costs

[Frank]

Also remember that this new throttling is a result of an FCC ruling against Comcast and a large fine imposed for its previous secret throttling methods.

[Ignitionnet]

Also remember that this new throttling is a result of an FCC ruling against Comcast and a large fine imposed for its previous secret throttling methods.

Yep, which is nice.

Talking of which Frank I note you left Rogers, now there was a hardcore company in the field of shafting their customers. Hard caps, overage charges and P2P throttling. :)

[Phormic Acid]

In addition the following being done to assist with P2P usage:

– Tracker optimizations (optimizing & localizing P2P flows)
– Caching
– P2P client optimizationsYou might like to read TorrentFreak’s Uncovering The Dark Side of P4P (http://torrentfreak.com/uncovering-the-dark-side-of-p4p-080824/). While TorrentFreak has a certain bias, the concerns raised about monitoring and control are valid.

Also remember that this new throttling is a result of an FCC ruling against Comcast and a large fine imposed for its previous secret throttling methods.Comcast will not be fined (http://news.cnet.com/8301-13578_3-10021222-38.html?hhTest=1).

[Ignitionnet]

Please could this be merged with http://www.cableforum.co.uk/board/63/33637781-comcast-shows-how-implement-fairer-stm.htm

Caching and 'tracker optimisation' has been happening for a while, P2P flows being redirected to on-net peers to save transit bandwidth. Potentially while P4P makes some intrusion easier it's fairly trivial still for ISPs to poke into P2P flows, especially with the sub-standard encryption that's employed in such clients. Worst case ISPs could even proxy the encryption transactions if they had to.

This is quite an interesting technology actually, I might have to do some reading into it and educate myself a bit :)

[indie1982]

Simple reason virign then need ot pay for the bandwidht outside there network and at the end of the day that why they stm so they reduce costs

But STM doesn't stop me downloading, I still use (legal) torrents, I still share and watch video on YouTube, I still download HD movie trailers etc.

They're still having to pay for the bandwidth as STM doesn't make me think about the way I use bandwidth, it just gets to me slower.

The way we use the internet has changed from what we did 10 years ago, media rich online learning, high bandwidth AV content, purchasing music and films online, software distribution and all the others things we now do.

VM have to evolve with this and increase the bandwidth available at their peering points not STM their customers. How they afford this is up to them, if they can't afford to do it because of the ridiculously low broadband prices they offer people they shouldn't be doing that.

Wholesale IP transit charges have come down in the last 10 years, not gone up, yes we're using more of it but I was still paying £30 a month 10 years ago for a 33kbit/s connection and using less bandwidth. Where has the investment been?

[Ignitionnet]

It saves bandwidth as ISPs pay for 95th percentile usage.

The 95th percentile is a widely used mathematical calculation to evaluate the regular and sustained utilization of a network connection. It is commonly used among all major internet transit and peering networks, as well as datacenters and ISPs for both capacity planning and/or calculating usage. It roughly means ‘if you peak at 20Mbps, your bill will be for 19Mbps' based on a normal web traffic.

Reduce the peak traffic load through STM and you reduce the 95th percentile and therefore the bandwidth bill.

EDIT: On VM's network however and most other ISPs the cost of the transit and peering will not be anywhere near as high as the cost of the access network bandwidth. A few quid a Mbit for transit versus the cost of optics, a CMTS card, fibre and lasers for 38Mbit/s makes the transit not that much of an issue.

[Joxer]

VM have to evolve with this and increase the bandwidth available at their peering points not STM their customers. How they afford this is up to them, if they can't afford to do it because of the ridiculously low broadband prices they offer people they shouldn't be doing that.

From the Comcast document.

Widely understood that you can not build out of a peak network
congestion problem which is largely the result of client software
designed to maximize bulk bandwidth consumption.

And VM are increasing available bandwidth by introducing 50meg connections on DOCSIS 3

Some form of traffic management is required, exactly what form and at what expense is the question. VM's current model sucks and I think Comcast's is certainly an improvement, though the best solution? I don't know. If you make any attempt to prioritse real time traffic, whats to stop someone writing P2P software that uses VOIP protocols (for instance) and pretends to be something it isn't?

[Stuart]

But STM doesn't stop me downloading, I still use (legal) torrents, I still share and watch video on YouTube, I still download HD movie trailers etc.

They're still having to pay for the bandwidth as STM doesn't make me think about the way I use bandwidth, it just gets to me slower.

Bandwidth (at least how VM measure it) is actually the amount of data being carried at ony one time. It is not the total amount of data carried. If you are using (say) 20 Meg, you are using that amount of bandwidth whether you download a 1 meg file or a 1,000 meg file. You are just using it for longer if you are download a larger file.

They aren't trying to stop people downloading (if they were, they'd have hard caps, so your download stops at a certain point, or they'd charge for downloads above this point). What they are trying to do is reduce the strain on the network of people downloading.

I am not saying whether that way is right or wrong though.

The way we use the internet has changed from what we did 10 years ago, media rich online learning, high bandwidth AV content, purchasing music and films online, software distribution and all the others things we now do.

VM have to evolve with this and increase the bandwidth available at their peering points not STM their customers. How they afford this is up to them, if they can't afford to do it because of the ridiculously low broadband prices they offer people they shouldn't be doing that.


They are suffering (now) from a lack of foresight years ago. When I first started using the net in 1994, there were small signs that this sort of stuff (streaming TV, music downloads). Admittedly, nothing specific, but there were signs that broadcasting was coming.

There *is* a technology available that if enabled, would reduce a lot of bandwidth requirements (at least for live streams). Multicasting. The standard way of transmission on the Internet requires a connection from the server to the user, so the server may end up dealing with 10,000 users (and use a lot of bandwidth). This is called Unicasting. Multicasting uses routers along the way to cache and distribute the data. So, if an ISP has (say) 10,000 people watching a live stream, with Unicasting, they'll have 10,000 copies of the same data travelling over their links to the Internet. If they use Multicasting, their own routers will handle distribution of the data, and they'll only have a few copies of the data travelling over their link.

The trouble is, Multicasting only works for live streams, and only works if every network the data travels through is multicast enabled. As such, it is unlikely to happen.


Wholesale IP transit charges have come down in the last 10 years, not gone up, yes we're using more of it but I was still paying £30 a month 10 years ago for a 33kbit/s connection and using less bandwidth. Where has the investment been?

What do you base the assertion that wholesale transit charges have gone down on? The broadband ISP market (in this country anyway) has traditionally run on loans, venture capital, cross subsidy from other products and loss leading products, so the prices we are charged probably bear little resemblance to what it actually costs for any ISP to serve us.

[Ignitionnet]

Multicast wouldn't help with the major congestion on VM's network, the connection between the uBR and the customers though.

Wholesale transit costs have gone down greatly. You can pay less than £10/Mbps/month now without buying in the sort of quantities a multi-million customer ISP would, it's not that many years ago that three times that was an excellent price. Internet bandwidth has never been cheaper.

http://www.dslreports.com/shownews/95203

£2/month per Mbps on a 10GbE port with a 3 year contract in the states. Just a Google search for transit shows under £10/Mbps/month in the UK from resellers of transit.

[dev]

http://www.dslreports.com/shownews/95203

£2/month per Mbps on a 10GbE port with a 3 year contract in the states. Just a Google search for transit shows under £10/Mbps/month in the UK from resellers of transit.

it's a shame that £2/month requires consuming a whole 10GigE port which i dont many users would do :p: and the fact it's cogent would put me off massively, there is a reason they're that cheap.

[Ignitionnet]

it's a shame that £2/month requires consuming a whole 10GigE port which i dont many users would do :p: and the fact it's cogent would put me off massively, there is a reason they're that cheap.

Good reason why the news story called it McBandwidth :)

[Stuart]

£2/month per Mbps on a 10GbE port with a 3 year contract in the states. Just a Google search for transit shows under £10/Mbps/month in the UK from resellers of transit.

Now multiply that £10mps by 20 Meg. Assuming VM get some sort of bulk discount, we can be generous and assume it costs them £100pcm to give you your 20 meg. That's a 1:1 connection (which admittedly, VM don't offer). This is why they use many to 1 contention (they state the average is 20:1), so that cost can be divided up amongst many customers.

Their business model (along with that of most ISPs) is based on the assumption that most of the users will use hardly any of their available bandwidth. From what I have been told, VM take quite heavy losses on their top-tier product, and make up these losses using the lower tiers.

Add in things like iPlayer (or any web based VOD service, even Youtube), and all of a sudden, the people who rarely bothered using their connection to it's full potential start to do so more often. Too many of these people do this, and the business model falls apart.

Unfortunately, it is not really practical for ISPs to offer broadband for the low prices it is offered at now, unless they can subsidise the cost with another product, use STM or Cap the service.

[haydnwalker]

I, personally, don't have a problem with STM as such. Yes, I think the limits could be slightly more generous, but I do think ANY ISPs network needs some sort of traffic shaping on it. ISPs, despite what many users may think, don't have an infinite amount of cash to splash on peering/bandwidth arrangements.

I think though, that VMs pricing structure is slightly skewed... In the way that people on lower tiers are subsidising the higher tiers. My question is WHY? People, if they want faster BB should PAY!

I'm only on 2mb and I pay 18/month right now. So much in my opinion (as opposed to some providers) that I'm considering changing phone/bb to accomodate this. Afterall, I could get 16mb for £10/month (on O2 LLU). I'm not a heavy downloader so I don't worry about STM/Traffic Shaping. Theres no need for the heavy users to moan because they should be paying much more for the service they currently get.

[eth01]

I, personally, don't have a problem with STM as such. Yes, I think the limits could be slightly more generous, but I do think ANY ISPs network needs some sort of traffic shaping on it. ISPs, despite what many users may think, don't have an infinite amount of cash to splash on peering/bandwidth arrangements.

I think though, that VMs pricing structure is slightly skewed... In the way that people on lower tiers are subsidising the higher tiers. My question is WHY? People, if they want faster BB should PAY!

I'm only on 2mb and I pay 18/month right now. So much in my opinion (as opposed to some providers) that I'm considering changing phone/bb to accomodate this. Afterall, I could get 16mb for £10/month (on O2 LLU). I'm not a heavy downloader so I don't worry about STM/Traffic Shaping. Theres no need for the heavy users to moan because they should be paying much more for the service they currently get.

the pricing structure is why STM exists..

[dev]

I, personally, don't have a problem with STM as such. Yes, I think the limits could be slightly more generous, but I do think ANY ISPs network needs some sort of traffic shaping on it. ISPs, despite what many users may think, don't have an infinite amount of cash to splash on peering/bandwidth arrangements.

i dont mind STM simply because it treats all traffic the same and doesn't stop you downloading either

[AbyssUnderground]

i dont mind STM simply because it treats all traffic the same and doesn't stop you downloading either

Agreed. If we must have throttling at all I'd like to know by how much and when. VM provide that info and stand by it in most cases (some cases of STM outside specified hours...)

No STM would be better though, obviously.

[eth01]

Agreed. If we must have throttling at all I'd like to know by how much and when. VM provide that info and stand by it in most cases (some cases of STM outside specified hours...)

No STM would be better though, obviously.

Must be crippling you, no?

[AbyssUnderground]

Must be crippling you, no?

Actually no. My last 2 months usage has been under 75GB each. I often do 3-4 times that.

[LaineY]

I dissaggree with the amount of customers that are phoning up moaning about something and being offered discounts on BBI..

i know someone who is on 20MB BB.. full TV package and Full phone package for 37 quid a month because they threatned to go to Sky

im paying 37 quid a month for 20 MB.. how is that justified?

No wonder their network cant cope with the amount of downloads..

in all honesty.. i work in customer services and sometimes you do have to give customers discounts to keep them.. but i personally think with the storys iv heard and actually saw.. its being done way too often.

If they are wanting or threatening to go to sky...
then yeah you ask why.. and what the problem is...
the answer should then be.. No problem il cancel you're account for u.
Not "oh il give u 20 meg.. .Full TV... Phone .. and a cuddly toy for a 20 quid lol"

meh

[Ignitionnet]

I can't say I'm looking forward to it, usage isn't that high especially considering that I'm on the new WoW expansion beta and have been patching it:

Uptime: 10 days, 10:11:48

Data Transferred (Sent/Received) [GB/GB]: 1.99 / 9.73

But a lot of this is at peak times so quite likely I'll trip STM every so often, even if my usage is less that 30GB/month downstream which is hardly excessive especially for an XL customer. :(

[Mad Ad]

I prefer to look at it the other way. In the days of 56k we used to get ISPs with horrible contention at peak times. As VMs network was approaching similar capacity problems they couldnt just add another dialup server, instead they thought up this great wheeze called STM allowing them to resell every L and XL users peak time bandwidth to 3 other people (and M users b/w to 1 other person) by only incurring the costs of the cisco software.

This was instead of spending on CMTS upgrades to upgrade hardware to give us what we paid for, which follows from the decreasing revenue they receive from BB accounts.

Since hardware upgrades do happen, they chose to keep STM hidden from as many subscribers as possible and omitted it from all advertising so as to not create a landslide before they could actually upgrade the hardware (docsis 3) providing more headroom on both segments allowing them to remove or relax the STM regime.

Kinda the same thing, just from a different perspective. Doesnt make it right tho, I still think their anti-customer stance and the way its hidden from advertising is more offensive than the need to manage a scarce resource, which is obviously their aim in the end, just theyve gone about it in a very negative way.


I would welcome congestion based measures, infact one of my early arguments against STM was why it was installed on UBRs in areas that had no congestion problems, my own is only 22% utilised the last 2 times I asked, why STM me and my neighbours at all? I could run 20 meg day and night and not cause anyone a problem (based on their insulting propaganda about how everyone downloading over a long period would adversely affect other people). We all know why now, its all for the sake of capacity numbers in some stupid spreadsheet.

[dev]

I would welcome congestion based measures, infact one of my early arguments against STM was why it was installed on UBRs in areas that had no congestion problems, my own is only 22% utilised the last 2 times I asked, why STM me and my neighbours at all? I could run 20 meg day and night and not cause anyone a problem (based on their insulting propaganda about how everyone downloading over a long period would adversely affect other people). We all know why now, its all for the sake of capacity numbers in some stupid spreadsheet.

problem with that is, people in the uk like to complain when someone appears to have something better, imagine the nightmare of people phoning and asking why they get STM'd before someone they know?

[Ignitionnet]

problem with that is, people in the uk like to complain when someone appears to have something better, imagine the nightmare of people phoning and asking why they get STM'd before someone they know?

You think we complain less than the US? Have you ever dealt with customers from the States? The US is much more customer satisfaction focussed than we are and customers are far more willing to complain.

Have a read of these... http://www.dslreports.com/forums/4

[Mad Ad]

problem with that is, people in the uk like to complain when someone appears to have something better, imagine the nightmare of people phoning and asking why they get STM'd before someone they know?

I see what you are getting at but at least they could (fairly) say its because of local congestion, and to give everyone a fair share STM was temporarily installed until local upgrade work can be carried out.

Anyway we dont complain anyway near enough in this country, we just whinge and do nothing about it and because of that we get kicked about by anyone and everything with bigger boots than our own. American consumers get right up in arms about it- and god forbid if you ever anger the christian right with some product or service youd know about it.

[Matth]

The basic fact, looking at the Virginmedia side, is that they are bumping headline grabbing speeds higher, without adequate capacity to support them, so that super speed broadband can burst speed a bit higher (if the other end and route is good enough, but the overall throughput is just being stretched even more thinly).

I was a 1Mbit opt-out, preferring the old 300k and generous quota to 1Mbit and having to watch your back for usage, as it turned out, they dropped the tight caps and the opt-out as it was no longer relevant.


I can't see the point of STM punishment stretching beyond peak times, if the object is to control bandwidth, then throttling or biased contention when bandwidth needs to be controlled would be more logical.

[akira]

I prefer to look at it the other way. In the days of 56k we used to get ISPs with horrible contention at peak times. As VMs network was approaching similar capacity problems they couldnt just add another dialup server, instead they thought up this great wheeze called STM allowing them to resell every L and XL users peak time bandwidth to 3 other people (and M users b/w to 1 other person) by only incurring the costs of the cisco software.

This was instead of spending on CMTS upgrades to upgrade hardware to give us what we paid for, which follows from the decreasing revenue they receive from BB accounts.



Well they are actually giving you more than what you pay for. If your on the 20meg with a 20:1 contention, Then your actually paying for a 1meg connection which is burstable to 20meg when theres spare bandwith. With STM they only drop you to 5meg which is a lot more than a 20th of 20meg.

[Jonathan90]

http://www.dslreports.com/shownews/Comcast-250GB-Cap-Goes-Live-October-1-97294

[Noob91]

Virgin is probably one of the best ISP's in the world for the amount you can download a month now! 250gb cap, that's around what Sky have here.

[Hugh]

I didn't think Sky capped?